Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login
The Wall Street Journal to Close Google Loophole Entirely (digiday.com) similar stories update story
263.0 points by sornars | karma 542 | avg karma 11.78 2017-02-11 02:26:07+00:00 | hide | past | favorite | 345 comments



view as:

So... is this good reason to stop posting WSJ on HN yet?

It might be, if it happens.

Hmmmm. In the FAQ, under the question "Are paywalls ok?", there's this:

>It's ok to post stories from sites with paywalls that have workarounds.

If this loophole is closed, will there be any workarounds? If that's the case, then I wonder if it'll still be OK to post WSJ links to HN.


As of earlier today you could still follow links from WSJ social media, for example their official twitter feeds, and get access to the whole article. Any way to set that up as a link in HN?

For what it's worth, whenever you share an article from the WSJ as a subscriber it seems to generate a custom link that bypasses the paywall for the recipient. I hope they retain this feature because I do enjoy sharing articles with people that might not subscribe.

First of all, I think it's great that journalists can get compensated for their work.

but, strictly speaking as a consumer, I must say their paywall is completely out of touch with the way I want to pay for news consumption. I don't mind paying 5c or even 10c per article, but I'm certainly not going to sign up to pay 120$ for 6 months, if I'm only going to read 1 or 2 articles here and there. but since they've done it this way, I can only conclude that most consumers prefer it that way.


I would consider perhaps $5-$10 a month if the WSJ was going to be my only source of news. If I could trust it to provide me with all I need every morning while I eat my breakfast.

Unfortunately I live in Australia so most stuff in WSJ I don't care about. (And we have the ABC)


Why is a full feed of quality news worth less to you than Netflix?

Netflix provides many times more hours of entertainment than the WSJ could ever hope to.

If you consider news only an entertainment product, that's a fair comparison. If you want to know what's going on in the world, it's not.

Netflix costs too much for what it is. News that you can find anywhere else is certainly not worth much. Yes, depending upon how you wish it to be documented and such, it may be worth it but I don't understand the want to see any more news than the local news and what Google can provide. I'm not against supporting; I just wonder why anyone would spend more than an hour a day reading news from one source.

Netflix is $9 a month here in Australia.

"quality news" is rarely consistent or reliable.

wsj in particular had some rather absurd hit pieces on bernie sanders.


Nobody pays for Netflix; they just use their cousin's/sister's/girlfriend's/etc.'s account.

Not sure if this is a good deal, but take a look: https://myaccount.news.com.au/theaustralian/subscribe?pkgDef...

$16 a month for The Australian and the WSJ for 3 months, then $32 a month. That is a bit steep given we all already pay for http://www.abc.net.au/news/

The Washington Post, one of the three best papers in America (NYT, WSJ, Post), is $50/year if you're a prime subscriber.

I'm only seeing a $144/year option?


https://blendle.com/ seems to meet your needs.

Thanks for the link! I just signed up requesting a beta invite. $0.25 per article, with an instant refund sounds fair. I have wanted access to The Economist, etc., but not enough for a full subscription.

You should see if your local library gives you access to Zinio - I get the Economist (and numerous other publications) for free through ours.

Good tip, but when I was subscribing to the Economist, I found it worth using their app, which was much better, although, not free.

IMHO the Economist has the best iPad app for reading longer news/analysis content I have come across. The simplicity of the navigation and the non-nonsens approach to article viewing is just very good. I don't get why others can't do the same.

The Economist is the only news organ I have consistently found to be worth subscribing to, despite its faults. The stuff in it that you think you're not interested in is almost invariably worth knowing about.

I had a subscription to the Economist for years and was reader for much longer, but the content has gotten much worse and they insist on displaying video ads right in the middle of articles even if you pay for a subscription.

I have Economist subscription and U Block, never seen any video ads.

Why do you think the content has gotten worse?


I feel the content has become less evidence based, less thorough, less anlytic, less principled and less hard hitting than it used to be. There's more purely anecdotal/cultural stuff that I can get anywhere.

They used to ask important questions, investigate them thoroughly and then make the case for a particular conclusion. You could agree or disagree, but now it's often just an exercise in careful fence sitting, in "being sensible", even on matters where taking a principled stand is the obvious thing to do.

That said, I'm not entirely certain if it's me who has changed or The Economist or both.


This is exactly how I want to support journalism & consume news. I don't want to commit to a single source of news, but I'm happy to pay per use.

But this is another layer of intermediation without value add - these people are basically journalist taxing, and isn't this another place where smart folks are effectively deciding what other people want to read?

If there's no value add, then why are people using this service? It's not like it has exclusive access to all those articles.

I really enjoyed Blendle for a while, but I found that I frequently ended up paying for articles on the Blendle platform that I could have accessed for free from the publisher.

The amount you want to pay for news is not anywhere close to the value it is providing you or that they make from ad revenue.

That's not the problem. I might be willing pay a lot for news. What I don't want is 100 different newspaper subscriptions. And to say that you should just pick a favorite newspaper and get everything from there is like demanding you decide what books you want before visiting a library.

That's exactly the problem with the person I replied to. Maybe there's a micropayment solution that someone will crack the nut on after decades of people talking about it. But that solution is definitely not 5¢ an article for WSJ quality news.

My personal opinion is that news orgs should operate as non-profits, since profit motive drives them to editorialize and publish things to get money vs publishing things to inform people about what's going on. I want reporting, I want facts, I don't want entertainment.

You think non-profit "news" sources don't editorialize? Democracy Now doesn't editorialize? State run media doesn't editorialize?

Blendle is a decent attempt at per article payment for journalism.

I tried it for a few months and was happy, but found out I don't have time to read articles in full anyway. I'm sure it will work for someone else though.


> The amount you want to pay for news is not anywhere close to the value it is providing you or that they make from ad revenue.

What amount would be close to the value they make from ad revenue?


I would speculate they make between $5 - $20 CPM and they show 3-5 ads per article view. That puts the revenue between 1.5¢ and 10¢ per article pageview.

Micropayment approaches and payment processor rates vary, but I'd generalize to probably 30% take rate. So assume 3.5¢ to 7¢ payment per article. Do you pay for every article you load? Do you pay again if you load the article twice? What if you don't like the article and don't want to pay? I don't think it's quite as simple as 3.5¢ to 7¢ because no one has really proven this model to work yet.


> What if you don't like the article and don't want to pay?

YOu read the first free paragraph, if you like that you pay, if you dont like the rest of the article you dont get a refund since you have already consumed the article.


A $20 CPM? I'm thinking it's nowhere close. That's exceeding TV level CPMs.

The WSJ is very valuable for enterprise advertising. TV is going to hit a much broader segment of audience, which will include many people who won't care about EnterProse Corp's new widget for Sarbanes-Oxley compliance.

Let's say you're standing in a field strewn with gold bars. A man comes up to you and offers to sell you a gold bar for $400,000. You point out that you are standing among hundreds of gold bars, all of them free for the taking. He points out that it was a lot of hard work extracting gold from thousands of tons of ore, casting it, assaying it, etc.

Just because something cost a lot to make, doesn't mean it's worth a lot. Sorry.


To be fair, we do not exactly have an oversupply of WSJ caliber journalism.

To also be fair, WSJ is hardly high caliber journalism, and since the sale to Murdoch has degraded significantly in quality

I would disagree. Their opinion pages have become more conservative, but their business news and investigative reporting is just as good as it always has been.

I agree. However I do always wonder why conservative views are devalued. I am a paid NY Times digital subscriber and I think the price is robbery however despite being conservative, I do find value in the other viewpoints. It seems like many equate conservative viewpoints as 'not worth supporting.' In my opinion, it's worth it for me to read the other side because I don't necessarily need an echo chamber reinforcing my own bias. Perhaps ironically, I subscribe to The NY Times and not the WSJ.

And therin lies the situation. WSJ is worth very little to the very many. I see it as just another editorialized website. Was it always that way? No. Paper editions of the wall street journal were very good, on occasion. Web editions in the new age of journalism, not even close. No where near worth even trying to go around their paywall.

The beginning of the end for me with the WSJ was when they changed the printed size.

Ah, yes, this is also why the US has no sweat of the brow doctrine, with the prime example being "trying to copyright a phone book".

But I don't live in a world surrounded by gold bars. I live in a world where news comes from Buzzfeed and CNN and Fox News and less scrupulous "news" organizations who are only looking towards the next eyeball. I am surrounded by bars of iron. Your parable simply doesn't match the world I live in.

Consider BBC, Al Jazeera, MIT News, and many many others. I won't say those are all gold, but they aren't lead.

Perhaps that's true for the top 10% most engaged WSJ users.

But, how do you know how much value I'm getting from that news source? I usually get my news from many different sources. And sometimes, I don't even read the entire article. Only a small percentage of the articles are actually of interest to me.


If you're a student, WSJ has a plan for $50/yr.

Always worth checking the school library database first as it might be available for free (well, 'free'... not including tuition..)

I hadn't considered that, but you're right. I was able to find fresh WSJ and Economist for "free" on ProQuest. I still prefer the format of the WSJ site/app, but this could work just as well.

I totally agree with you, it would also make me read the article worth the money spent. As a company they must pay salaries of established staff, that sort of stands at odds with per article revenue. Its like this, if business hires people for at least 6months they ought to make an effort to get those people paid and projecting income from penny articles can be substantially difficult.

I posted this somewhere else on this thread, but I've seen a lot of the "I just want to pay 25c per article" comments. Try https://blendle.com

I'd be happy to buy an issue at a time - I think $.5 would be a fair price given that there is no printing cost and that the advertising is more valuable as it's micro targeted to me. I'd buy an issue to read a particular article or because I want one to read for general amusement.

Subscription feels like monogamy, I've got lots of different things to spend cash on, I don't want to commit at this scale to spending it on things I may not want.


They need to go iTunes way and unbundle payments for individual articles. The past has shown that people will pay for quality contents. Also they need to make the payment UI super efficient. If I have to take out my credit card every time then forget it.

While I agree, it is probably a case of not being in their target market:

- Average age: 40+

- Average household income: 250k+

- Average household net worth: 1.5m+

From http://www.wsjmediakit.com/files/uploads/201410/2016_WSJcom_....


link doesn't work

Here is a working link that contains all the the info parent cites:

http://www.wsjmediakit.com/files/uploads/201410/WSJ.com%20Au...

Also, this is the _actual_ audience not their _target_ audience.


How do you know it's not also the target audience?

I wager their print edition subscriber base is considerably wealthier.

So 15 minutes of entertainment time to you (the time spent consuming a WSJ article) is worth 5 or 10 cents? Imagine if you were compensated at the same rate for your work.

I suggest you try to find a substitute for those entertainment pennies. If you can, and that item is "free", good for you. But I doubt 5 or 10 cents gets you much.


A Netflix subscription is around 30 cents per day.

Not to mention that 15 minutes is a very generous estimate, the median is probably closer to 3-5 minutes. Given that the article isn't written for me, that sounds rather fair for accessing it. A print subscription to the WSJ is $400/yr or $1.30/issue, and if I spend an hour reading it that comes out to 6 to 11 cents for a 3-to-5-minute article.


If my work could be simultaneously consumed by millions of people and they all wanted to pay 5 or 10 cents, I'd be totally ok with that.

That's probably more expensive per article then the print product, which included costs of physical production.

Journalism has always been about advertisers and scale. The problem now is lowered barriers to entry drove up supply, reducing advertising $ available to "real journalism" and entities like nyt and wsj still have vestigial anchors pulling them down further.

Couple that with growing adblocking and it's a bad recipe. How do you put that genie back in the bottle? The net effect has been lowered standards by former standouts in a race to the bottom. I don't see a fix, much less an easy one.


I speed read. Usually I'm done in less than 30-60 seconds. There's so much filler stuff in there, often.

I think this is great news. I'd love to see WSJ ad free and funded with subscriptions.

I'm not sure I would subscribe, but I want to live in a world where it's possible.


I would love to see the NYT completely ad-free. I subscribed for a few months thinking my subscription fee would remove the ads, but that wasn't the case.

I want a good source of news, ad-free, and I'm willing to pay for it. Unfortunately it seems like I'm in the extreme minority.


I'd like that, too. However, subscriptions alone—unless raised significantly—won't cover the costs for the NYT (or other paper). Note that the physical paper always carries ads and each paper is paid for—either the cover price or through a subscription.

Hopefully subscriptions will encourage better quality ads, headlines, and stories. Subscriptions do allow for less click bait.


Financial Times has nearly no ads.

Ms magazine was completely ad-free for many years starting in 1991. They currently accept only 'mission-driven advertisements from primarily non-profit, non-partisan organizations'.

I hope you're not in an extreme minority, because I'm right there with you. I currently subscribe to the NYT, and I'd welcome the opportunity to entirely pay for my content without having to worry about ads. I get around it now by using ABP -- with no guilt, since I'm a subscriber -- but it would be nice to not need such interventions.

They're totally set on the advertising business model. They started displaying ads on the front page of the paper in 2009. If they change anything, they will raise prices and display even more ads.

Sadly for most newspapers, WSJ is in a special position. It has the highest circulation of any paper in the US, and it has a target audience that really doesn't mind paying.

A paywall wouldn't work out well for many others.


I suspect most of the others would be better just posting teaser material on the web and trying to make the rest up in the paper edition.

The Houston Chronicle has a similar dual system. The print newspaper is online but paywalled: houstonchronicle.com the "chron.com" version is mostly buzzfeedish fluff that's free.

So does this mean Google is going to stop indexing the text of WSJ articles?

There is still incentive for WSJ to allow Google to index, they probably just will redirect you to sign up when you click through. Perhaps they'd let Google just index the preview text to prevent using the cache

Doesn't Google punish sites that show different content to Googlebot than they do to a user?

It seems like they should otherwise it would seriously hurt their search quality (unless they did something like personalized search ranking for users who they could determine were signed into those particular walled off sites).

Yes, that's "cloaking".

The old experts exchange rule where they put all the content at the bottom :/

"Perhaps they'd let Google just index the preview text" would take care of that.

Google should absolutely not index, at least not under the current organic and news search. Why should they send traffic to articles that people can't freely view?

I would support some kind of premium news offering from Google where they do index these and you are allowed access through a revenue share subscription made via Google.


> Why should they send traffic to articles that people can't freely view?

what if WSJ paid google to index, but not give out for free? Or, rather, partnered with google so that you can pay for WSJ (or individual articles) via google's payment infrastructure?


I wouldn't be against Google showing the articles clearly marked as non-free. I would however be disappointed if I clicked through a regular link and hit a paywall.

https://support.google.com/news/publisher/answer/40543?hl=en...

According to this, it will need to mark WSJ links differently in Google News.

Interestingly, I tested it and links from Google news still had a "First Click Free", contrary to OP. To test:

1. Go to https://news.google.com/ from a computer that hasn't visited any WSJ articles that day

2. Click on a WSJ link (mine was https://www.wsj.com/articles/daniel-tarullo-federal-reserve-...)

3. See if the whole article is displayed. For me, it was, but when I tried again it only showed a snippet. So it seems like they're only showing the very first click, which does not comply with Google's guidelines above (which require 3 articles a day to be viewable). Anyone else want to offer data points?

Edit: oh, OP says it's only starting Monday.

(Of course, WSJ may be exempt because of direct communication with Google, which I'd expect anyway at their size and at the fact that they weren't banned when they started testing this. But Google should really note any exceptions in their policy.)


Update: WSJ articles that are paywalled are now marked as subscription in google news.

Paywalls have been tightening up recently. The Washington Post now has stronger anti-ad-blocking. So do the Chicago Tribune and the Los Angeles Times. The New York Times is still accessible.

It's a concern that the newspapers which have actual reporters in the field are disappearing behind paywalls.


Actual reporters need to be paid :) The idea of free access to news is relatively recent, since wide-spread web access. Prior to that, people paid for subscriptions, or bought papers at new stands or from boxes. Sure, people might share a few papers at the office or at a restaurant, but having access like we've had for the past 20 years or so is pretty remarkable. I think it's becoming increasingly clear that purely ad-based publishing is encouraging an increase in yellow journalism.

I was under the impression that the subscription fee for newspapers was to cover printing and delivery. And that the ads were how the publishers got paid.

That's true. But it was a bundle that couldn't be separated at the end of the day. So given print ad dollars to digital dimes the subscription fee needs to cover web readership as well.

The idea of free access to news is relatively recent, since wide-spread web access.

Not really. Radio. TV.


Both of which are commercial-supported and transient. Print news is a different beast.

Free print news has existed since 2002 in France :

https://en.wikipedia.org/wiki/20_minutes_(France)

In actual print form.

It seems even older in the US : https://en.wikipedia.org/wiki/Free_newspaper

Also the transient nature of tv and radio has been changed by the internet. Most tv channels in France, be they private or state owned, give you the ability to watch older airings whenever you want on their websites. Consuming those media for most young people barely differs from a daily newspaper. You don't have to be there and sit when it airs, you do it when you feel like it.


> Actual reporters need to be paid :) The idea of free access to news is relatively recent, since wide-spread web access. Prior to that, people paid for subscriptions, or bought papers at new stands or from boxes.

True for physical media, aka paper news, but not so much for radio and tv. The internet didn't invent anything new here in giving free access to news, it just provides a written form in addition to video and audio forms.

> I think it's becoming increasingly clear that purely ad-based publishing is encouraging an increase in yellow journalism.

That is true but that form of mediocre journalism can coexist with alternatives. In France we still have paper news that does not depend on ads at all, and that have healthy revenue, like Le Canard Enchaine. Just because mediocre journalism can be accessed for free on the internet does not mean paper news has to die. Le Canard is one of the most profitable newspaper in France, despite requiring a subscription and having zero ads (None, at all, ever. A great tradition they've always maintained). They also have heavy restrictions on their employees for the sake of objective journalism : they cannot play with the stock market nor can they accept gifts or official honors. Just the fact that they don't have any ads at all in their newspaper sets quite a different standard from things like the NYT or WSJ. Those two keep begging for subscribers, but even if you buy their actual paper news, you still have to suffer their ads and wonder how much ad revenue can influence them. Maybe it would be more acceptable and they would gain more subscribers if they became subscriber only in exchange for removing all ads and not depending on ad revenue anymore? As a French Le Canard has proven to me and the rest of us that a newspaper can live without ads at all so I feel a lot less willing to pay for forms of news that compromise with advertisers and pollute our minds.

There is also in France, and accessible through the internet, the availability of state owned media, which is free for the poor segment of the population, and lives through a mandatory tax on the rest. It isn't quite as good as Le Canard, but still a lot higher quality than the average privately owned news. They have written form of news on websites like francetvinfo, radio, tv channels and so on. I'm particularly fond of the tv magazine Envoye Special and its in depth coverage of specific, selected topics during its airings.


> The idea of free access to news is relatively recent

Recent or not, it's an extremely important idea.

That said I agree with others that the real problem might be that no one so far is willing/able to replicate the kiosk model on the web.

Imagine if the only (legal) option to consume a physical newspaper were via subscription. Sounds absurd? It's just as absurd on the web.


I'm curious as to how much their viewership has dropped. I haven't read an LA times article since they banned ad blockers. Not that they care much about viewers that only cost them money.

NY Times digital subscriptions have soared over the past few months and right now they apparently have 1.6 million digital subscribers.[1] Though, revenue is still down as their print advertising income continues to contract.

1 - https://www.nytimes.com/2017/02/02/business/media/new-york-t...


If anyone's looking for another workaround, the article actually hints at the solution: Facebook's l.php redirect that it uses when you click on a link shared through the platform. It functions similarly to the Google redirect, and you don't even need a Facebook account for it to work.

E.g. https://www.facebook.com/l.php?u=https://www.wsj.com/article...


I would give you HN gold if it existed (now we just need a bookmarklet/extension to automate this).

Do we know that this will still work on Monday? Maybe this will also be closed and you'll need a special link?


Bookmarklet for Facebook:

  javascript:window.location="https://m.facebook.com/l.php?u="+encodeURIComponent(window.location.href);
Bookmarklet for Google:

  javascript:window.location="https://www.google.com/url?rct=j&url="+encodeURIComponent(window.location.href);

This is awesome. Just for folks who aren't clear how to create a bookmarklet:

Create a bookmark of any page, by going to a URL such as https://www.wsj.com/articles/SB122878081364889613 (note that only the top part of the page loads) and enter Cmd (or Ctrl) D. Now Save the bookmark to your bookmark bar. Find the bookmark, right click, and replace the WSJ URL with the javascript URL in the parent post. Save and now test by clicking the bookmark in the bookmark bar. After a redirect through Facebook, the full WSJ article should display.


I may be misunderstanding your instructions, but I think the intended use (in Chrome, at least) is to highlight the javascript and drag it to your bookmarks bar. This creates a bookmark that executes the javascript, which you can click to unblock the WSJ article you're currently viewing by redirecting the current page through Facebook.

Another useful bookmarklet for deleting annoying DOM elements:

  javascript:(function(){document.styleSheets[0].addRule(".highlighted_to_remove","background:red !important");var e=function(e){if(e.keyCode==27){i()}};document.addEventListener("keydown",e);var t=function(e){e.stopPropagation();this.classList.add("highlighted_to_remove");return false};var n=function(e){e.stopPropagation();this.classList.remove("highlighted_to_remove");return false};var r=function(e){this.parentNode.removeChild(this);i();e.preventDefault();e.stopPropagation();return false};var i=function(){var i=0;var s=document;while(s=document.body.getElementsByTagName("*").item(i++)){s.removeEventListener("mouseover",t);s.removeEventListener("mouseout",n);s.removeEventListener("click",r);s.classList.remove("highlighted_to_remove")}document.removeEventListener("keydown",e)};var s=0;var o=document;while(o=document.body.getElementsByTagName("*").item(s++)){o.addEventListener("mouseover",t);o.addEventListener("mouseout",n);o.addEventListener("click",r)}})()

Same thing for firefox. And you can right-click the bookmarklet to edit its properties and make a nice name.

How does this work, step by step?

Beautiful.

Instead of buying internet points for someone who figured out a little hack to access journalism, maybe you could instead pay for journalism?

A noble thought. My problem comes not from the fact that I don't want to pay but rather from the fact that it's super annoying.

What if my browser could just say "article will be 25 cents?" and I could say y/n? But instead, I have to sign up for a service, hand over my credit card, blah blah, and now you've wasted enough of my time that it was worth just me stealing it.

I'm pretty convinced it's just aUX thing.


See, I look at this the exact opposite way - if I have to decide whether each individual article is worth paying for, I'm probably not going to read any articles. Its much easier for me to pay $10 a month and know that I can, with zero friction, read whatever article I want from the publications I respect. We're talking about the Wall Street Journal here; odds that they're going to only occasionally publish something I'm interested in are pretty low - I'm either interested in subscribing or I'm not.

For me, subscriptions are a deal breaker. Whenever possible, I avoid signing up for a recurring fee for something I only use occasionally.

For me, the deal breaker is that even with a subscription I am usually saddled with consuming the content the way they want me to and they only want me to consume it that way so they can still advertise to me.

Content providers are fragmented, you can't access it while abroad, you can't access it on additional device, you cannot copy it, you are subjected to fair use limitations, you must latest version of this insecure software to access the content...

The problem is, assuming you read 20 or so online newspapers at least occasionally, this has you keeping $200 a month of subscriptions, of which in most cases you will read a 2 or 3 articles a year because they ended up HN news or some other news aggregator you actually use.

This is even true if you only subscribe to relatively reputable traditional news sources, btw. Here are 12 that individually are worth it and most informed individuals would be served by reading on occasion, but few people would want to have individual subscriptions to them: WSJ, NYT, WaPo, The Economist, The Atlantic, Foreign Policy, Chicago Tribune, SF Chronicle, The Guardian, Le Monde, El País, Die Welt. Nevermind the fact that I do have subscriptions to 2 of those, and they compound the problem by serving annoying video ads even to subscribers.

Edit: A model that I would consider fair though (and the journals might not, I don't know their cost structure) is: 2-5 free articles a month without subscription of any sort and with unobtrusive ads, paid subscription if you read more articles a month from that particular journal. Another option is a "library" subscription model, in which you pay $20 or $30 a month, but get access to a broad catalog of newspapers (about what you would find at a large public library in print) and they get profits based on what you actually read. And yes, I have seen this as a startup idea, Netflix for news, but newspapers never seem to go for it wholeheartedly enough for it to be worthwhile to consumers, though.


Seems to me, as readers, we're less in need of 'Netflix for news' and more in need of either consolidation or decentralization in journalism.

I have much less brand loyalty now than say 10 years ago. I have less respect for both the WSJ and NYT than I did. I don't think their incentives line up with mine anymore, and it leaves me without a news source I trust the way I once trusted both of them.

A larger entity that doesn't need to worry as much about getting me to hit the clickbait would serve me better as a reader (and I'd then be more likely to subscribe). Or, if I could subscribe to individual reporters or small groups of them in some way (such as Dave Pell's style of content delivery, but more availability and let me pay them some how), it would also better serve me as a reader.

Instead we're stuck with a structure where they don't make enough money, and thus continue devaluing their brands, and we as readers don't have an easy way to know who we can trust to respect us as customers (no surprise video ads, unbiased reporting, etc). So I guess I'll stick with using sites like HN to aggregate the news for me, and stop clicking on the WSJ links. The brand no longer signals unbiased quality journalism to me.


That all might be true, but I am still not sure the solution is simply being subscribed to a single very credible newspaper. That still means a single editorial line and no ability to share (links to) articles between people. It seems like an artifact of an old medium of distribution. When you had to get a bunch of pages at your doorstep, it made sense to have a single "provider" that bundled the news and analysis for you and decided what you got.

Now a days what we have is: individual journalists/teams that investigate and produce analysis, aggregators like HN or Reddit who point our which content is interesting/relevant and traditional media outlets which mostly serve to vouchsafe to some degree that the content is factual and credible. We can't get rid of the last component because we end up with no way of distinguishing real news from propaganda, disinformation and innuendo, and also because we haven't found a way to directly pay the first group except through the media orgs. A fully centralized system that certified news "accuracy" is the stuff of Orwellian fiction, though, so that's a non-starter. You could have a federated certification system, I suppose, where articles are published by individual journalists and then groups and institutions vouchsafe for the veracity of stories or the record of the journalist (sort of like TLS CAs or a web of trust of journalist peer review), but you still need to come up with a good way to pay all of the parties involved. That said, in principle, I'd rather pay for each individual article, or a monthly fee for access to nearly-all the articles available, than for a subscription to a very specific collection of articles in the way you do now with news subscriptions.


So check the "automatically pay everything below 30 cents" setting in your browser payment add-on or subscribe to the advertised flatrate shared between publishing houses (like spotify).

  Its much easier for me to pay $10 a month
WSJ is $32.99 (+tax) a month, for digital access. That's a little harder to justify if you only occasionally read their content.

The Brave web browser has this built in, more or less. You give it bitcoin and it does micropayments for content you read. You can set max amounts, etc...

Or, what if they promised to cancel your subscription at the end of the month.

As a good faith guarantee; if they failed to cancel your CC, they would reward you with say a $100.00?

I don't mind paying, but not reoccurring.


There's already a system for selling ad space, based on tracking data and real-time bidding. Why would it be so hard to use the same sort of system for paid access, replacing ad resellers with readers?

That is what Google Contributor once did, but was shut down recently to be completely re-thought from the ground up (and hopefully will be relaunching in the next few months) because:

1. Nobody used it

2. It didn't block ALL ads, just most (based on if you won or lost the bid)

3. There was a LOT of backlash against it because it was google and even though you can opt out of tracking nobody believes them.

4. The old "adblock is free" argument.

It used to be that you'd pay an amount between $1 and $15 per month and it would use that money to "bid" on ad spots like a normal advertiser would. If you won the bid, you'd see a picture, a pattern, or anything else you wanted in that spot. If you lost, you'd see another ad.

The fee split worked identically to a regular ad, so Google would take their cut, the site would take the rest. And any money not spent at the end of the month was refunded back to you (not rolled over, actually put back in your bank account).


Yes, that was an interesting experiment.

But I'm arguing that publishers could simply sell per-article access. As an alternative to subscriptions. I used the current ad-brokerage system as an example, because it's so hugely cumbersome. But I'm not arguing that users should simply bid against advertisers.


I used it, but:

- it was poorly advertised (irony!); I only found out about it from a Google engineer that said, "This is probably the kind of thing you'd pay for"

- US only


Can't reply to taek, but https://blendle.com/ lets you pay per article. There's an HN sign up code somewhere but couldn't find it on my phone

This. Has been my solution for most "major news outlets with paywall" in German for quite some time now. It has a couple of English papers as well (could be better, though).

You can even get your money back when reading accidentally (guess that doesn't work 10 times in a row, but it's a good feature).


Imho, the solution should not involve a middleman. We don't want a single party gatekeeping our news.

I wouldn't want a single cent of my money going to that bottom feeder Rupert Murdoch

Then don't read articles on a site he owns.

It is a small conundrum, I don't particularly like Murdoch either, but his paper is well regarded and many important pieces have appeared there (like op-ed from public figures for example), and I miss out when I can't have access to WSJ articles.

I wish there were a way to buy access to individual articles .


You are aware that you probably consume other valuable goods and services from companies whose CEOs are just as deplorable, but not as well publicized, right?

If you think a product is valuable, but you don't want to pay for it because you don't like the person selling it that makes you a retributive thief. You don't even have ideology to hide behind, you just don't like someone.


Being a thief requires theft. The GP didn't discuss stealing.


You cant get a refund for the time wasted reading drivel and opinion of facts, and how do you know its drivel or spin until you have read it?

Its like with businesses, you buy a product, it turns out to be rubbish, unfit whatever, so you take/send it back and get a refund, but you don't get refunded for your time wasted finding out something is unsuitable. In the mean time, said business has benefited from your money for the period of time its had it, if its put the money to work. Capitalism is the most surreptitious control system going, because if you don't have any money, you cant survive, you cant live, you just become an even lower grade slave to the system.

I don't know if this hack still works, but changing your browser agent to pretend its a Google Spider might also work, if it doesn't, then the online content provider might be using Google's ip address ranges as another metric before allowing unrestricted access to allow the search engine's spider to do its job.

So search engine's want to be the users starting point, an interface to be directed to everything online. I wonder how much revenue a company would make by excluding search engines and sticking to traditional advertising methods to raise customer revenues. Will we see a resurgence in printed papers like we see a resurgence in Vinyl, because one thing a printed piece of paper wont do, is hack your systems, wasting your time even more.


It's working right now

How come this isn't culturally perceived in the same way as not paying for software or movies or music? I've read various reasons why people don't want to pay. That's fine. But then simply don't read it.

> How come this isn't culturally perceived in the same way as not paying for software or movies or music?

Perhaps it is? I've almost never had anyone remark that I was doing something bad when not paying for a software or movie or music.


Well it is bad. You're basically a thief, no matter how you try to justify it to yourself.

Really, you should be ashamed, rather than nonchalantly unconcerned about your behaviour.


You have to care to be ashamed.

Theft != copyright infringement.

Ah, that tired old linguistic argument that completely misses the point.

if copyright infringement is theft, then theft is surely murder?

Illogical. Please show your working.

Ah, that tired old PIRATE-COPIER-MURDERER-TERRORIST argument.

You do realize that copyright protection is an invention to prevent artists from being fucked over by commercial publishers, and was originally not intended to limit people from copying content noncommercially?


An interesting history lesson, but irrelevant to the current discussion.

If I steal your car, I've deprived you of the use of your car. If I copy your program, have I deprived you of the use of your program?

How is that a linguistic argument?


Because you're using an unreasonably narrow definition of theft that doesn't include intangibles.

Tell that to the SCOTUS:

interference with copyright does not easily equate with theft, conversion, or fraud. The infringer of a copyright does not assume physical control over the copyright nor wholly deprive its owner of its use. Infringement implicates a more complex set of property interests than does run-of-the-mill theft, conversion, or fraud.

By the way, the infringer was acquitted, since the court agreed he hadn't "stolen, converted or taken by fraud", despite even making money from his infringing activities.


A counterpoint to this interpretation of the Dowling verdict - http://lawtheories.com/?p=1881:

> This lack of “physical removal,” as required by the statute, meant that Dowling’s infringing copies were not “the sorts of goods whose interstate shipment § 2314 makes criminal.” But this doesn’t mean that Dowling’s acts of infringement in creating the physical records were not themselves acts of theft or conversion.


Unconvincing argument. The court's statement, as I reproduced above, is not about the specifics of the statute, otherwise it would have said "it's theft, just not physical removal". The wording is very clear.

The example of the other case (which, by the way, was only a District Court decision) also seems weak. AFAIK (IANAL), the fact that copyright law preempted state law doesn't mean the two are equal, it just means that both theft and copyright infringement occurred - which is easily seen to be the case, since physical plans were actually removed from a place.


Let me know when your definition makes it into a legal dictionary.

Imagine there is a technology that could copy tangible goods such as cars. Now somebody makes a copy of your car. Your car was not stolen but you may not be happy with the fact that somebody now also has a copy of your personal belongings like a smartphone that was inside the car.

This is where copyright law comes into play which only allows copying the car only under your rules. If somebody doesn't follow the rules that doesn't mean they stole your car but it still means they should receive some form of punishment to prevent future copyright infringement.


You are correct. In fact the distinction between your two examples i.e the car (a tangible good) and the program (an intangible good) is what economists refer to as rivalry [0].

A car is a rivalrous good. A program is a non-rivalrous good.

The main distinction is that the marginal cost of producing a non-rivalrous good is zero.

[0] https://en.wikipedia.org/wiki/Rivalry_(economics)


That's a bit beside the point parent was making: it's true that music piracy is somehow usually not frowned upon, at least based on my anecdotal experience. People don't see it as a thing to explicitly avoid, any more that they would avoid sharing someone's (copyrighted) photo in Facebook.

I expect that when I click a link via google, that I get the same content that google gets. It took years for google to demote expertSexChange and now that WSJ is fully paywalled, they shouldn't get a pass on this.

Let's pretend you developed a piece of software you want to sell. People download it without paying you. What do you do in that case?

Rethink my business model?

it's a logical fallacy to assume that people who downloaded your work would have discovered it and actually paid for it if there was no free option

a lot of artists would PAY to get millions of downloads/views


You mean pirated? It's a logical fallacy to assume that your otherwise non-discovery makes your decision moral.

Let me know when you can pay rent or buy groceries with retweets.

it's unclear if you do not realize thousands of people make their living full-time as promoters on social media. They literally pay rent with money that was paid to them proportionate to their retweets. Or if this was hyperbole to underscore your disagreement with a freemium business model

this seems like an equivocation

> it's a logical fallacy to assume that people who downloaded your work would have discovered it and actually paid for it if there was no free option

it's a "logical fallacy" to assume they wouldn't


I recognize that review comes from two sources, direct through sales and indirect, through discovery. So I try to make sure discovery reaches people with the best material and spend the most money to move the most people, persuading them to pay along the path of least resistance....

...which is exactly what the WSJ has been doing.


Well, hear it now. I'm judging you.

crickets

Also there are still countries where downloading copyrighted materials is not illegal (distributing it usually is, though).

> the Journal also started letting people read for free links that are shared on social media by subscribers and staffers.

Because the publisher explicitly allows it.


No. The publisher wants their subscribers to be able to share the stuff they pay for with other people. If you use that feature to read for free any content they publish, you misuse it. Now, I am very much against any DRM, I don't even really believe that intellectual property should be protected by law, but I believe that content creators have legitimate (but not necessarily legal) right to be paid for their content. (If they want to have legally enforceable income, they need to find different schemes.) If you avoid paying, you're denying them that right. Again, I don't think this should be covered by law, but I also don't think it's fair.

Don't believe IP should be protected? So every single author out there should not get paid? If their content isn't protected, they it's illogical to suggest they should also get paid. Anyone could reproduce that content and redistribute it. That's ludicrous. Film crews ought not be paid? Because if they can't sell their work, then what are they supposed to eat while making movies? Without IP protection, you no longer have new medicines. If the answer is "the government," then the question is -- where does the tax money come from to pay for it? If you wiped out IP law, you'd destroy a significant portion of the economy. The only people that would make any money would be those that control land or raw materials. Innovation would disappear.

Obviously, this is too complex a problem to discuss here, so I'll make only a few points: (1) It isn't really relevant for my point about paying the publishers. If anything, IP protection makes my point stronger. (2) I'm not suggesting to abolish IP protection, because I'm not sure I'm right. That's why I said I believe IP protection is not needed. I know I cannot argue my opinion sufficiently. (3) My biggest issue with IP protection is that since there is no unambiguous definition of what IP is, all IP laws are necessarily too restrictive. Leading e.g. to the current craziness with patents. (4) Film crews out to be paid. There are different models of financing than copyright. Music industry got pretty far with the transition. (5) As for drugs, the governments should announce which drugs are needed, and when companies deliver such drugs, they should get a one-time payment. In present language, the governments would buy the copyright from the companies and release the recipes to public domain. It would be financed from health insurance.

Etc, etc. But again, I know there are holes in my reasoning.


> drugs ... one time payment

That'd have to be an incredibly large one-time payment. And how would we know how much to pay?


> And how would we know how much to pay?

It doesn't seem like an unsolvable problem. The drug companies themselves must have budgets, and the publicly traded ones have a stock price. So someone somewhere must be able to create estimates for this.


The publishers want their subscribers to ADVERTISE the stuff they they pay for.

By definition sharing means giving it away for free.


do you claim to know the exact intent of other people, and are prepared to judge some other people based on sharing your view and adhering to it exactly?

This argument comes up often but it's not convincing. You can't have your cake and eat it too. You can't distribute your content on a public (free) medium and also control how people consume that content. If you want control of how people use a thing, the internet is the worst possible choice.

It's really simple, if you want to put your content behind a paywall, then put it behind a paywall. Don't leave a backdoor open and then complain when people use that door.


Everything on the web used to be free. We got used to that.

It is, music, software and movies are pirated illegally too.

Can you pirate legally? Sounds like a pleonasm.

I'd be happy to pay, under these conditions: 1) payment per article, at prices comparable to aggregate ad income; and 2) payments accepted via Bitcoin, or other methods which can be anonymized.

Then just wait 18 hours for you $0.03 transaction to be confirmed so you can read the article.

No, not that way. Put 100 USD worth in an account, for instant debit.

> 1) payment per article, at prices comparable to aggregate ad income;

Unfortunately, this kind of micropayment will not work for most people and perhaps more importantly will lead to less revenue for the publisher.

Clary Shirky has written about why micropayments [0] is the wrong model for consuming content more than a decade ago.

[0] http://shirky.com/writings/fame_vs_fortune.html


Well, ads are currently placed through a micropayment system. And there's no reason that customers would need to make individual decisions, any more than advertisers do.

You can often loan out movies, music, books and sometimes software from a local library, free of charge. (At least in the UK.) It was also common to loan a friend a CD back when CDs were a thing. Perhaps it is more pervasive now, but I don't think it is fair to say that everybody who consumes media pays for it.

This is the equivalent of reading a magazine in the bookstore and not buying it. It would be different if you walked out the store without paying.

I don't they're the same at all. In your example if you walk out of the store with the magazine the store has to cover the cost of the magazine. If you stay in the store and read it then the content provider didn't get paid for someone consuming their content however the magazine is still there. The store could still sell it.

The l.php script basically allows someone to read the magazine without paying for it. The only entity that loses something in that situation is the content provider.


> The only entity that loses something in that situation is the content provider.

What do they lose?


Why the hell should I pay for poor journalism that is riddled with ads?

There is a reason I subscribe to Netflix but not Hulu.


Hulu has an ad free plan

They really should have been more vocal in announcing its release. I find that a lot of people still believe the only available options are ad-supported.

I had only discovered it when I wanted to resubscribe for other reasons; I probably would have done it earlier if I had known the option was available.


Because there is no Spotify or Netflix for written content.

Are you willing to pay each month

    SPIEGEL:  $  9.40
    FAZ:      $ 44.90
    WSJ:      $ 30.85
    Guardian: $  6.99
    KN:       $ 19.99
    
    -----------------
    Total     $112.13
just to read a handful of articles per newspaper?

I’m sorry, I’m a student, I don’t even have Netflix because that’s too expensive.


Which culture are you talking about? Lots (lots!) of people view and collect media without paying for it.

The reason I posted the workaround is because I feel like the publisher is trying to have their cake and eat it too. In other words, they enjoy the publicity they get from having their articles shared on Google, Facebook, Hacker News, but these sites usually don't want to feature articles that require their users to pay, so WSJ sets up a soft paywall that lets some of these users view the article for free.

But of course that becomes a bit of a blurry moral line. If I start browsing Google News in the hopes of stumbling upon a WSJ article so I can read it for free, is that immoral? What about if I google the headline of an article I'm interested in? Automatically setting my referer to facebook.com whenever I visit wsj.com probably goes too far, but then where do you draw the line?

Anyway, I feel very strongly that either everyone from HN should be able to freely read an article shared here, or no one. If no one can read it, then the link shouldn't be allowed here.


> everyone from HN should be able to freely read an article shared here, or no one. If no one can read it, then the link shouldn't be allowed here.

Then we'll have the laments: oh the ads, the humanity; or, why does all journalism suck?


I'm fine with their paywall... however, they should no longer come up on google news... their rules have been quite clear, and why google is letting WSJ skate on the issue is beyond me.

i would be perfectly fine with this. just give me a way to exclude wsj from my newsfeed and search results without me having to install a trusted browser extension

If they don't want people to read for free, the WSJ should simply not serve it.

We're free to send them any Referer headers we wish... in a request. It's up to them to reply however they wish.


The only downside with this is that facebook gets a small amount of data about your browsing habbits.

You can create a fake account.

It still helps them if it all aggregates under one account.

Then create a fake account every week. On a fresh VPN/Tor connection so it's not tied to your IP. Better use a different browser too so it's not fingerprinted [0]. Make sure you disable those sharing links and the JS that loads them in.

Let's be real, they get data about you in a million ways. It'll all be aggregated somehow, and I wouldn't be surprised if they did aggregate in other ways like IP/browser, and those sharing links definitely leave cookies behind.

[0]: https://panopticlick.eff.org/


Just to read an article from wsj? No thanks.

It's the same data that Google was getting.

Also, I think Facebook (and Twitter, and others) already have a Share button on all WSJ articles that would inform them when you visit that page.


Ah, the WSJ, hotbed of climate change denial. I don't click through to any of their articles anyway. I wish them well with their stupid paywall

Does it work by checking the referrer? Maybe you can workaround by configuring a browser add-on (like RefControl) to just send facebook.com as a referrer while on WSJ.

I believe this was discussed in one of the last threads as being potentially against some dumbass 25 year old law.

Not everyone is american

I've paid for the WSJ for over 15 years. There's nothing wrong with actually paying for good content. That business model doesn't need to go away.

I completely agree. The real problem is pricing.

The difference between absolutely free like most content on the internet and the WSJ standard monthly rate of $33/month is truly enormous. The NYT, which I also feel is priced too high, is $15/month.

Remember Netflix costs $10/month-- that's the value proposition they need to compete against.

These content producers should charge $5/month, with no scummy "special introductory offers" where you just _know_ they're going to raise the price and screw you in a couple months. At that price I would subscribe to everything I read.


If they lower the price they might not get more subscribers. So they are pricing it to maximize revenue.

I can't speak to that-- all I can say with certainty is I personally would subscribe at $5/month.

Same here... while I've appreciated the articles I've clicked through to, I do not feel their content is worth over $20/month to me. I pay for netflix, cable, etc, and don't even use it much, I mostly download for convenience, but I pay for it because I feel guilty if I don't.

If I click on a google search link, I expect to see the same content google does.. for that matter, it's part of google's own rules... WSJ should not get a pass, and google should now deindex WSJ based on those rules.


do other countries have laws?

It's easier to not read Rupert Murdoch owned content.

I prefer Reuters. They seem to have less bias, their articles are free and many stories at other news outlets are bought from Reuters anyway.

There's also AP.org http://hosted.ap.org/dynamic/fronts/RAW?SITE=AP&SECTION=HOME

Many news agencies are just editorializing AP news anyway.


When I realised that, I stopped considering the majority of "news regurgitation" worth my money.

Has any publisher tried out a paywall along the lines of the following?

"Hey, you've spent 29 hours so far this year reading 119 articles. Suggested donation: $62"


The Guardian is doing something similar but without tracking the hours spent. They just say "Hey, we see you spend a lot of time around here. Why don't you help us out a bit so we can stay afloat?"

I feel like that's missing the sine qua non of the idea. You gotta remind people that, like, "You're read the equivalent of five novels here this year!"

I fully predict that the first one that does that will be decried on HN as the NSA. The hordes of privacy cargo cultists will descend upon it like a pack of angry wolves.

Before anyone claims that I'm being derogatory and dismissive of privacy claims, I'm claiming there's a loud minority of people that create a PR problem for companies by complaining about insignificant things.


The Guardian's tactic actually got me to pay for news for the first time in my life. For some reason I'm really turned off to the idea of buying a ~$100 subscription to a single paper, but I'll gladly toss $25 here or there in response to good journalism.

Wikipedia did the same thing for me. They request donations and I used it so often that I felt like I had to.

I value the guardian's work, but I give a lot more money to the FT (despite not particularly agreeing with their politics). Their world coverage is amazing, they actually cover european/middle eastern/asian politics with local reporters and I have never felt so well informed.

They are also a reliable source of Tory puff-pieces, but that doesn't interest me as much.


I am just wondering what would be equivalent to scanning some articles not reading them, conflating two destroys potential value to offer this proposition. I don't often read articles but I want to see if its worth reading. So it is a bit of a hardsell on part of wsj, but they do cater to more of the affluent public so my guess they stand to gain more from playing hardball with people who do have more disposable income than rest of the internet public.

Do you realise the kind of tracking that entails?

Most paywall newspapers already keep track of how many articles you've read this month, and count it down for you; this would just add a timer, too. Which their analytics code is surely already reporting.

"Hey, we kept track of all the things you read here. This month you spent 13 hours looking at clickbait-boobies, 2 on politics and 27 on coverage of divorce cases. Wouldn't it suck if someone else used your machine right now or if we gave this information to tens of third-parties (we already did)?"

I would support that. Might make people more aware of how pervasive tracking is.


It's like I started a conversation about haircuts and you're bringing up dismemberment.

I often open 10-30 tabs with stories I want to read later.. and I leave those tabs sometimes open for a few hours or up to a few days until I have time/interest to read them. How would a paywall deal with that? Or perhaps I am the exception with this type of reading.

Only count times where your tab is visible, and/or the user occasionally scrolls? I'd expect that's a fairly standard feature in tracking JS.

Analytics code solved that problem long ago.

With every single news outlet taking polical position lately - it became intolerable to read or find somewhat objective news source.

I don't mind paying <$10 / mo for clean no BS objective and detailed coverage.


That makes sense. But your pricing is unreasonable. I don't mind paying $10k for a Ferrari.

ferraris are toys where each copy bears an unavoidable cost of production.

news articles have a single upfront cost, but subsequent copies carry near-zero production cost.


Is WSJ not a political news source? They're owned by News Corp and at least in the UK that conglomerate is responsible for the most politicised and polarising media.

yes, wsj is politicized.

I am just wondering if sites can have affiliated material subscriptions like with hn have free clickthrough for 2$ a month or 5$ a month to all sites from where the articles originate from.

I am just wondering if sites can have affiliated material subscriptions like with hn have free clickthrough for 2$ a month or 5$ a month to all sites from where the articles originate from.

Fuck the Journal then. Above everything else, I will never reward their bad behavior by paying them money.

Out of curiosity I searched for the Wall Street Journal subscription page. I selected my country (UK) and then clicked on the subscription offer. I was then taken to a Hong Kong page with no obvious way of changing it back to the UK.

If they want subscribers, they're going to have to do a better job than that.


I agree their pricing page is confusing. I just got information about a discount offer but its not clear how much is the full price after the discount.

It took me a couple times to find it. Go to https://buy.wsj.com/wsjuspresday17 and click the "Act now" link.

$32.99 (plus tax) a month, for digital access.


They are 'striving' to hit 3M subscribers? Sounds like they are trying very hard to go right out of business anyway.

Another way to get around many paywalls is to change your browser's user agent string to 'Googlebot'. I've got a subscription to the Financial Times, but if there's a one-off article somewhere else that I want to read, that's my go-to method for getting through a paywall. I wonder if their fix will close it off too.

Google recommends using reverse DNS lookup to determine if a client claiming to be Googlebot is really from Google – https://support.google.com/webmasters/answer/80553?hl=en

It isn't hard for paywall website operators to use this to block Googlebot fakers. (Rather than doing a reverse DNS lookup on every request, you'd want to maintain a local cache of those results, so you only need to do DNS lookups on new client IPs.)

I believe Cloudflare already does this, and possibly some other CDNs too.


DNS lookups are expensive at server scale. Bot traffic ranges from maybe a quarter to 80% of traffic, though googlebot was only a subset of that.

Cheaper to look for Google ASNs.


> DNS lookups are expensive at server scale.

Even if you use a distributed in-memory cache to store the DNS lookup results? If User-Agent contains "Googlebot", then check for client IP in cache – if cached as "Google" continue, if cached as "Not Google" display error page. If client IP not in cache, do DNS lookup and then update cache – or, you could even let the request continue if there is a cache miss, and do the DNS lookup on another thread. That might let a Googlebot faker get a few requests through, but they'll be blocked very quickly unless they keep on changing IPs (something your average paywall bypasser will find hard.)

You don't need to consult the cache for every request, only for requests with User-Agent contains "Googlebot". (And any other bots you are choosing to let through your paywall, e.g. Bingbot.)

> Cheaper to look for Google ASNs.

How does one do that as an application developer? Do you need to speak to a BGP server?


A local caching DNS server should offer high-speed response. You could build out a tier if necessary. High-capacity, high-speed DNS is not a greenfield.

The genuine Google requests will typically be cached quickly, and given HTTP codes, you could give a nonpermanent error whilst you're adding new entries, so there's that.

The problem is your misses may come from all over. So they're both expensive and (potentially) large.

Even if Googlebot is only a fraction of requests, it's a large fraction, and I've not investigated how many of same are suspect.

For ASN, either the CIDR Report or Routeviews.org offer information. The latter has downloadable tables, in CIDR format, for IP-based lookups. I've long advocated that this be incorporated into routing hardware with reputation data allowing for linespeed determination of traffic acceptability and rate-limiting. Shifts toward VPN and Tor are making this slightly less useful than previously, but it should still be a generally worthwhile option.


My pain at the loss of the Google loophole is tempered by the realization that the WSJ has largely become a propaganda organ of the GOP.

Luckily every other major newspaper has you covered then.

most of them smeared sanders and fawned over clinton, so that's not a guarantee.

I see in the comments there are still some workarounds to this paywall. Good to know. Cheers.

Just a reminder another workaround to the paywall is to you know actually subscribe to the newspaper. That is an option they offer. Obviously not possible for everyone. But if you have the money and want to read and support journalism... WSJ is really far from perfect but it's not the worst newspaper one could be subscribed to.

Also I think their discounted corporate plans program is huge, so depending where you work you may get a discount or it could be free, never hurts to ask.

Also, maybe YC could negotiate some sort of group WSJ discount for Hacker News users above a certain karma count? Just an idea...


I don't think it's smart to pay to read stuff from a group that also gets money from advertisers. You're either the product or the customer, when in doubt you'll be the product.

Real subscription journalism doesn't do advertising.


What 'real subscription journalism' big-name newspapers that don't do advertising can you name? I am honestly struggling to think of any.

Sorry, I'm French, so I can only cite you 2 French ones: Le Canard Enchainé (100 yo, no website) and Mediapart (pure web).

We are talking serious news: the first one discovered that a minister had sent 1500 Jews in a train during WWII, the second one has discovered the minister charged of tax collection had an undeclared bank account in Switzerland.


Thank you for your reply. I can read a little bit of French, so Mediapart is something I'll keep an eye on in the future.

Advertising-free journalism has a far higher price. Always has. Independence isn't cheap.

https://archive.org/stream/commercialismjou00holtuoft#page/1...


Yes, and it needs to be that way, because they need to keep a treasure for the lawyers too.

The larger problem here is that unless they plan to have their Google search results all say "click here to pay" or similar, this entire scheme violates Google's cloaking policy and should result in their being deindexed. This has traditionally been the entire reason that sites offered this workaround - it wasn't out of the goodness of their hearts.

Here's hoping Google actually enforces their policy.


If history is any guide, they absolutely will.

Google doesn't want people having a bad experience on Google which would drive them to consider any other source for search. If a search turns up an article and preview and a click-through doesn't show the same content, Google users will be annoyed, which will drive Google to act.

Google's anti-cloaking policy, while presented as a benefit for users [which it is], is strongly in Google's best interest to enforce as well.


What I don't get is that they could just leave the first two paragraphs and let every see that, then require you to subscribe for more - google would accept that and they would still show up in search, right?

They could, but then only those two paragraphs would be indexed. That would make the content less rich, less relevant, so it wouldn't rank as high.

That's for the normal organic search results. I don't know the policy for showing up in Google news and the carousel. I suspect it wouldn't be allowed.

Edit: You can be subscription-only and be in Google news, but you have to show the same limited snippet to both Google and end users. And you get 'tagged' in the UI as subscription only. https://support.google.com/news/publisher/answer/40543?hl=en


That's what Experts Exchange was doing. Back in the days before Stack Overflow, EE was one of the hot places to go for answers to software engineering problems. It was a free site for many years, then one day they decided to require a subscription to read the answers. Google still indexed the question, so that you would open up a EE link, only to be greeted with "the answer requires a subscription". Google allowed that for a number of years before finally deindexing their results, but it took bloody forever!

If I remember correctly EE had the real answers waaaaaay down on the page. They were banking on people not scrolling.

From memory you could scroll to get the results IF it detected you clicked on a google search result to get there, otherwise the answers were not displayed.

The WSJ is a fantastic source of information. I no longer subscribe since I no longer live in the US but I easily made more money from the information in the paper than I paid in.

It was also very informative.

If you can't derive value from it, I understand all the trouble people go through to bypass it. It's just that I see that people's stated aims and revealed preferences are in contradiction. Everyone wants high quality news. No one seems to want to pay what it costs.


I would pay if I don't fear them. They constantly trying to push a propogenda, and you have to pay for it is indigestible

>The Journal is a rarity in publishing in that it gets more money from readers than advertising

This is probably a good indicator of quality.


It's true that it indicates quality, but I suspect it also indicates the exceptional wealth of WSJ's readership.

The problem is that newspapers believe the myth that they were really efficient at reporting on and delivering news, when really their core competency was their ability to deliver print advertising to every doorstep in a given market on a daily basis. This is similar to the music industry, whose core competency was not developing talent or setting the standards of taste, but rather delivering precisely molded disks of vinyl to far-flung stores every Tuesday.

Oh internet, what myths do you believe about yourself? That you are the harbinger of universal enlightenment and democracy?


Archive.is is another frequently viable workaround.

Tangent: I'm not sure I can think of recent money I've spent better than a year-long Washington Post subscription.

Not because I want to support journalism (though I do), but because having a whole newspaper to flip through every day is so much more productive and efficient than getting dribs and drabs through Twitter and free news sites --- many of which are ultimately sourcing through the journalism that organizations like WaPo do anyways.

Realizing in retrospect how trifling the amount of money we're talking about for a subscription is (it's a tiny fraction of my monthly information services bill [cable/Internet, Netflix, &c]), I feel kind of embarrassed for not having done this a long time ago.

I like WaPo but you might prefer something else, like NYT or WSJ. I'm sure all the major papers are solid choices. Most of you are younger than me. Don't be like me! Subscribe to a damn newspaper already.


To lower the barrier even further, if you're an Amazon Prime subscriber and subscribe via Amazon, the first 6 months are free and then the cost is $4/month thereafter.

What's funny/terrible about this is that I am in fact a Prime subscriber, and didn't know this. I'm tempted to revise my subscription stuff and add the NYT. :)

Maybe this doesn't count as supporting, but: you also get free access with a .edu, .gov, or .mil email address.

really? I cant find that option. How does one sign up?

It's not as obvious as I'd remembered. I think these instructions work:

http://help.washingtonpost.com/link/portal/15067/15080/Artic...


I am trying to find it on Amazon, and failing. Can you please provide a link?

How do you get this deal? I'm having trouble finding it online.


It's amazing how many people I know who have subscribed to the WaPo in the last 12 months. I know, I know - echo chamber - but it gives me great hope that quality news organizations have a future. If executed properly, most people are willing to pay for quality content (side note: a major pet peeve of mine is when you pay for a subscription and are then bombarded with obnoxious pop-up ads whenever you try to access said paid content online).

I've been a long-term WSJ subscriber and recently switched to the digital-only subscription due to the quality of their tablet app, but it remains a guilty pleasure of mine to buy up a selection of hard copy newspapers on a rainy weekend and lock myself away for hours.


> but it gives me great hope that quality news organizations have a future

I hope that high quality news organizations do have a future, but the WaPo? I'm not sure that label fits.


Why do you say that? Their political coverage and some of their deep-dive investigative pieces have been top drawer IMO.

I think this article pretty much sums up how I feel about the Washington Post: https://theintercept.com/2017/01/04/washpost-is-richly-rewar...

How about these two pieces published within about a week of each other?

https://www.washingtonpost.com/news/fact-checker/wp/2017/01/...

https://www.washingtonpost.com/posteverything/wp/2017/01/23/...

You'll note that the two stories are directly contradictory. The claim made by the WaPo is stronger than Senator Sanders' and yet Senator Sanders received 4 Pinocchios for the claim.

I also like FAIR's analysis of the WaPo's opinion leanings (though it leaks over to supposed factual reporting too): http://fair.org/home/washington-post-ran-16-negative-stories...


Thanks, I hadn't seen that before. I still value a lot of the work they did last year, but it's a great reminder to keep a critical eye on what I consume. Now, please don't tell me the WSJ and FT are guilty of the same offenses? I'm not sure I can handle that much disappointment in one day.

I hate to be the bearer of bad news, but the WSJ is Rupert Murdoch's paper. Take a look at their editorial page.

In this case I think the WaPo fact-checker was unfair in handing out Pinocchios. Four Pinocchios should be reserved for bald-faced, baseless falsities. In this case, Bernie had a source, based directly off of research. The "fact checker" criticized the details, application, and methodology of these conclusions, and offered its own, completely speculative conclusion that the real number is lower. I'm not saying this didn't deserve a Pinocchio or two, but I do think it's a bad call. That doesn't make it worthless, but I do think it certainly undermines its credibility a bit.


The second article in fact references the prior article and makes an argument that the writer is being over optimistic.

The second article in fact references the prior article and makes an argument that the writer is being over optimistic.

Personally, I think it's refreshing to see that kind of dialog in one publication, especially on an issue as complicated as this.


Lying about fake news:

https://sputniknews.com/us/201612081048298376-washington-pos...

It's lies like that that legitimize Trump in the eyes of his followers.


Then subscribe to some other paper. You might like the WSJ. The point is not which paper, and let's not waste time litigating the choice.

Once flying from Dubai to DC I got a free copy of Sunday Financial times. It had huge number of topics. I would have never gotten all this stuff in one place sitting in front of Internet.

Funnily enough, I'm also a subscriber of the FT! The app has the full newspaper ordered in the same way (Lex, The Big Read etc) and provides a pretty great experience for a digital consumer. It's not quite the same as having a hard copy in your hands, but for ease of access and portability - I can read it wherever I am in the world - it really is a fantastic product.

They have this deal where Amazon Prime subscribers get several months free. That's why I'm a subscriber to WaPo now.

The Wall St. Journal is the single highest quality news source I'm aware of.

I'd have a tough time deciding between the NYT and WSJ next, especially since the editorial/opinion section of the WaPo is the least valuable to me (informed opinion is something you can get everywhere on the Internet for free).

(WaPo vs. NYT was basically a coinflip decision for me, but doing WaPo and then NYT would probably create a pretty significant overlap, and WSJ wouldn't. On the other hand, the NYT is a substantially better site than WSJ).


WSJ + WaPo is great IMO. My morning routine is WSJ, FT, WaPo, and sometimes NYT. The NYT is my least favored source partly because they skew too heavily toward my biases and I prefer reading high quality news from alternative perspectives more. YMMV.

I'd suggest the FT as among the highest quality papers. Or a good monthly.

One axis of quality measures here, using the Foreign Policy Top 100 Global Thinkers list:

https://www.reddit.com/r/dredmorbius/comments/3hp41w/trackin...


It's good for American news. For a wider angle and a good website and app, Financial Times is great.

I've found WSJ to be a very good paper, especially for geopolitics and finance.

NYT and WaPo's political slant is almost becoming unreadable for someone who's moderate and disinterested in either US party. I can't speak for WaPo but I've been reading NYT for a decade now and their total abandonment of any attempt at political neutrality was disheartening this past year.

My only problem with WSJ is it's very expensive compared to NYT. About 3-4x as expensive as a Canadian. So I've been using WaPo recently, they're slightly better than NYT but only slightly. Probably going to go full WSJ soon.


"The Wall St. Journal is the single highest quality news source I'm aware of."

I prefer the NYT and think the reporting is better and broader - especially with regard to the fluffier sections like Sunday Styles, the Magazine, the Book Review and so on ... if I had to pick a single news source I think the A section of NYT would be my choice.

However, I have to say that I enjoy reading the Financial Times more because of their practice of condensing a story into a single page without a fold and without a jump to another page. They explore the story in more depth deeper in the paper, but you can read the front page headline column from start to finish without unfolding and page turning.

Also, the NYT has this habit of vastly over-backgrounding quotes ... a ten word quote will have 2-3 inches of comma-delimited background and qualifications and it just breaks the story and is hard to read.


They dropped in quality sharply when Murdoch acquired them, and fired a big chunk of the team. Then they started focusing on offensive conservative propaganda on their editorial page. At that point I stopped reading or paying attention to them.

Frankly, I welcome the news about the paywall since it will mean I see fewer links to their "journalism".

Has something changed in the last few years? Did they somehow turn things around?

[edit: Just wanted to point out that the events I'm referring to were about a decade ago. Concretely, before the acquisition, it seemed like the WSJ was driving something like 50% of the daily news cycle with a difficult to discern bias.

Upper management intentionally destroyed that, at least from an outsider perspective.

Reading the other comments and stories about the whole fake news phenomenon makes me think no one has managed to replicate the function they used to serve.]


The WSJ editorial page was famously a conservative bastion long before Murdoch bought them.

But this conversation is a rat-hole. We can generate 1000 comments about any major newspaper. I think my point was just: pick one and subscribe to it.


Their opinion pages have always been filled with grade A crazies and that does actually seem to have gotten worse, but their reporting is still good.

This was one I read recently that actually made a good job of covering the real story in detail: http://blogs.wsj.com/indonesiarealtime/2013/08/26/the-strike...


The economist is the only other news source that I know of that is on par.

I've heard that it's reporters are actually quite young and inexperienced, but the editorial anonymity masks that:

https://en.wikipedia.org/wiki/The_Economist#Editorial_anonym...:

Is there any truth to that?


The Economist has a list of their corresponds and editors on their website http://mediadirectory.economist.com

They don't seem to be significantly younger or more inexperienced than other journalists. However, they don't have journalistic heavyweights that do month-long investigations or pump out one price winning report after another. Those are rare anyway.

I know that they use a lot of stringers that remain unnamed. It is not unusual to have freelancers do research without naming them but as far as I know they also contribute whole articles. Likely these are people that also work for other news outlets.


Yeah, if you're willing to read a single source, it's not expensive.

This is exactly why I hadn't subscribed in the past: I figured it was pointless, because what I really needed was the gestalt from all the papers.

Boy was that dumb of me. I'm sure I would benefit from having perspectives from the WSJ and NYT to pair with WaPo. But my previous state was having no newspaper at all to rely on, just free sites like (bleh) Slate and (unbelievably bad) cable news web sites like CNN. Adding a single newspaper to that has helped enormously.

And it's not just, or even mostly, about improving my understanding of world affairs. It's that the process I use to bring myself up to speed every day is so much more pleasant than it was before. It's like a version of the blogaverse where everything is reported by professionals. It feels kind of like a miracle.

Like I said: it was worth the money. So much so that I'm kicking myself for not having spent the money before. I would guess I get approximately as much value from the WaPo subscription as I do from Amazon Prime.


>just free sites like (bleh) Slate and (unbelievably bad) cable news web sites like CNN.

I feel genuinely sorry for you. I can't imagine reading that every single day. That's a lot of exposure to, quite frankly, a whole lot of clickbait. Don't NYT and WaPo have a limited amount of free pageviews per day?

Come to think of it, there are millions of people in the world whose primary source of news is social media like Facebook, Twitter (both news from your own circle), and Reddit (probably the worst offender? Clickbait titles generate likes, news that a small majority dislikes gets hidden).

[I had written a rather lengthy bit of text about how that's all a pretty good thing, still. Far better than hearsay and an inability to verify national broadcasting, and let's not forget that HN falls in the above category as well and I enjoy this. But this summary says much the same as the very lengthy text.]

Personally I regularly read the Economist. The additional audio format is nice; completely ad free news and you can speed up the less relevant bits. It focuses on some of the more impactful events in the world, which sometimes gives me just enough time in the week to even read news about Africa. Besides, as it is a weekly, it expands on topics in the way that sometimes I'll not bother reading the dailies when something tragic happened. Preferring to wait until Sunday to have some of the speculation filtered out. Also a company subscription to NYT, FT, and some less known ones.


I didn't stop at Slate and CNN; they (along with Bloomberg View) were just my only "magazine" format daily reads. Obviously, I read important stories on NYT or WaPo.

But the point isn't that you can read the "important" stuff with a free pageviews. The point is that you can click your bookmark bar link for WaPo and just browse the whole site, which is an entirely different experience than just reading a couple stories from them over the course of the week in the process of drilling into a story that's trending on other sites.


I always grab a print copy when they're available somewhere free but they're really unpleasant to navigate. Especially when you get into the inner pages and have to unfold the whole paper. Unless you're sprawled out on the floor or have a giant table, it's practically impossible.

Interestingly, a real impenetrable paywall makes me respect the print edition more. FT is a good example. I also like their format, layout and paper quality. But they are a financial paper so the economics are better.


Sorry, I should have been clearer. I just subscribe to the digital edition.

Friendly tip: if you have Amazon Prime, you can subscribe to WaPo for six months free, then 50% off after that.

"Not because I want to support journalism (though I do), but because having a whole newspaper to flip through every day is so much more productive and efficient than getting dribs and drabs through Twitter and free news sites"

I agree with this.

I would like to suggest that you extend this discovery from "daily" iterations with, as you suggest, WaPo/NYT/FT to "weekly" iterations with the London Review of Books[1]. Or the New Yorker. Both are very, very good.

Personally, I prefer reading LRB physically (with scotch) and the Kindle delivery of the New Yorker which I read with meals eaten out.

[1] LRB is about 1/3 editorial/commentary/journal and about 2/3 actual book reviews. The editorial/commentary is a very different flavor from any other media source I know of an is very interesting and refreshing. It's also quite progressive and left leaning which is interesting to me because I am not (so much).


"I'm sure all the major papers are solid choices. "

The problem is that they aren't. Without even getting into operation mockingbird style infiltration of the orgs, the state of journalism as a whole is abysmal, even on the previously well respected journals, and I get this from journalists themselves, at least the ones with integrity.

Honestly right now my method is to simply find the best journalists and writers, and then follow their work regardless of outlet or publisher, and then I smatter in main-stream-media so I understand what the Fox/CNN viewers and WSJ/HuffPo readers, and NPR/BBC listeners are thinking.

In the end though, the problem is all of them are so tied to the money they get from advertising and very subtle but real and powerful editorial influence from the owners/investors/advertisers, that even the ones people have learned to think they trust, like NPR for example, are shining examples of propaganda disguised as anything but, equivalents of Fox news. The oligarchy has infiltrated our mediums of communication because they are a threat.

The internet is the latest version of that threat, which is why we will continue to see attacks on the freedom of speech of the internet as well.

Major shoutouts to the outliers in this equation though who are briding the gap between traditional broadcasting and the itnernet, such as CSPAN, PBS, Charlie Rose, and the universities doing programs such as Conversations with History (UCTV) and Hoover Institute, et al.

PS: WaPo has gone way downhill in their journalistic integrity lately, I expect they are a major mockingbird pivot point. The best traditional source of news I have found? London Financial Times.


Ironically, I've mostly abandoned big-name news sites to listen more to people like tptacek. What I see in the papers is mostly people pushing a story, usually with seemingly little technical expertise, and without the important context, or worse still selective context to support their arguments. The comments on e.g. HN probably have about the same 'median' quality (OK, lower) but if the topic is relevant to the site's interests, the highest quality post will greatly exceed anything you will find in a paper. If it's a contentious point, someone will oppose it. I remember a thread on the Fukushima disaster (sadly I can't find it) containing a detailed discussion of the situation between a few people who had designed / worked on nuclear plants, and its depth and balance made the journalists look like a bunch of children shouting for attention. If there was an issue concerning security, I know I'd want to know what tptacek had to say about it; conversely, I happen to know a couple of WSJ journalists and frankly I'd take anything they say with a bucket of salt, apart perhaps from wine recommendations.

StratFor (for example) seems maybe worth it but I feel like the generalist reach and short form style of big news stops it from ever truly educating. I guess what I'm trying to say here is OK, the experience might be more pleasant, but what makes you think WaPo / Guardian / NYT content is even comparably as good as a blogger dedicated to the topic? (honest question to anyone)


Recent events have convinced me not to follow such a course of action. Most of the media seems more intent on printing fake news that fits their agenda than honest investigation, leading to the more sarcastic among us to refer to WaPo as Jeff Bezos' blog, etc. The media has a lot to do to recover their standing in my eyes before I'll read their work, much less pay them for it.

Something people don't appreciate about the culture wars: as left-facing publications paywall, they yield ground to free right-facing ones.

This is why google needs to get into social search. WSJ is basically moving discovery from traditional search to social search. how many searches across social platforms is google completely missing out on? I know for certain topics, i run to twitter to search as opposed to google. buy google already

This article makes no sense.

The Google loophole only works half the time at best.

All articles posted on HN invariably end up with the same discussion:

1) "Behind a paywall. Can't read it."

2) "You need to use the google link"

3) "The google link also gives a paywall."

4) "It worked for me."

5) "Doesn't work for me."


If you cant just click on it, its paywalled. Idk why we put up with all this bs for the so so rag that is the WSJ. I just downvote paywalls and move on.

>I just downvote paywalls and move on.

I am confused. I always thought that on HN only comments can be downvoted.

Did you mean "flag paywalls and move on"?


I welcome this move. It will stop their articles being posted here and in other forums, we simply can't read them. While WSJ does have good content, I will not pay for it and this will make sure it doesn't trick me into clicking links and have to close browser.

On other hand, for their business to continue to grow, they need to come up with way to show us that content somehow, maybe not the one they charge for but something that can display excellence.

Good examples would be ArsTechnica and Nautilus.


I have found that the best way to get past editorial bias is to aggregate using Realclearpolitics.com

after this election i'm even less interested in any newspaper subscriptions than i was before.

they're all biased liars, or at least run stories buy such authors.


So the best offer the WSJ can provide is "All Access Digital" for $198 over 12 months, after which the price is doubled to $396 for the same length of time. That means, eventually, you'll be spending about $33 a month just to read what are mostly AP stories without even the guarantee of an ad-free experience.

How is this attractive?


I have used Oasis ha­cker quite a number o­f times and he has ne­ver disappointed me. ­He does all types of ­mobile hacks; get unr­estricted and unnotic­eable access to your ­partner/spouse/anybod­y's facebook account,­ email, whatsapp, tex­t messages. He also m­ake changes in any da­tabase/website such a­s your college/univer­sity grades. They are­ very reliable contac­t him at dataoasis247@outlook.­com.

Legal | privacy