Do you have a link to the source used to classify some news as fake and other news as real in this paper? How do we know liberal cohorts sources are any more reliable? Eg vox and cnn has a similar type of tilt as breutbart and fox.
Without validating that the definition does not preclude the conclusion I don’t see how we can trust it’s conclusikn.
>Posts containing links to external websites are cross-referenced against lists of fake news publishers built by journalists and academics. Here, we mainly use measures constructed by reference to the list by Silverman (7), but in the Supplementary Materials, we show that the main results hold when alternate lists are used, such as that used by peer-reviewed studies (2).
It's domain level, not article. You can follow the links in the paper to see how those groups come up with these lists.
soundwave106 replied in another message with the sources. Buzzfeed seems to be the primary source of the classifications, a far-left organization, so the conclusion is with all likelihood predetermined by politically motivated definitions of fake news.
These are the classification sources according to soundwave106:
A) The primary source was a list of fake news sites compiled by Buzzfeed Media [1]
B) The study was cross-checked with a list of sites from a peer reviewed paper (H. Allcott, M. Gentzkow, Social media and fake news in the 2016 election. J. Econ. Perspect. 31, 211–236 (2017)) and according to the paper was similar to buzzfeed suggesting an ideological tilt.
There is some additional methodology in the study link.
The claim of the study is that they can investigate prevalence of fake news sharing of different groups, so the concerns is more expansive than that and puts the scientific validity of their conclusions on dubious grounds.
There are four questions that need to be answered:
1) are the classifications complete over the data set in the study, not an arbitrary different data set
2) is the populations studied representative of the populuation in general
3) are the classifications unbiased
4) are the classification structure sufficiently granular to represent uncertainty
On #1 they did their study on a three-month segments beginning 9 months from election day, while the study by YouGov is over a different set of voluntary users in a different time period.
On #2 the population was chosen by voluntary particiation and I don't see any mention of them doing necessary statistical analysis to make sure it is representative of the general population.
On #3 and #4 buzzfeed said they collected the classifications by searching for fake news of interest to people of their ideological tilt. Some of which is admittedly not fake news, such as hillarys mishandling of government emails:
-- BuzzFeed News used the content analysis tool BuzzSumo, which enables users to search for content by keyword, URL, time range, and social share counts. BuzzFeed News searched in BuzzSumo using keywords such as "Hillary Clinton" and "Donald Trump," as well as combinations such as "Trump and election" or "Clinton and emails" to see the top stories about these topics according to Facebook engagement. We also searched for known viral lies such as "Soros and voting machine."
and
-- Two of the biggest false hits were a story claiming Clinton sold weapons to ISIS and a hoax claiming the pope endorsed Trump, which the site removed after publication of this article. The only viral false stories during the final three months that were arguably against Trump's interests were a false quote from Mike Pence about Michelle Obama, a false report that Ireland was accepting American "refugees" fleeing Trump, and a hoax claiming RuPaul said he was groped by Trump.
I am sure many other problems could be found if I looked more, but just one of these would put a nail in the coffin of their conclusion and together they just put it on dubious grounds.
Without validating that the definition does not preclude the conclusion I don’t see how we can trust it’s conclusikn.
reply