Following the herd: the influence of online ratings

The comments section of online news sites is intended to be a place where everyone can express their own opinion. But according to a study published in the journal Science, many people are heavily influenced by the positive opinions of others in such outlets, while less swayed by the negative ones.

The research was conducted over five months, on a major news-aggregation website, by Lev Muchnik of the Hebrew University of Jerusalem; Sean Taylor of New York University; and Sinan Aral of the MIT Sloan School of Management. In order to see how perceptions of favourability affected people’s judgement about others’ comments, the researchers randomly altered the favourability ratings of 101,281 different comments. Through this approach, they could see how readers evaluated the same comments when those comments were given different ratings.

They found that comments whose ratings were manipulated in a favourable direction saw their popularity snowball, becoming 32% more likely than untreated comments to receive a favourable rating from the next viewer of those comments and 30% more likely than untreated comments to obtain a very high favourable rating. The researchers said this “created accumulating positive herding that increased final ratings by 25% on average”.

“This herding behaviour happens systematically on positive signals of quality and ratings,” said Aral. He added that the results “were asymmetric between positive and negative herding” - comments altered to have negative ratings attracted more negative judgments, but that increase was drowned out by what the researchers call a “correction effect” of additional positive responses.

“People are more sceptical of negative social influence,” said Aral. “They’re more likely to ‘correct’ a negative vote and give it a positive vote.”

So what does this mean for the unsuspecting readers? Aral noted that just as the researchers manipulated the ratings for their own ends, other parties with less well-meaning intentions may do the same, such as political operatives, marketers or anyone who stands to profit by creating an exaggerated appearance of popularity. He said the effects of herding can be found in real-world markets such as housing and shares: “The housing bubble was a spread of positivity, but when it burst, some people lost their savings and their houses went underwater.”

There were some limitations to the study though - stories under the rubrics of ‘politics’, ‘culture and society’ and ‘business’ generated positive herding, but stories posted under the topics of ‘economics’, ‘IT’, ‘fun’ and ‘general news’ did not. Furthermore, the researchers say the herding was “affected by whether individuals were viewing the opinions of friends or enemies”.

Nevertheless, the study provides a good introduction to the analysis of crowd-based opinion aggregation and provides a launching pad for future experiments. Aral suggests future work could “explain the psychology of the correction effect on the negative side” as a way of understanding how collective judgments are formed.

The point of the study, said Aral, “is that you need solid science under the hood trying to understand exactly how these mechanisms work in a broad population, what that means for the diffusion of opinion and how can we design the systems to be fair, to have less incentives for manipulation and fraud, and be safe in aggregating opinions”.

In the wake of a federal election, when Australian news sites will be filled with stories and comments on various candidates and policies, this research is particularly significant. Users should ensure they do not just follow the crowd, instead (up)voting for what they truly believe to be right.