The positivity problem: why online star ratings are too good to be true

Researchers have found that reviews on websites like Yelp and Amazon are overwhelmingly positive – something they believe renders star ratings unreliable.

A new study published in Nature has shown that upwards of 80% of online reviews rate items with four or five stars. That creates what they’ve termed the “positivity problem” for consumers hoping to use star ratings as a guide for where to dine or shop.

“The basic idea behind the positivity problem is how do you know what to choose if everything is rated so positively? How do I choose one-four-and-a-half star restaurant versus another?” researcher Matthew Rocklage, an assistant professor of marketing at University of Massachusetts Boston, tells Guardian Australia.

You might think people are more inclined to log on and shout about what they hate rather than what they love – but not so, says Rocklage.

“This is what surprised us too. It is really one-sided. You definitely get a bit more one-star reviews, but it’s dwarfed in comparison to the number of four- and five-star reviews. It’s not even close. So, really, there’s just this overwhelming degree of positivity.”

Researchers also found that star ratings don’t predict much when it comes to a product’s future success – for instance, how a movie will go on to perform at the box office. Instead, they found the degree of emotion present in written reviews is a more reliable indicator of how a product will fare in market.

For consumers, then, the key is to ignore the stars and read the reviews.

“What you should really be looking for is the feelings that people are expressing in their text,” Rocklage says. “So not just reading the text for general information, but OK, generally does this person seem to express feelings behind their words?” That can mean looking for words like “inspirational” or “enchanting” rather than just “good”, he explains.

While this study looked at US websites, Australian product reviews are a similarly positive place.

A representative for ProductReview.com.au says more than 64% of reviews on the website are four or five stars. About 8.5% are two or three stars and the rest are one star.

On TellMeBaby, a popular baby product review website, reviews are even more positive – the average rating across all categories is 4.19 out of five stars.

“Eighty per cent of our reviews are positive, 10% are negative and the other 10 are neutral. But of the 80% that are positive, 50% are five stars,” the TellMeBaby community manager, Mel Mahoney, says. “It’s really heavily weighted to being very positive.”

So how, then, do TellMeBaby users find the best baby products?

“I think that’s where the detail of the reviews comes into it,” Mahoney says. “Some of them get really detailed and those are the kind of reviews that people like best. They write about the pros and cons; sometimes they share tips on how to get the most use out of a product. And that’s what our community really enjoys – it’s like they’re reading notes from a friend.”

Mahoney believes community reviews serve an important function, especially in the age of social media. She says TellMeBaby’s research has found that 99% of their users trust consumer reviews but only 65% trust recommendations from influencers or bloggers. “It’s very much parents talking to each other peer to peer that helps them make their decisions,” she says.

So in a sea of positivity, Rocklage says the lesson for shoppers is to “think twice” before relying on star ratings. Instead, try to base your buying decisions on other things you see in the reviews.

“And I know that’s not a crazy message, right? Read the reviews, don’t rely on the ratings. But I do think it is surprising how little the ratings can be relied on.”