Do online product reviews have a positivity problem?
Summary: When reading product reviews online, we often see lots of extremely positive ratings—and just a few negative ones. What’s happening when there’s almost no nuance in how people rate things? Do these ratings have a positivity problem?
When shopping online, we often read product reviews. After all, online product reviews can serve as useful guesstimates, gauging how people rate a product, on average. However, these ratings will only be accurate when they represent a collection of honest and diverse viewpoints. In other words, ratings won’t be trustworthy when disproportionately influenced by just one point of view, whether overly positive or negative.
In this light, something interesting has been happening with online product reviews. Several of them show extremely positive ratings—and just a few negative ones. That’s a bit surprising, since most things in life aren’t unequivocally good or bad. Usually, they end up being decent in some ways, while not so great in other ways. So, what’s happening when there’s almost no nuance in how people rate things? Do these ratings have a positivity problem?
The positivity problem: when ratings don’t match marketplace success or quality
It’s certainly no secret that a lot of social media have what we might call a negativity problem. Due to their manipulative algorithms and addictive design, these platforms draw a disproportionate amount of attention to the most outrageous points of view on the Internet.
In contrast, many online product reviews seem to have the opposite problem: a positivity problem. They appear to showcase enormous, sometimes suspicious, amounts of praise for products that people may consider for purchase. Why would all this positivity create a problem for online product reviews?
First off, it turns out that positive ratings don’t have much of a connection to a product’s marketplace success.
- For example, in a study published in Nature Human Behavior, researchers found that anywhere from 80% to 100% of product reviews had positive ratings. And these ratings held steady even when the product didn’t see any marketplace success (that is, it didn’t sell well).
OK, if positive ratings have little connection to a product’s marketplace success, what about the product’s quality? Surprisingly, positive ratings don’t have much of a connection to a product’s overall quality either.
- In another study, researchers found a low correlation between positive ratings of products and the overall quality of those products. For instance, when it came to product reviews on Amazon, there was little correlation between high ratings and high measures of quality (such as product testing scores by Consumer Reports).
In sum, it looks like there’s a clear mismatch between many online product reviews, on one hand, and the product’s marketplace success and overall quality, on the other hand.
Should we trust online product reviews?
In general, I had assumed, at least in theory, that it’s reasonable to trust online product reviews. But research on the positivity problem is making me question that assumption. Going forward, I’ll probably be more skeptical when I read online reviews of products.
As to why the positivity problem plagues so many online product reviews, there could be multiple reasons. Maybe bots are adding fake reviews. Perhaps some places are pushing people to write only positive ratings, such as through kickback, bribes, or some other quid pro quo.
It’s also possible that ratings systems have the same problem as social media: incentivizing simplistic and extreme points of view, while marginalizing more reflective and nuanced voices. The danger here is that truth, which isn’t so black and white, gets pushed to the margins too.
References
de Langhe, Bart, Fernbach, Philip M., and Lichtenstein, Donald R. (April 2016). Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings. Journal of Consumer Research, 42 (6), 817–833. https://doi.org/10.1093/jcr/ucv047
Rocklage, M.D., Rucker, D.D., and Nordgren, L.F. (2021). Mass-scale emotionality reveals human behaviour and marketplace success. Nature Human Behavior 5, 1323–1329. https://doi.org/10.1038/s41562-021-01098-5