Amazon has fallen to Campbell's Law
Reviews, the centerpiece of trust in the platform, are no longer reliable.
In this piece, I’ll talk about why Amazon’s quantitative systems are failing, and why that was entirely predictable based on a cynical 60 year old theory of quantitative social measures.
At 10 weeks old, my little girl isn’t watching much. We’re trying to put it off as long as possible. But my wife and I have been huge fans of Studio Ghibli animation for a long time (My Neighbor Totoro for example). Anticipating the time when my daughter will start watching cartoons, I started looking around for a Studio Ghibli DVD collection. Like many, I went to Amazon.
I found this collection. Good reviews, too. Here’s a link, but I took screenshots in case the item gets taken down.
But as I started to look over reviews, the “headline” 4.5 stars came into question.
Amazon was selling a bootleg version of Studio Ghibli’s movies, with the product showing up as the first result when searching “Studio Ghibli DVD collection”. Search results also include a lot of other bootlegs, a half dozen in fact. A look at the reviews on my particular bootleg shows a pile of negative reviews, but the 5 stars outweigh everything else. This item looks legit, the reviews seem good, and Amazon gets their cut.
The problem is that after this experience, plus increasingly more over the last two years in particular, I’ve lost trust in Amazon’s review system. At this point, I’ve stopped trying to look on Amazon and instead I’ll buy DVD’s direct from the Ghibli website.
There’s any number of items I’ve encountered that seem like knockoffs. As a result, Amazon is starting to look like a flea market. And flea markets aren’t known for reliability.
Amazon’s Review Problem
None of what I’ve said is groundbreaking. Problems with reviews are mounting for the commercial giant. The past three years have stories of Amazon and the FTC “cracking down” on fake reviews and bootleg products like the ones I’ve identified (2019, 2020, 2021).
However, we’re entering the holiday season and I don’t see any sign of this going away. So what’s going on? Three possibilities:
Those crackdowns failing to work.
Those crackdowns not actually happening.
Some combination of the two.
Unfortunately for Amazon, they’ve fallen into the classic Campbell’s Law trap of becoming a big, important enterprise with lots of social capital. Let’s talk about how.
I am a fan of organizational theory, especially the cynical kind (I own multiple copies of Parkinson’s Law). Campbell’s Law is one of the more cynical views on organizations, public policy, and perverse incentives. It states that:
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
I reference Campbell’s Law here, but only because it’s the recorded first of a number of similar laws. Goodhart’s Law says similar things and has been the subject of a lot of discussion by economists. It’s all more or less the same idea.
I could put it another way:
People will try to game any quantitative measure when it becomes sufficiently important. The longer this process goes on, the less useful the measure becomes.
Examples of how this happens abound. School funding tied to standardized testing means schoolteachers have incentive to teach to the test is a good example.
Amazon’s review system was bound for corruption.
Amazon’s central place in consumerism made positive reviews on its platform immensely profitable. Unscrupulous vendors can pay for reviews from fake accounts, or create items that look close enough to the real thing to garner good reviews. I’ve seen an enormous number of knock-off items lately, which seem close to the real thing, close enough to get the positive reviews needed to continue making money. Everyone selling goods has incentive to game reviews. Many don’t, but if enough do, the system starts to break down.
At the same time, I think that Amazon has had perverse incentive to not crack down on reviews.
Why not? Because good reviews sell items.
People buy items with good reviews. I don’t know how the algorithm to display reviews worked with the DVD collection above, but despite the reviews trashing the bootlegs, it still showed up as 4.5 stars, which is a decent level of review. When reviews all are good, consumers will continue to spend money on the platform and only pause when they get burned.
What happens next is a matter of trust. Does the customer continue to have trust in the brand, or do they go someplace else after getting screwed by a product where quantitative reviews are gamed?
It’s a hard, if not impossible question to answer. You can pay a marketing department a ginormous amount of money to try and figure out how much a brand is trusted and what consumer behavior will take place after something like my example above. Further, Amazon is such a cornerstone of commerce that the question of consumer trust may not be an issue.
If trust in the brand is maintained, Amazon has reason to take the 3rd choice above. Their reports of cracking down just serve to boost trust enough that the game can continue: record-setting revenue growth, sky-high stock prices, continued elbowing of competitors, and so on. Amazon, thus, has some incentive to keep the gamed reviews in place while simultaneously fighting them.
What happens into the future is unknown. As more people get burned and trust in the brand fails, Amazon may have to resort to real steps to shut down the corruption in the system.
In the meantime, welcome to the flea market.