Why You Shouldn’t Trust Positive Online Reviews—Or Negative Ones, For That Matter

SMART SPENDING
Go to article
By BRAD TUTTLE | @bradrtuttle | August 28, 2012 |

Researchers and online review sites alike are trying to root out efforts to manipulate the system. By outing dubious, planted, solicited, and otherwise inauthentic reviews, consumers will have more reason to trust the reviews that remain, the thinking goes. But judging on how prevalent the manipulation seems to be, it’s arguable that online user ratings and reviews are less trustworthy than ever.

Over the weekend, a New York Times story about online reviews focused on the experiences of Todd Rutherford, who has this to say about the world of online reviews:

“When there are 20 positive and one negative, I’m going to go with the negative,” he said. “I’m jaded.”
Rutherford should know: He spent years writing and commissioning others to write thousands of what he calls ““artificially embellished reviews” of books. Often, the book “reviews” required little more than a 10- or 15-minute literal “review” of the book in order to produce 300 words of glowing, somewhat relevant and customized praise. Before Google suspended his ad account due to his pay-for-positive-reviews business, Rutherford was pulling in as much as $28,000 per month. One author admitted to paying $20,000 over the years to various services so that they would review his books.

The expose on fake book reviews is the latest example of why you shouldn’t trust online reviews. Or at the very least, you should review the reviews with an especially skeptical eye.

Another example comes from a recent post at Automotive News, which relates that Google is suddenly and without warning deleting dozens, sometimes hundreds of reviews of car dealerships at Google+ Local. Google isn’t explaining why, exactly, reviews are disappearing. But it did release a boilerplate statement noting that “these measures help everyone by ensuring that the reviews appearing on Google+ Local are authentic, relevant, and useful,” giving the indication that there was reason to believe the reviews were fake or somehow inauthentic.

The dealerships maintained that the deleted ones “are legitimate, obtained by requests over several months to sales and service customers,” according to Automotive News. But some would question the authenticity of reviews that come as the result of the pleas of the businesses being reviewed. Review giant Yelp actively discourages review requests by businesses because it taints the objectivity of reviews. What business, after all, would ask a displeased customer to offer his honest thoughts, feedback, and gripes in an online review? “Self-selected reviews create intrinsic bias in the business listing,” Yelp explains, “a bias that savvy consumers (read: yelpers) can smell from a mile away.”

Yelp tries to filter such reviews out of its ratings, and apparently so does Google+ Local. As a result, one auto dealership saw its number of reviews drop from 300 to just 11, and the reviews that were left were mostly negative. Before the mass deletion, the dealership had a rating of 29 out of 30 points; afterward, the rating was in the single digits.

Which rating is more accurate—the one before or after Google got rid of hundreds of reviews? It’s hard to say. Google and Yelp admit that their systems sometimes wind up flagging perfectly legitimate reviews. But that’s better than leaving up reviews that are very likely bogus, they claim.

Bear in mind that it’s not just positive reviews that are sometimes fake. After examining half a million online hotel reviews, a new study highlighted by the Talking Travel Tech blog comes to the conclusion that small, indie hotels are likely faking reviews. They’re not only generating fake positive reviews for their businesses, but they’re also producing fake negative reviews for nearby chain hotel competitors.

How do the researchers reach these conclusions? They compared reviews at TripAdvisor with those from Expedia. The big difference with reviews from these sites is that at Expedia, it’s necessary to have stayed in a hotel—and booked it through Expedia for proof—to write a review of it. TripAdvisor reviews are under no such obligation, and as a result, anyone can pen a review of a hotel, even someone who has never been a guest of the property.

Previously, other researchers have concluded that it’s sites like TripAdvisor, where no proof of customerhood is required for reviews, have the highest prevalence of fake reviews. The latest study backs up this theory by focusing on small mom-and-pop hotels—properties that stand the most to benefit from the reviews system because, unlike chain hotels, they have no national reputation to lean on.

The study shows that small hotels are “about 10% more likely to receive five-star reviews on TripAdvisor than they are on Expedia, relative to hotels owned by large corporations.” What’s more, researchers say that when a better-known hotel brand is located near an indie hotel, the chain hotel is more likely than one of its isolated sister hotels to get one- and two-star ratings at TripAdvisor. Overall, it’s estimated that a chain hotel down the block from an indie hotel will get hit with five more fake negative reviews than the same brand hotel that isn’t directly competing with a small, independently run hotel.

The researchers found that there was “relatively more positive manipulation than negative manipulation, even though the order of magnitude of the two is similar.” The big takeaway is that the system is being manipulated with fake positive and fake negative reviews—and that’s all negative for consumers who are using them to try and make smart choices.

Brad Tuttle is a reporter at TIME. Find him on Twitter at @bradrtuttle. You can also continue the discussion on TIME’s Facebook page and on Twitter at @TIME.