Experts give low rating to online reviews

Botto Bistro is far from the worst restaurant in America. But it doesn’t mind if you think so. A small Italian place in a strip mall across the bay from San Francisco, Botto is just a few miles from my house. The other night, my family headed there for dinner.

I remembered one of Botto’s reviews on Yelp said, “The pizza tastes like the rag at Denny’s that they use to wipe down the counters and tabletops,” so we decided to get that, plus a beet salad.

The attitude is a little brusque. “We have no ice, no butter, no ranch, no lemon,” a sign behind the counter warns. “We charge for bread. We charge for everything.”

Give Botto five stars for undermining Yelp. The bistro did not want to be reviewed and let itself be subject to the whims of people with no names but plenty of opinions. But Yelp doesn’t allow businesses to opt out.

Some outfits try to load the dice by buying favourable reviews, but Botto asks people to trash it. When we left, the co-owner, Davide Cerretini, gave me a sticker that said, “I gave Botto one star on Yelp.” If I used it, my next pizza would be half price.

More than half of Botto’s 250 reviews are one-star. Cerretini seems to enjoy the game.
“It may sound to you like a suicide mission, but our business is up,” he said.

If Botto’s critical notices on Yelp are often written to be outrageous and unbelievable (“the pizza arrived at the table with a dead rat under the cheese”), they also reflect the confused state of reviewing on the internet. Even as researchers are finding that reviews are less than reliable, more people are relying on them. On Yelp alone, the number of reviews now exceeds 100 million.

Reviews tell us what to read next, where to eat dinner and what to order there, where to go on vacation and what doctor to call. Soon, as Google demonstrated with the introduction of its voice-activated Google Home device in May, reviews will be read aloud to you as you lie on the couch, wondering what movie to see next.

But there are also persistent controversies over how many of the reviews on the internet were bought by the subject rather than written by a neutral party, and whether that distorts all results.

In May, Yelp issued 59 new Consumer Alerts, which are notices it puts on the pages of businesses that have been caught trying to pay for better reviews. Among those cited were a Beverly Hills plastic surgeon and an emergency room in Humble, Texas. FiveThirtyEight.com reported that “men are sabotaging the online reviews of TV shows aimed at women.” (Why? Because they can.)

Bart de Langhe, an assistant professor of marketing at Leeds School of Business at the University of Colorado, used to accept numerical reviews online implicitly. Then, when his son was born three years ago, he needed to buy a car seat. De Langhe noticed that the seat rated lowest by Consumer Reports got a high rating on Amazon, and the one rated highest by Consumer Reports received a low rating on Amazon.

The more popular seat on Amazon was also more expensive. Were reviewers, he wondered, paying more attention to things like price and brand than the objective, measurable ability of the seat to protect its occupant? With two other researchers, Philip Fernbach and Donald Lichtenstein, de Langhe began a study that compared online reviews for items like air-conditioners and car batteries with the evaluations in Consumer Reports.

“Navigating by the Stars” was published in April in The Journal of Consumer Research. After analysing 344,157 Amazon ratings of 1,272 products in 120 product categories, the researchers found “a substantial disconnect” between the objective quality information that online reviews actually convey and the extent to which consumers trust them.

In other words, the consumer saw a number — 4.6 stars out of 5 — and took it much more seriously than it merited.

Nearly half the time, Amazon reviewers and the Consumer Reports experts disagreed about which item in a random pair was better. Moreover, average user ratings did not predict resale value in the used-product marketplace, another traditional indicator of quality.

Julie Law, a spokeswoman for Amazon, said “Amazon customer reviews reflect the feedback, tastes and concerns of real customers, not professional reviewers. That’s what makes them powerful.” She also said the company was now giving more weight to the most recent helpful reviews from purchases that were verified.

De Langhe stuck with this recommendation: “You should rely much less on reviews than you currently do.”

That would be hard, because consumer reviews have wormed their way into offline life. In Amazon’s first physical bookstore, opened in a Seattle mall, good reviews from readers both help get the books selected for the store in the first place, and then are used to compel a sale. So the potential buyer is told via a “shelf-talker” — a card below the book — that Vladimir Nabokov’s “Lolita” was awarded 4.2 stars by readers, while George Orwell’s “1984” got 4.5 stars, as did “Pride and Prejudice.”

The store also carried some novels that are self-published or from small publishers, and these often seemed to receive better ratings than the classics. Tamara Lyon’s “Post-Traumatic Brazilian Wax Syndrome,” in which an interior designer named Bristow Sparks tries to regroup after colliding with a park bench on a first date and suffering other disasters, got 4.7 stars.

Jordan Nasser fell just shy of a perfect score with his first novel about a gay New Yorker returning home to Tennessee, “Home Is a Fire.” He got 4.9 stars.

In an email from Stockholm, where he is now living, Nasser said the praises were an embarrassment of riches.

“Honestly, I wish ‘Home Is a Fire’ had received more three- and four-star reviews, in order to balance out the final result, because I think the sequel, ‘The Fire Went Wild,’ is a much better book,” he wrote.

In any case, Nasser said his book was not comparable to the classic books with lower ratings. “Absolutely not, and I would never imply that in any way,” he said.

Although that is exactly what anyone strolling through the store would think. Law of Amazon said, “We don’t think there’s anything here that needs to be fixed.”

The problem with reviews and ratings, said Joseph Reagle Jr, author of “Reading the Comments: Likers, Haters and Manipulators at the Bottom of the Web,” is that we have elevated them to supreme importance.

“They have always been somewhat subjective, arbitrary, capricious, befuddling, and bemusing,” he said. “I think what’s different is that we forget this is the case, particularly given the sheen of big data and quantification.”

Six months or so after Botto Bistro began trolling Yelp, the review site blinked. While it did not bend on the restaurant — a Yelp spokeswoman said Botto was simply cheating itself out of customers — the site reportedly began exploring offers to sell itself.

Yelp, which never officially commented on the sale process, was saying in essence that reviewing, at least for the moment, might have peaked. No deal was done, however, after the company took itself off the market last year. So Yelp soldiers on.

So does Botto. If you dig on Yelp, you will find reviewers who said they genuinely were not impressed, which is about where I fell. An adequate pizza, a good salad and a beer were $46 (Rs 3,072), not including the minimum “suggested gratuity” of 18 percent — which, considering this is a place where you set your own table and pick up your food from the counter, was rather aggressive.

I’d write a review saying all this, but I doubt it would do any good.