Whose face is it anyway?

Facial recognition technology is a staple of sci-fi thrillers like “Minority Report.”  But of bars in Chicago? SceneTap, a new app for smartphones, uses cameras with facial detection software to scout bar scenes. Without identifying specific bar patrons, it posts information like the average age of a crowd and the ratio of men to women, helping bar-hoppers decide where to go. More than 50 bars in Chicago participate.

As SceneTap suggests, techniques like facial detection, which perceives human faces but does not identify specific individuals, and facial recognition, which does identify individuals, are poised to become the next big thing for personalised marketing and smartphones. That is great news for companies that want to tailor services to customers, and not so great news for people who cherish their privacy. The spread of such technology – essentially, the democratisation of surveillance – may herald the end of anonymity.

And this technology is spreading. Immersive Labs, a company in Manhattan, has developed software for digital billboards using cameras to gauge the age range, sex and attention level of a passer-by. The smart signs, scheduled to roll out this month in Los Angeles, San Francisco and New York, deliver ads based on consumers’ demographics. In other words, the system is smart enough to display, say, a Gillette ad to a male passer-by rather than an ad for Tampax.

Those endeavors pale next to the photo-tagging suggestion tool introduced by Facebook this year. When a person uploads photos to the site, the “Tag Suggestions” feature uses facial recognition to identify that user’s friends in those photos and automatically suggests name tags for them. It’s a neat trick that frees people from the cumbersome task of repeatedly typing the same friends’ names into their photo albums.

“Millions of people are using it to add hundreds of millions of tags,” says Simon Axten, a Facebook spokesman. Other well-known programs like Picasa, the photo editing software from Google, and third-party apps like PhotoTagger, from face.com, work similarly.
But facial recognition is proliferating so quickly that some regulators in the United States and Europe are playing catch-up. On the one hand, they say, the technology has great business potential. On the other, because facial recognition works by analysing and storing people’s unique facial measurements, it also entails serious privacy risks.

Using off-the-shelf facial recognition software, researchers at Carnegie Mellon University were recently able to identify about a third of college students who had volunteered to be photographed for a study – just by comparing photos of those anonymous students to images publicly available on Facebook. By using other public information, the researchers also identified the interests and predicted partial Social Security numbers of some students.

“It’s a future where anonymity can no longer be taken for granted – even when we are in a public space surrounded by strangers,” says Alessandro Acquisti, an associate professor of information technology and public policy at Carnegie Mellon who directed the studies. If his team could so easily “infer sensitive personal information,” he says, marketers could someday use more invasive techniques to identify random people on the street along with, say, their credit scores.

Today, facial detection software, which can perceive human faces but not identify specific people, seems benign.

Some video chat sites are using software from face.com, an Israeli company, to make sure that participants are displaying their faces, not other body parts, says Gil Hirsch, the chief executive of face.com. The software also has retail uses, like virtually trying out eyeglasses at eyebuydirect.com, and entertainment applications, like moustachify.me, a site that adds a handle bar mustache to a face in a photo.

But privacy advocates worry about more intrusive situations.  Now, for example, advertising billboards that use facial detection might detect a young adult male and show him an ad for, say, Axe deodorant. Companies that make such software, like Immersive Labs, say their systems store no images or data about passers-by nor do they analyse their emotions.

But what if the next generation of mall billboards could analyse skin quality and then publicly display an ad for acne cream, or detect sadness and serve up an ad for antidepressants? “You might think it’s cool, or you might think it’s creepy, depending on the context,” says Maneesha Mithal of the Federal Trade Commission.

On Facebook, people who find the photo-tagging suggestion program creepy may turn off the system that proposes their names to friends who are uploading photos. If people opt out, Facebook deletes their facial comparison data, according to the site. Users may also preapprove or reject being listed by name in a friend’s photo before it is posted on their profiles.

Those options may suffice for many.  But in Germany, where German and European privacy regulations require private companies to obtain explicit permission from a person before they store information about that individual, merely being able to opt out does not go far enough, says Johannes Caspar, the commissioner of the Hamburg Data Protection Authority. 

Caspar says many users do not understand that Facebook’s tag suggestion feature involves storing people’s biometric data to re-identify them in later photos. Last summer, he asked Facebook to give current users in Germany the power to delete their biometric data and to give new users in Germany the power to refuse to have their biometric data collected in the first place.

Caspar said last week that he was disappointed with the negotiations with Facebook and that his office was now preparing to take legal action over the company’s biometric database.