Your Favorite Stores Could Be Tracking You With Facial Recognition

Racked is no longer publishing. Thank you to everyone who read our work over the years. The archives will remain available here; for new stories, head over to Vox.com, where our staff is covering consumer culture for The Goods by Vox. You can also see what we’re up to by signing up here.

It’s no secret that customers who shop in stores are being watched. Whether it’s a big-box retailer like Target or a luxury boutique like Chanel, stores have overhead cameras that record shoppers to make sure no one is stealing.

But what if your face is constantly being surveyed through a system that’s looking for criminals? As facial recognition technology becomes more advanced, and more accessible, an increasing number of stores are turning to facial recognition technology to catch shoplifters. Per to the National Retail Federation, retail companies lose about $48.9 billion annually because of theft.

According to a report from McClatchy DC, many stores are turning to a California-based company called FaceFirst for their facial recognition needs. When reached for comment, FaceFirst said it could not list its clients to Racked because of “customer NDAs and confidentiality agreements.” In a promotional video on its website, though, FaceFirst says its retail clients include big-box stores, department stores, grocery stores, and pharmacies, and that it works with many Fortune 500 companies.

Companies that use such software upload photos of people they want to watch, like customers who’ve already shoplifted from their stores, disgruntled former employees, or persons of interest who can be found on local watch lists. FaceFirst’s technology promises to know “the moment a shoplifter enters your store” and says brands that have used its services have seen a 34 percent reduction in theft.

Facial recognition software has been on the market for some time, although up until now it’s been relegated mostly to casinos, airports, and use by the police. According to the John Jay College Center on Media Crime and Justice, the cost of such software has dropped, so more stores are considering it a viable option.

Algorithms have now apparently developed “deep learning,” according to John Jay: The advancement of technology has improved troubleshooting on problems like poor lighting, low resolution, and the inability to identify faces through facial hair. FaceFirst says it can see past disguises like eyeglasses, hats, or wigs, and FaceFirst CEO Peter Trepp says the technology “can match a face against the database of 25 million people in just under a second.”

This type of surveillance is becoming more prevalent in retail. In 2015, Walmart admitted it was using FaceFirst to catch shoplifters but said the program had since been terminated. In 2016, industry experts told Bloomberg that Macy’s and Benetton use facial recognition technology, although both brands denied the report. In England, the BBC found that one in four stores used the technology.

Local grocery stores in Rhode Island and Massachusetts are also reportedly using facial recognition technology from a company called StopLift. Two years ago, Saks Fifth Avenue revealed to the Guardian that it was using the technology in its Canadian stores, via software from a company called Axis Communications.

Those who support such technology say it’s in the best interest of shoppers. Mark Heckman, a retail consultant, told the NRF that it can be seen as “additional security safeguard against fraud, shoplifting and even VIP shopper identification.”

“Given the increasing number of credit card fraud cases, stolen identities and other personal information breaches, I believe most consumers understand that it is to their benefit to make sure they are who they say they are when a payment card is being used,” Heckman said.

But it also has major implications for the privacy of shoppers — something the American Civil Liberties Union has already begun to look into. Just two months ago, the ACLU reached out to 20 of the biggest stores in the US to ask if they use facial recognition technology, and published its findings online.

Only one company, Lowe’s, openly said that it does, and only one, Ahold Delhaize, which owns supermarkets like Stop & Shop and Food Lion, said it does not. The rest of the stores that were contacted, including T.J. Maxx, Target, Home Depot, Macy’s, and Best Buy, all refused to answer.

As the ACLU wrote in its report on facial recognition technology in retail, even shoppers who feel such security is beneficial should at least be made aware of it — especially considering that since the technology is so new, there is little regulation.

“What’s really needed is for retail use of face recognition technology to be subject to sensible regulation to ensure it comports with basic norms of privacy and fairness,” it writes. “Some Americans may disagree with the ACLU’s view that we should not allow this technology to become widespread and routine in American life — but we can all agree that people should have the information they need as to whether they want to patronize stores that decide to use it in.”

The ACLU goes on to warn that such technology can be “the beginnings of an infrastructure for tracking and control that, once constructed, will have enormous potential for abuse, to create chilling effects, and to change what it means to be in public in the United States.” This brings to mind Steven Spielberg’s 2002 film Minority Report (remember the creepy system at the Gap?), although John Jay notes that “most systems immediately discard images of anyone who isn’t a match for a known shoplifter.”

Still, according to a report from the MIT Media Lab, racial bias is rampant in artificial intelligence. According to research released back in February after three different facial recognition technologies were studied, white men were identified correctly 99 percent of the time while women with dark skin were identified incorrectly 34 percent of the time. Such algorithmic mistakes could result in unfair treatment of shoppers of color — not that they aren’t getting that already. (Check out Starbucks, Nordstrom, and Barneys, just to name a few.)

MIT has previously written about how algorithms can be biased from the get-go if they are created without regard for underrepresented gender, age, or race. You might recall how this played out in 2015, when Google’s image recognition algorithm for Google Photos was labeling black people as gorillas; Google still hasn’t been able to fix the problem and has since just censored gorillas from the algorithm completely.

Like all industries, the facial recognition sector is about to blow up now that Amazon is involved. Today, the ACLU reported that Amazon has been selling its real-time facial recognition system, Rekognition, to police departments in Washington, Oregon, and Orlando, Florida.

For a fee of $6 to $12, the police departments have been scanning mug shots against real-time footage from cameras all over the cities. The ACLU says that Amazon tech “violate[s] civil liberties and civil rights” because it can track those who are dubbed “people of interest” by local law enforcement — which the ACLU notes is very likely to be political activists, minorities, and immigrants.

The ACLU, alongside a long list of civil rights groups, has written an open letter to CEO Jeff Bezos demanding that Amazon stop helping the government with surveillance. “Amazon Rekognition is primed for abuse in the hands of governments. This product poses a grave threat to communities, including people of color and immigrants, and to the trust and respect Amazon has worked to build,” they wrote. “People should be free to walk down the street without being watched.”

That liberty has likely been lost from shopping.