What Can Augmented Reality Tell You About Your Skin?

Racked is no longer publishing. Thank you to everyone who read our work over the years. The archives will remain available here; for new stories, head over to Vox.com, where our staff is covering consumer culture for The Goods by Vox. You can also see what we’re up to by signing up here.

I first encountered the HiMirror at a trade show preview, which was held at the end of a long, drizzly day. (Read: I looked like I’d been dragged backwards through the woods.) It looked like a small bathroom mirror. It showed my reflection like a mirror. But when I stood in front of it, it sprang to life like an anthropomorphic sidekick from a Disney movie, with the mirror transforming into a screen similar to a tablet.

HiMirror, a virtual mirror that can do everything from analyze your skin to play YouTube beauty tutorials, isn’t a touch screen. It instead operates by gestures, so I used my hands to “swipe” over to the skin assessment feature. A slave to my skincare regimen, I was dying to put my skin to the test. If this thing was so smart, what would it have to say about me, lover of antioxidants and eye cream?

It had a lot to say after analyzing metrics like dark spots, dark circles, redness, pores, clarity, fine lines, and wrinkles — and grading each accordingly on a scale of 100. None of it was very good. I didn’t realize how much redness I had. My skin clarity sucked. I didn’t have pores but craters, and I do not want to even get into the state of my dark circles. I left feeling like my skin had gotten a big, red F and went home to drown my face (and feelings) in glycolic acid.

HiMirror is one of a handful of new platforms that harnesses the power of augmented reality — that is, a view of reality (i.e., a photo of your face) combined with a computer-generated augmentation — and uses it to analyze and diagnose skin, track changes over time, and, in some cases, recommend products that could help. Earlier this year, Samsung announced their development of a device with similar capabilities, called Lumini, at the 2017 Consumer Electronics Show. Others include Olay’s online Skin Advisor, as well YouCam Makeup and ModiFace, two popular smartphone apps that apply AR to makeup and have only just expanded into skincare. The apps in particular widen accessibility to users, revealing a larger trend.

“As technology becomes ingrained in who we are, from our smartphones to our iPads to wearable tech, beauty has picked up on the convenience and lifestyle that technology provides consumers,” writes Larissa Jensen, beauty industry analyst at The NPD Group, in an email to Racked. While augmented reality, or AR, in makeup apps has pretty much entered the mainstream — thanks in part to partnerships with heavyweight beauty brands like Estée Lauder, Elizabeth Arden, Laura Mercier, and L’Oréal — AR in skincare has yet to see this kind of growth.

In fact, facial devices within the skincare category fell by 20 percent in the first half of 2017, keeping with a downward trend from 2016. There’s a learning curve with some of this software, and if consumers aren’t properly educated about how to use them, they lose interest. It’s not as simple as an overlay of a lipstick color. For instance, HiMirror works entirely through hand gestures (though the company recently introduced a remote) and offers so many functions, from playing YouTube videos to reporting the time and weather report, that just working your way through the menu and over to the skincare assessment can be tricky for a new user.

Plus, AR in skincare might not be as wildly popular as its makeup counterparts because it’s just not as shareable. YouCam Makeup has a built-in social media network, called Beauty Circle, which encourages users to share selfies and watch in-app tutorials. (It resembles Pinterest, if Pinterest had less wedding inspo.) Other AR makeup apps, including MakeupPlus, encourage you to link up your social media platforms for easy sharing. After all, the final look is like a playful Snapchat or Instagram Story filter — with eyeliner instead of dog ears.

“Having that link to social media provides fun as well as inspiration,” explains Jensen. Plus, that inspiration can come directly from brands in the form of makeup looks, red carpet events, and fashion week sneak peeks. “Allowing consumers to see a brand’s makeup applied backstage at a fashion show gives them the feeling of being part of the brand, and enables them to connect more emotionally with it,” says Jensen. Skincare doesn’t have that sort of shareable aspect. After all, would you want to send a makeup-free selfie to your Instagram followers — and on a regular basis?

But that doesn’t mean there’s a low chance of success of AR within skincare. Experts are actually expecting gradual growth in both AR and virtual reality, or VR within the beauty realm, and it’s possible that that may crop up within skincare. It may be slow, but it’s growth all the same. “Because the role of technology in our lives is only expected to aid in convenience and further impact the way we have traditionally done things (ex. self‐driving cars and supermarkets without cashiers), the role of AR and VR in beauty will only increase in significance,” writes Jensen. How, exactly, that’ll pan out remains to be seen.

AR is already gaining ground within existing apps. Perfect Corp, the Taiwan-based parent company of YouCam Makeup, is one of the companies introducing this technology to the masses. It added a skin diagnostic function to the YouCam app in August of this year, having worked with a team of dermatologists in Asia to devise a skin-grading algorithm. The company, which owns patented facial-recognition technology, created the algorithm to compare your skin against a library of data from other users. Then, it gives you a score. “If you get a skin score above a 90 — which is amazing — it means your skin health is in the top 10 percent of all users,” says Adam Gam, the VP of Marketing at YouCam. The average is anywhere between 66 and 89.

But the number isn’t there to rank you — SAT scores these are not. “The number just gives you a baseline for yourself,” explains Gam. “If you use the tool when you wake up with a clean face, no makeup, and in the same place every day, you can start tracking progress against your own score.” There’s a reason that the feature is called a “Skin Diary” on the app. It’s not grading you on a scale of your peers so much as providing a way for you to track your whether a new product is working and understand how your skin is changing. It’s your skin today versus your skin last week.

Tracking your skin over time could benefit it in other ways. “This technology can assess specific skin issues, offer reminders to keep on track with your prescribed regimen, and even show you your progress,” writes dermatologist Joshua Zeichner, MD, director of cosmetic and clinical research in dermatology at Mount Sinai Hospital in NYC, in an email to Racked. “Some people actually stick to their skincare routine better if they have the motivation that technology can provide.”

Still, the concept can be unsettling, depending on your skin concerns and how you feel about them. You could want to step up your skincare routine — or become discouraged by the number. “I think that this kind of technology plays into peoples’ vulnerabilities,” says Amy Wechsler, MD, a dermatologist in New York City who’s also board certified in psychiatry. “It makes people feel bad, like when you shine a blue light on people’s faces.” She compares it to a magnifying mirror, which literally lets you get up close and personal with pores, fine lines, and dark spots. In some cases, this laser focus on the skin can even lower self-esteem and exacerbate anxiety, says Wechsler.

To minimize these repercussions, HiMirror actually did away with the scoring system, since users thought that they were being compared against one another. (They weren’t, but the “score” concept was misleading.) And for Beautiful Me, a skin assessment app from makeup AR brand ModiFace, the creators made a conscious decision to include a positive spin. “Motivational and instructional tips highlight the great features of a person's skin and help enhance or maintain them,” Modiface founder and CEO Parham Aarabi writes in an email to Racked.

In the end, simply approaching these AR platforms as a tracking tool — versus a grading system — may be the best way to avoid feeling discouraged about your skin and obsessing over it. Phair Tsai, the senior manager of marketing and communications at HiMirror, notes that most users do an analysis on their skin once a week, while others do it every other week. In that respect, it’s not that different from weighing yourself.

And that’s if you even want to go there. “Usually I see two groups of people,” says Tsai, who attends a lot of trade shows similar to the one where I first used the HiMirror. “Those who are eager to see their skin truth — and people who don’t want to know.” Who can blame them?