---
title: Reading Skin from a Photo, Keeping Only Ingredients with Evidence — An Ingredient Recommender Built to Dodge Where Skin-Analysis Apps Collapsed
url: https://oosioo.com/en/p/%EC%82%AC%EC%A7%84%EC%9C%BC%EB%A1%9C-%ED%94%BC%EB%B6%80%EB%A5%BC-%EC%9D%BD%EA%B3%A0-%EA%B7%BC%EA%B1%B0-%EC%9E%88%EB%8A%94-%EC%84%B1%EB%B6%84%EB%A7%8C-%EB%82%A8%EA%B2%BC%EB%8B%A4
date: 2026-10-14T14:55:00+00:00
author: SYSOP
summary: Services that use a camera to analyze skin and recommend cosmetics have come and gone for a decade now. We examined where twenty-two prior cases broke down, then built a tool that shows ingredients instead of products — and only ones whose effects are confirmed by randomized controlled trials. It also notes what a photo can never measure, down to numbers that shift with a single light source.
---
# Reading Skin from a Photo, Keeping Only Ingredients with Evidence — An Ingredient Recommender Built to Dodge Where Skin-Analysis Apps Collapsed

![A screen showing Rembrandt's "Bust of a Young Woman" with the forehead, nose, both cheeks, and chin marked off in white lines, next to the title "Reading skin from a photo, keeping only ingredients with evidence"](/uploads/44c749546094e0db.webp)

I used to spend a long stretch of time on analyzing skin from camera photos — reading, from a single image, whether someone's skin was dry or oily, what kind of blemishes they had, all the traits that differ from person to person. Even then, the question that always tripped things up at the end was the same one. So what are you going to tell them to put on their skin?

Most services filled in that answer with their own company's products. This time I wanted to do it differently. Instead of a product, the tool tells you the **ingredient** — and keeps only the ones whose effectiveness has been confirmed by research.

## What I set out to do

I settled on three things.

- Measure only what's visible in the photo: shine, redness, blemishes, breakouts, skin texture
- Ask about what a photo can't measure: dryness, sensitivity, pregnancy status
- Attach an evidence grade and the underlying papers to every ingredient. Drop any ingredient backed only by test-tube studies

But before building it, there was something to check first. Plenty of companies had already made things like this — so why does none of it stick in anyone's memory?

## First, why did they all collapse?

I gathered twenty-two cases, domestic and international, and traced what happened to each.

**Almost every service that bundled in a dedicated device shut down.** Shiseido's Optune was a ¥10,000-a-month subscription where you photographed your skin with the app's camera and a dedicated machine mixed that day's serum. It launched in July 2019 and [ended on June 30, 2020](https://www.shiseido.co.jp/optune/term_of_use/). P&G's Opté, a $599 device more than ten years in development that used a camera to find pigment flaws and spray serum only on those spots, [withdrew in April 2023](https://beautymatter.com/articles/pg-shutters-precision-skincare-brand-opte). Neutrogena [unveiled](https://www.engadget.com/2018-01-05-neutrogena-smartphone-skin-scanner.html) an iPhone-mounted scanner at CES 2018, and eventually retreated to selfie-based analysis.

**What survived is free selfie diagnosis, and every single one of them funnels into sales.** Olay, L'Oréal, and Olive Young's Skin Scan all work this way. Olive Young [reported](https://newsroom.cj.net/cj-olive-young-to-expand-in-store-skin-analyzer-service-nationwide) (company figures) that customers who used Skin Scan converted to purchase at 78%, versus 43% for those who didn't. The performance metric is, in effect, revenue. The Neutrogena app once recommended only its own products, and dermatologists called it "primarily a sales tool."

Four sticking points kept recurring.

1. **A photo can't measure moisture.** A 2023 study trained on 914 images found that a model using only visible-light photos was [insufficient](https://pmc.ncbi.nlm.nih.gov/articles/PMC10363786) for predicting hydration. That's why early products bolted on a separate moisture sensor.
2. **Results swing heavily with lighting.** A dermatologist's [report](https://www.dermatologytimes.com/view/do-facial-appearance-apps-actually-work) noted the same person's estimated skin age came out roughly three years younger under incandescent light than under LED light.
3. **Accuracy varies by skin tone.** When dermatology AI models were re-evaluated on photos spanning a range of skin tones, performance (ROC-AUC) [dropped](https://pubmed.ncbi.nlm.nih.gov/35960806/) by 27 to 36 percent.
4. **When diagnosis and sales are fused, people stop trusting it.** If the result always points to the same company's products, the diagnosis itself reads as an ad.

Regulation has caught up too. In January 2025, Korea's Ministry of Food and Drug Safety [added](https://www.newsseoul.co.kr/news/view/1065576619317500) a ban on phrases like "reduces skin age by n years" to its cosmetics advertising guidelines.

On the other side of this sits Hwahae. With no camera at all, built purely on ingredient information, it [posted](https://www.unicornfactory.co.kr/article/2025030709591562223) revenue of 82.4 billion won in 2024. Still, it spent a long time embroiled in controversy for using EWG ratings in a way that conflated hazard with actual risk. An ingredient-first approach can be a real business — but if its evidence framework is shaky, that becomes its own weak point.

So I settled on this design.

| Where others collapsed | What this tool does instead |
|---|---|
| Dedicated hardware | Just a phone camera. Nothing to install |
| Moisture can't be read from a photo | Dryness is collected only through the questionnaire |
| Results swing with lighting | If a photo fails the quality check, its numbers are discarded. No comparison against other people |
| Diagnosis = sales | Ingredients, not products. Product examples follow the same rule across every brand |
| Facial photos are personal data | The photo never leaves the browser |
| "Skin age" / diagnostic language | Not used |

## What I built

You can try it yourself. The photo never goes to a server — everything is computed inside this screen. On a phone, tapping the button opens the camera.

```map /embed/skin-ingredient-2026-v3.html 3:4
Upload a photo and it divides the face into forehead, nose, both cheeks, and chin to measure shine, redness, blemish candidates, red breakout candidates, and skin texture. Combined with seven questionnaire items (tightness, shine, breakout shape, stinging, pigmentation, wrinkles, pregnancy/breastfeeding), it shows a table of products containing evidence-backed ingredients for each concern. The left column is the product, the right column is the evidence-backed ingredients it contains along with their grade (A or B); the study concentration and PubMed citation appear when you expand "View evidence." Prescription-only drug ingredients are listed separately under "View by ingredient." You can also use it with just the questionnaire, no photo. On first opening, it shows a worked example computed from the Rembrandt portrait and a sample questionnaire.
```

Under the hood, there are three parts.

**Splitting the face.** Google's face landmark model places 468 points on the face, and those points are used to cut out five regions: forehead, nose, both cheeks, and chin. Eyes and lips are excluded.

**What's measured from the photo.** Color is converted into a color space closer to human vision (Lab) before measuring. Spots that are brighter and less saturated than their surroundings are read as specular reflection — i.e., shine. Areas redder than their surroundings are counted as redness and breakout candidates; small patches darker than their surroundings but not red are counted as blemish candidates. Holding a white sheet of paper next to the face and photographing it, then tapping on the paper, corrects the color.

**Picking ingredients.** Eight skin concerns were paired with 37 ingredients and the evidence organized accordingly. Grade A requires either a meta-analysis or two or more consistent randomized controlled trials; Grade B requires one or two controlled trials. Anything backed only by test-tube or animal studies was excluded. All 118 cited papers were individually looked up on PubMed and checked against their titles.

A few things turned out to be surprises while sorting this out.

- **Not a single** topical ingredient has been confirmed to directly shrink pore size. The only real options are managing sebum and dead skin cells.
- Retinol's evidence was weaker than expected. In a systematic review, four of nine randomized controlled trials showed no effect, which brought it down to a B.
- Alpha-arbutin and kojic acid were excluded because no randomized controlled trial tested them on their own — despite being common ingredients in brightening cosmetics.
- A good number of the ingredients with the strongest evidence are **prescription drugs** in Korea — tretinoin, adapalene, and high-concentration azelaic acid among them. So those ingredients carry a "consult a doctor or pharmacist" notice.

### Leading with the product, not the ingredient

The first version of the results screen was an ingredient list. For each concern, ingredients were ranked by evidence grade, with products collapsed underneath. But what someone looking at this actually wants to know isn't the ingredient name — it's what to buy. So I flipped the order. For each concern, the product sits on the left and the evidence-backed ingredients it contains sit on the right, with the citations and study concentrations hidden behind an expandable "View evidence."

![A worked-example table. On the left, a chosen product for each concern (bumps/blackheads, shine/sebum, dryness/barrier, blemishes/pigmentation); on the right, the evidence-backed ingredients in that product with A/B grade badges. Ingredients relevant to that concern are bolded, the rest are dimmed](/uploads/ead3daa91cf0c62b.webp =720)

For the products, I pulled the full ingredient lists of 119 items across 51 brands from official brand pages and public ingredient databases. Which product gets surfaced at the top is the part of this tool that needs the most care.

I got this wrong twice. Looking at the first list I assembled, one brand was dominating the exposure — because that brand had every product it sells entered, while others only had a handful picked per ingredient. The second time, in the example where shine was the top concern, a moisturizing cream rose to the top — because a moisturizing ingredient found in nearly every product, like glycerin, was inflating its score. So I applied the same rule across every brand.

- The ingredient form must match the form used in the evidence studies. A vitamin C derivative, for instance, doesn't borrow the evidence built on pure vitamin C studies
- If no concentration is listed, the ingredient must appear within the first 15 on the full ingredient list. Anything listed after preservatives or fragrance is treated as under 1% and excluded
- Rinse-off products are excluded
- For each concern, the score is based on the single strongest matching ingredient. It goes up with a higher grade, an earlier position on the ingredient list, and a disclosed concentration. Adding more ingredients for the same concern doesn't raise the score further
- Basic moisturizing ingredients found in nearly every product (glycerin, hyaluronic acid) lower the score
- At most one product per brand per concern, and at most two per brand across the whole table. Ties are broken randomly

104 products clear these rules and make it onto the table. The table doesn't claim the product works. It only states how far up that product's ingredient list a research-backed ingredient appears. For products that don't disclose concentration, it also notes below the table that there's no way to know whether it's present at the studied concentration.

## Where it still gets it wrong

**What a photo can't measure, it simply can't measure — in the end.** Dryness and sensitivity rely entirely on the questionnaire answers. Making sure the photo analysis never pretends to know more than it does was the thing I paid the most attention to in this tool.

**A single light source shifts the numbers.** I ran the same calculation on one public-domain portrait photo, changing only its color and brightness.

| Condition | Cheek redness | Blemish candidates | Skin texture |
|---|---|---|---|
| Original | 2.0 | 6 spots | 0.59 |
| Warm-toned light | 1.1 (-45%) | 6 spots | 0.53 |
| 40% darker | 1.4 | 4 spots | 0.40 |
| 30% brighter | 2.4 | 8 spots | 0.72 |

The face itself hadn't changed, yet redness dropped by nearly half. At first I didn't even notice the warm-toned lighting. So I added a warning that triggers when the skin's average color skews yellow or blue, and put a line in the results screen: "Don't compare against other people — take the photo in the same spot again and look at the change."

**Shadows get read as blemishes.** The shadow cast by a smile line showed up tagged as a blemish. That's a limit of counting small patches that are darker than their surroundings and not red.

**At first, it even analyzed black-and-white photos.** Film grain in an old black-and-white photo got counted as 28 blemishes. I fixed it so that when there's almost no color variation that can't be explained by brightness alone, the photo is treated as black-and-white and its numbers are discarded. Sepia-toned photos, though, still slip past this check.

**It hasn't been tested on bare-faced selfies.** What was used for testing was public-domain portrait photos and paintings, plus versions of those photos with altered lighting. How it performs on a made-up face, a phone-filtered photo, or darker skin tones is still unknown. The baseline for the numbers was also set without ground-truth data, which is why it's shown only as a "reference."

**The published concentrations for regulated ingredients still need checking against the primary source.** Concentrations for ingredients listed under Korea's functional-cosmetics notices were only partly verified, using industry materials.

## So, to make this click

Taking the photo is the easy part. What's hard in this field is two things: getting the photo to say "I don't know" about what it doesn't know, and keeping the recommendation separate from the sale. Where the earlier companies collapsed was mostly these two things, not the technology.

AI handled gathering the cases, cross-checking paper citations one by one, and filtering 119 full ingredient lists under the same rule. Setting that rule was a human's job — especially the moment I noticed one brand was showing up far more than any other.

This tool doesn't diagnose skin. If redness or breakouts are severe and persistent, seeing a dermatologist first is the right call — because a good number of the ingredients with the strongest evidence are prescription drugs.