Skincare & Ingredient Intelligence

Product information sits across thousands of inconsistent pages. Ingredient names vary by spelling, format and naming convention. Some pages bundle several products together. Images can be missing, wrong or linked to the wrong database record.

That makes direct comparison unreliable without a large amount of manual cleaning.

I built an extraction, cleaning and matching pipeline backed by Airtable. It converts public product pages into structured records, maps raw ingredient text to a standard ingredient list and prepares consistent product images.

1. Discover and scrape candidate pages with Firecrawl. Write progress and failure states back to Airtable so batches can resume.
2. Classify each URL as a product or non-product page, then extract brand, category and ingredient fields into a fixed schema.
3. Normalise raw ingredient text. Rules detect bundled products, product codes, percentage prefixes and several ingredients joined into one value.
4. Create an embedding for each cleaned ingredient name. An embedding is a numeric representation of the text. FAISS searches those numbers to find the closest entries in the standard taxonomy.
5. Send uncertain top matches through a separate verification step. Failed values are cleaned again and re-matched instead of being silently accepted.
6. Use fuzzy string comparison to identify likely duplicate ingredients that survived the earlier checks.
7. Download candidate product images, verify their contents, remove backgrounds and upload approved files to Cloudflare R2.
8. Run repair tools that compare R2 filenames with Airtable records and report missing or mismatched files.
9. Benchmark several language models on the same extraction set. Measure accuracy, success rate and cost per product before choosing a model.
10. Use asynchronous workers with explicit rate limits. Produce timestamped JSON and CSV reports at every major stage for checking and rollback.

A batch of 1,000 products would require 30 to 50 hours of manual extraction and ingredient matching. The pipeline completes the first pass in hours and leaves about three to five hours of exception review.

That represents about an 85% reduction in manual time. Each batch also produces a list of uncertain matches, missing images and failed records for targeted correction.