Sumus: AI Product Discovery for Sustainable Fashion

Sumus is an AI product-discovery engine for fashion. A shopper scans a garment and Sumus shows its ingredients, quality, safety and availability alongside an ESG footprint, then lets them compare sustainability across brands side by side. The aim is to put information about materials, ethics and environmental impact in front of people at the moment they are deciding.

We built the AI tool behind that experience: the scanning and recognition pipeline, the ESG scoring engine and the brand-comparison layer. This post explains how those pieces fit together and why we made the choices we did.

The stack

Sumus runs on a NestJS REST API with PostgreSQL and Redis, MinIO for images and files, and a Next.js web app alongside a React Native mobile app, deployed on Railway. Shopify integration connects product catalogues, and Stripe and NextAuth cover payments and authentication. For models, we used both Ollama for self-hosted open models and OpenAI for hosted ones.

The web and mobile clients talk to the same API, and the AI pipeline sits behind it rather than inside either client. That keeps model choice, prompts and scoring rules on the server, where they can change without an app release, and it means a shopper gets the same answer whether they scan on their phone or look a product up on the web.

From a scan to a product

The first job is recognition: given a photo or a scanned label, which product is this? That breaks down into a pipeline rather than a single model call:

1. Capture — the image goes to object storage and the request gets an ID, so every step after it is traceable.
2. Extract — read what is on the label or tag: brand, composition, care and origin text.
3. Match — resolve that to a known product or product family in the catalogue, including those coming in from Shopify.
4. Enrich — attach what is known about materials, manufacturing and brand practices.
5. Score and explain — compute the ESG view and the plain-language summary the shopper sees.

Splitting the work this way means each step can be tested, measured and improved on its own. When a result is wrong, you can see which step got it wrong, which is the difference between a prototype and something a team can maintain.

Each step should also report how confident it is. A clean label that matches one catalogue product goes straight through. A blurry photo or an ambiguous match is better shown to the shopper as a short list of candidates to pick from, rather than a confident wrong answer. Results for products that have already been resolved are cached, so repeat scans of popular items are fast and cost nothing in model calls.

Scoring is rules, explanation is AI

The most important design decision was where not to use a language model. ESG scores have to be consistent: the same garment must get the same score today and next month, and two shoppers comparing brands must be comparing like with like. A language model asked to "rate the sustainability" of a product will not give you that.

So the scoring engine is deterministic. It takes structured inputs — material composition, certifications, origin, brand-level data — and applies defined weights. Models do the parts they are good at: reading messy label text, matching products, and turning a score breakdown into a short explanation a shopper understands in seconds. Every explanation is generated from the score's actual inputs, so the words cannot drift from the number.

Scoring rules also need versioning. When the weighting changes — because better data becomes available or the methodology is refined — each stored score should record which version produced it. That makes it possible to re-score the catalogue deliberately and to explain why a product's score moved, instead of numbers shifting silently underneath users.

Why both Ollama and OpenAI

Running both self-hosted and hosted models is a pragmatic choice, not an ideological one. High-volume, well-defined tasks such as extraction and classification are good candidates for smaller open models served through Ollama, where cost is predictable and data stays on infrastructure we control. Harder reasoning and generation tasks can go to hosted models where quality justifies the cost.

Keeping both behind one internal interface means a task can move between them as models improve, without touching the rest of the system. It also makes evaluation straightforward: run the same labelled test set through two models and compare, before switching anything in production.

Comparison that does not mislead

Brand comparison is where sustainability products most easily become misleading. Data coverage is uneven: some brands publish detailed supply-chain information and others publish almost nothing. A comparison that quietly fills gaps with guesses rewards the brands that disclose least.

We designed the comparison layer to make coverage visible. Missing data is shown as missing, not as an average, and the shopper can see which parts of a score rest on verified inputs. It is a less tidy interface, and a more honest one.

The same applies to sources. Each ESG input carries where it came from — a certification, a brand disclosure, a product label — so that when a brand updates its information, the affected scores can be recalculated and the change is traceable. For a product whose whole value is trust, being able to answer "where does this number come from?" matters more than any individual feature.

Lessons for AI product teams

• Build a pipeline of small, testable steps rather than one large prompt.
• Keep anything that must be consistent — scores, prices, eligibility — in deterministic code.
• Use models to read and explain, and generate explanations from the real inputs.
• Put self-hosted and hosted models behind one interface so you can switch per task.
• Show missing data as missing.

Building an AI product?

Sumus is a good example of AI development where the model is one part of a larger system. If your product depends on retrieving and explaining structured knowledge, our RAG development guide is a good next read. See the Sumus project, or get in touch.