I Outsourced Getting Dressed to an AI (and Accidentally Built Three of Them)

This is Part 1 of a series on building Wardrobe AI, a personal app that
recommends what to wear. This post is the map: what it does and how it's put together. Later posts go deep on the recommendation math, the feedback loop, and the two agents running underneath. Code samples are kept light here; the details get their own posts.

Everything in this series is a running app with public code and history:
github.com/JiamanBettyWu/mise (the repo is named mise, as in mise en place, but I'll call it Wardrobe AI
throughout). Later posts link the specific files and pull requests as they come up.

It started in front of my closet

A few months ago I was standing in front of my closet before work, running late, staring at a rack that was visibly full of clothes, and thinking: "I have nothing to wear."

Which was obviously false. I had plenty to wear. That was the actual problem — the rack was overwhelming. Too many options, no obvious answer. What I was experiencing wasn't a shortage of clothes. It was decision fatigue, the small daily tax of having to assemble a reasonable outfit from scratch every single morning before my brain is fully online.

So I had a second thought, the one that actually mattered: I should just outsource this decision.

That's the entire origin of the project. Not "wouldn't it be cool to use AI for fashion," but something more like "I have a recurring, low-stakes decision I'd happily hand to a machine, and I have enough context (my actual clothes, the weather, my calendar) to make a decent suggestion possible."

What it actually does

The app is a personal wardrobe catalog with an AI stylist bolted on. The setup cost is one-time: I photograph each piece of clothing, and a vision model tags it automatically with type, color, formality, season, fabric, even a rough warmth rating. I review the tags, fix anything it got wrong, optionally add a note, and save. After that, the catalog just exists.

From there, the day-to-day value shows up in two channels.

A push channel: the morning email. Every morning I get an email with an
outfit suggestion (sometimes a few, depending on what my calendar looks like that day), chosen for the actual weather and reasoned out in a sentence or two. I don't have to open anything; it just arrives.

A pull channel: on-demand generation in the app. The email is great for the default "what do I wear to the office today" case, but some days are not the default when I have a special event. So the app's Today's Outfit page has a notes box ("Describe theoccasion, then click Generate") where I can ask for something specific: dinner out, an interview, cold rain, a day on my feet. It generates suggestions for that context, and if the first batch doesn't land, I hit Regenerate.

There are two more capabilities I'll only mention here and unpack later: a trip packing planner (give it a destination and dates, get a weather-appropriate packing list, and if you're missing something for the trip it'll even find things to buy), and a style learner that quietly studies which suggestions I liked and distills durable preferences over time.

What success actually looks like: a warm start, not a verdict

It's worth being clear up front about what this app is for, because it's easy to assume the goal is an AI that dictates your outfit while you obey. That's not it. It isn't meant to replace my judgment or have the final say on what I wear. It's meant to do two narrower, more useful things.

First, it gives me a warm start. Cold start is the "I have nothing to wear" paralysis — a blank slate and no idea where to begin. A warm start is being handed a plausible draft to react to. It's the difference between a writer facing an empty page and a writer editing a rough draft, and the second is so much easier. That alone removes most of the daily decision fatigue,
which was the whole point.

Second, it nudges me toward outfits I wouldn't have assembled on my own. Left to myself I reach for the same handful of safe combinations; a system with a little randomness in it occasionally pairs things I'd never have tried, and sometimes that's the best part. This is the classic exploration vs. exploitation trade-off, balancing the reliable favorite against the worthwhile gamble, and it's a big enough idea that it gets real treatment in Part 2.

Both goals share a freeing consequence: the app doesn't have to be right, only useful. That tolerance is what lets me lean on randomness and learning instead of chasing an impossible standard of correctness.

The twist: this isn't one AI, it's three

When I started, I assumed "the AI" would be a single thing: feed it my clothes and the weather, get back an outfit. What I ended up with is three different AI systems, because the three jobs turned out to have different shapes.

1. The daily recommender. The thing behind the morning email and the on-demand button. It's actually the least fancy of the three under the hood: plain, straight-line Python that prepares the right inputs and makes one call to the model. No agent framework, no graph. (Most of its cleverness is in how it chooses which clothes to even show the model, which is Part 2.)

2. The trip planner. Packing is a bigger problem than "what do I wear today," because it can spawn follow-up work: if you're missing something the trip needs, the system should go find it. That "do a step, then maybe do a different step depending on what you found" shape is what people mean by an agent: a program that decides its own next move at runtime. I built this one on LangGraph, a library for wiring those steps into a graph. (Part 4.)

3. The style learner. Once a week, in the background, a second LangGraph reads my whole history of thumbs-up/thumbs-down feedback and tries to distill it into durable preferences ("leans toward neutral basics," that sort of thing). Same tool as the trip planner, completely different job, and, it turns out, completely different risks.(Part 5.)

If you only remember one thing from this post, make it this: the three systems share a single design principle, and that principle is the spine of everything that follows.

The one principle: stochastic for preferences, deterministic for physics

Early on I kept running into the same fork in the road: should this piece of behavior be a hard rule, or a soft nudge? After enough of those decisions I noticed I was answering them all the same way, and the rule generalized:

Use stochastic weights for preferences, and deterministic logic for physics.

Let me unpack the two words. Stochastic just means "involving randomness": behavior that's probabilistic rather than fixed. Deterministic means the opposite, the same input giving the same output every time, no dice rolls.

The insight is that the app's inputs split cleanly into two kinds:

• Preferences are soft, fuzzy, and accumulate over time. "She probably doesn't love this shirt." Getting one of these wrong on any given day is harmless; remember, it's a warm start. So preferences belong in the stochastic parts of the system: things like randomized, weighted sampling, where the wrongness washes out over time and a little randomness is actually a feature (it's what keeps suggestions varied).

• Physics are hard constraints about the real world. "It is 4°C andraining." There's nothing fuzzy about that. Recommending shorts isn't a charming bit of variety; it's just broken. So physics belongs in the deterministic parts: explicit filters and gates that run the same way every time and that I can debug when they misfire.

The reason this matters, and the reason I made it the organizing principle, is that mixing the two up creates the worst kind of bug. Encode a hard physical constraint as a soft probabilistic nudge and you get failures that are unreproducible: it recommends a parka on a hot day one time in twenty and you can never quite catch it. Encode a soft preference as a rigid rule and the app becomes brittle and samey, suggesting the same "safe" outfit forever. Keeping the two in separate homes is what keeps the system both lively and trustworthy.

You'll see this principle cash out concretely in every remaining post: in how the recommender randomizes which clothes it considers but hard-filters weather-absurd ones (Part 2), in why thumbs-feedback adjusts probabilities rather than banning items outright (Part 3), and in how the two agents separate "decide what to do" from "do it reliably" (Parts 4 and 5).

A note on the stack (because cheap was a constraint)

This is a personal app for exactly one user (me), and I wanted it to cost roughly nothing to run. That shaped a lot of pragmatic choices worth a quick mention: a Python/FastAPI backend, a React frontend, Supabase for the database and photo storage, and Anthropic's Claude models for the AI parts. The one choice I'm quietly proud of is the morning email: instead of paying for a hosted scheduler, the daily job runs on a free GitHub Actions cron, the same automation most people only use for running tests. All in, the whole thing costs me somewhere around one to two dollars a month, almost entirely model usage. That number comes from a dashboard, not a guess: the app meters its own model calls in a small ledger table and shows me the running cost. The constraint of "free" turned out to be a good forcing function for simple architecture.

Where this series is going

That's the map. Five core posts, each opening on the actual moment or annoyance that drove the work, then the engineering decision it led to:

1. This post: the overview and the physics-vs-preference principle.
2. The recommendation engine: how it avoids suggesting the same outfit four days in a row (a thing it absolutely did, at first) without ever putting me in a parka in July.
3. Closing the feedback loop: why a thumbs-down is trivial to record and surprisingly hard to learn from correctly.
4. Going agentic with LangGraph: the trip planner, and what "agent" actually buys you over a plain function.
5. LangGraph again, for learning: the weekly style learner, the same tool doing the opposite job, and why a system that quietly changes its own behavior needs very different guardrails.

And it won't stop at five: the app is alive, so the series will follow it. Whenever a new piece of the system is worth writing about, it becomes a Part 6, a Part 7, and so on. (The next candidate is already queued: how I built an eval loop to measure whether the recommender was actually getting better, instead of just feeling like it was.)

Want to poke around?

The code is public at github.com/JiamanBettyWu/mise. If you enjoy seeing how a project actually gets built (as opposed to how it looks once it's tidied up for a blog post), the repo carries two artifacts I kept while building: a session-by-session journal of what shipped and why (SESSIONS.md, and an unpolished, chronological log of every gotcha that bit me (LEARNINGS.md. Between them, they're the honest version of this series. Each later post links the specific files and pull requests it describes.

If the idea of handing a small daily decision to a slightly-unreliable machine sounds interesting, the next post is where it gets concrete. See you there.