Every AI headshot product works roughly the same way under the hood, and once you know the shape of it, most of the weird results stop being weird.
You upload 10 to 20 selfies. Twenty minutes later you get 100 portraits back. Some look exactly like you. Some look like your cousin. One looks like you wearing a jacket you've never owned, in an office you've never been to, and that one is usually the most interesting failure.
Here's what's happening in between.
The pipeline has four stages
- Face detection and crop. Your uploads get run through a face detector, cropped square, aligned so the eyes sit on roughly the same horizontal line, and resized. Anything where the detector fails, or finds two faces, gets dropped.
-
Captioning. Each image gets a text caption, usually auto-generated, containing a rare trigger token. Something like
sks person, wearing a blue shirt, indoor lighting. - Fine-tuning. A small adapter is trained on top of a frozen base model so the trigger token means your face.
-
Generation. The adapter gets loaded, and the product runs its own prompt library against it.
sks person, corporate headshot, grey backdrop, softbox lighting, a few hundred times with different seeds.
Stage 3 is where all the quality lives, so that's the one worth understanding.
LoRA in one paragraph that isn't hand-wavy
A diffusion model has a few billion parameters. Fine-tuning all of them for one person's face would take hours on serious hardware and produce a multi-gigabyte file per customer. Nobody's running a business on that.
LoRA, Low-Rank Adaptation, sidesteps it. Instead of updating a big weight matrix W, you freeze W and learn two skinny matrices A and B whose product has the same shape. At inference you compute W + BA. If W is 1024x1024 and you pick rank 16, then A is 16x1024 and B is 1024x16. That's 32,768 trained numbers instead of 1,048,576, so about 3%.
These adapters get injected into the attention layers, which is where the model decides what a thing looks like rather than where it goes. Train for a few hundred steps, ship a file that's a few megabytes, load it in milliseconds.
The rank is the knob. Low rank, say 4 to 8, and the adapter doesn't have enough capacity to hold your specific face, so you get a generic person who vaguely resembles you. High rank, 64 and up, and it has enough capacity to memorise your training set wholesale, which sounds good and isn't.
Why 15 photos and not 500
This surprises people. More data is supposed to be better.
The problem is that the adapter learns everything your photos have in common, and it has no way to know which of those things you consider "your face".
Upload 15 selfies taken in the same week and 12 of them will have the same haircut, the same lighting from the same window, and the same three t-shirts. The model learns sks person means a face plus that lighting plus those shirts. Then you ask for a corporate headshot and it fights itself, because half of what it learned about you is a grey marl t-shirt in a bedroom.
That's overfitting, and on faces it shows up in a specific way: the generated images look great and they all look like the same photo. Same angle, same expression, same background tone. The model isn't generating your face in new situations. It's reconstructing your training set with slight variation.
Fifteen genuinely different photos beat 100 near-duplicates, every time. Different days, different rooms, different clothes, a couple of different angles.
The failure everyone hits: identity drift
The opposite failure is more common in the cheap tools. You get 100 clean, well-lit, professional portraits of somebody who is nearly you.
Two things usually cause it.
Undertrained adapter. The trigger token never fully bound to your face, so the base model's idea of "a person" is doing most of the work. You get a composite of you and the average face in the model's training data.
Prior preservation pulling too hard. To stop the adapter from destroying the model's general concept of "person", training usually mixes in generated images of random people alongside yours. Too much of that and your identity gets regularised away.
There's a measurable version of this. Take a face embedding model, generate an embedding for your input photos and for each output, and compute cosine similarity. Anything above about 0.65 reads as clearly the same person. Below 0.5 and most people looking at it will say "that's not quite you", even if they can't say why.
If a product isn't filtering its outputs on something like that score, you're doing the filtering by hand, which is what it feels like when you get 100 images back and 12 are usable.
What this changes about the photos you upload
The practical version, given all of the above:
- Vary the background more than the pose. The background is the thing most likely to get baked into your identity, because it occupies the most pixels.
- Include at least 3 different tops. Clothing is the second thing that gets baked in.
- One or two photos where you're not smiling. If every input is a grin, every output is a grin, and it will look like the same grin.
- Skip anything with a second face in it, even blurred in the background. Detectors get confused and the crop goes wrong.
- Skip heavy filters. Instagram-style colour grading gets learned as part of your face.
Roughly 15 photos, ideally from more than one occasion. The person who takes 15 selfies in one sitting in one room gets noticeably worse results than the person who digs 15 out of their camera roll from the last year.
Trying it without paying for it
If you want to see what the pipeline does with your photos before committing, most of the products have a free tier that runs a smaller version. You can put a handful of selfies through a free generator and see the identity-drift problem for yourself, which is a faster education than reading about it.
The paid versions of AI headshots mostly differ in how much they spend on stage 4, the generation and filtering, rather than on anything exotic in stage 3. Everyone's doing LoRA or something close to it. The difference is how many candidates get generated and how aggressively the bad ones are thrown away before you see them.
Which is also why the outputs vary so much between products at the same price. If you want to see the difference in practice, it's worth looking at BetterPic side by side with Aragon rather than trusting anyone's sample gallery, including ours. Sample galleries are cherry-picked by definition.
The one thing worth remembering
The model has no concept of "your face" as separate from "the pixels in your uploads". Everything consistent across your training images becomes part of your identity as far as the adapter is concerned.
So the quality of your result is decided almost entirely before you hit upload.
Disclosure: I work on BetterPic, one of the tools linked above.
Top comments (1)
The overfitting point matters a lot for non-ML readers. People expect identity consistency, but they do not want a model that only knows one exact lighting setup or expression. The practical quality bar is not “looks like me once,” it is whether the identity survives normal variation.