DEV Community: PEPPERCORN

[Day 11] I turned my cat into anime art — and the AI drew a human girl instead. One photo through IPAdapter pulls it back to a cat

PEPPERCORN — Thu, 04 Jun 2026 04:13:23 +0000

Intro

Day 11! Back to cats 🐱

I took one photo of my cat (a black-and-white tuxedo boy) as a reference and had AI restyle him into anime, ukiyo-e, oil painting, and more.

The goal: change only the style while keeping "my cat" recognizable. But left alone, the AI started drawing humans instead of a cat. Here's what I did, step by step.

What I used: my home AI machine (DGX Spark) + an image-generation tool (ComfyUI) + one photo of my cat.

The reference is this one photo

A tomcat my family looks after for me, with yellow eyes and a slightly grumpy look.

Love that face. I'll turn him into various styles while keeping him recognizable as "my cat."

First, anime from text alone → a human

I started with no photo, just text: "a tuxedo cat, anime key visual." I clearly said cat.

Here's what came out. …A human girl.

Black hair, white collar. My cat's tuxedo pattern (black body, white chest) turned straight into clothing.

Next, I added the reference photo → still human

So I hand over the cat photo as a visual reference. The tool that applies it is IPAdapter.

What's the reference-photo trick (IPAdapter)? A tool that lets you pass a reference image, separate from the text prompt, and say "make it look like this." It's what preserves my cat's colors and face.

Surely this makes it a cat… nope. Still human.

And this habit wasn't limited to anime. Ask the same anime-style model for ukiyo-e or oil painting, and you still get anime-ish humans. It hijacks not just the subject (the cat), but the art style too.

Left: an "ukiyo-e" that's really an anime woman in a kimono. Right: an "oil painting" that's an anime woman in a tuxedo. Both are "humans painted in the cat's colors."

I tuned the settings → finally a cat

On top of the photo, I turned up its strength and added "don't draw humans" to the negatives (details below). That's when it finally became a sitting cat.

Why does it turn into a human?

Two reasons, as far as I can tell.

One: anime-savvy models tend to draw people, girls especially. Even with "cat" in the prompt, they drift toward a human if you let them.

Two: my cat's pose. He sits bolt upright, almost like a person, so the harder you push the reference, the more that upright posture rides along — tipping toward an "anthropomorphized" cat. The pop-art piece later is exactly that leftover.

Cyberpunk flipped to a cat with the photo alone

The interesting part: whether the photo alone was enough depended on the model. Anime was stubborn and needed tuning, but cyberpunk became a cat just by adding the photo.

Left (no reference): a human man in a neon city. Right (with reference): a cat with glowing ears.

I didn't change a single character of the prompt — the photo being there or not is the only difference between human and cat.

The styles that came out

Here's the gallery after the human problem was fixed — all with the reference photo, my cat as the base.

Top row, left to right: anime, ukiyo-e, oil painting (Van Gogh-ish), stained glass. Bottom row: cyberpunk, 3D (Pixar-ish), pop art.

"Likeness" and "style" are a tug-of-war

The oddly real 3D Pixar one shows this little trade-off nicely.

Left (no reference): a cute 3D cat, but "some cat." Right (with reference): it becomes my cat's face, but the 3D look washes out into basically a real photo.

Weaken the reference and the style shows but it's a different cat; strengthen it and it's my cat but the style fades. Finding that grip per style is what the tuning really is.

The boss I couldn't beat: storybook watercolor

"Gentle storybook watercolor" was the one style I never got to be a cat. Here's the result of seven retries.

A human, then somehow two cats, then a cat-eared girl holding a cat. "Single + watercolor + cat" wouldn't line up. Lower the reference → human; raise it → two cats. "Storybook" must be soaked in human imagery. Carrying this over.

The details

Here are the details.

The reference-photo mechanism (IPAdapter)

I added a custom node called ComfyUI_IPAdapter_plus to ComfyUI. It lets you hand over a reference image as a "visual guide," separate from the text prompt.

Model used: ip-adapter_sd15 (44.6MB, from h94/IP-Adapter)
The part that reads the image features: CLIP-ViT-H (reused an existing one)
The reference photo is cropped to a 768px square before handing it over

A number called the "reference strength (weight)" controls how closely it mimics. I moved between roughly 0.7 and 0.85 depending on the style.

What I did to suppress the "human" problem

I started at weight 0.7 plus words like "key visual" and "big eyes," which strongly invited humans. Three fixes:

Raise the reference strength to 0.85
Add human, girl, person, 1girl, humanoid to the "things I don't want drawn" list
Strip human-summoning words from the request and emphasize tuxedo cat, full body, animal

That corrected anime, ukiyo-e, and oil painting into cats. One catch: the phrase "tuxedo cat" itself tends to put an actual tuxedo (a suit) on the cat, so it cut both ways.

The base models I used

I switched the underlying image model by style.

Anime / illustration: AnythingV5
Realistic / 3D: Realistic Vision V6
Plain base: SD 1.5 (base)

When storybook failed, switching to the plain base gave a real cat but weak watercolor feel, and raising the strength split it into two cats — a real bind. The base model's "habits" matter a lot.

Common generation settings

Across all styles: 768px, 30 steps, sampler dpmpp_2m karras, cfg 7, seed fixed at 110011. I only varied the text request and the reference strength, keeping everything else equal for a fair comparison. Generation is fired at ComfyUI from a small script I wrote.

Next up

Next time it's cats again — and this time I'm planning video generation 🐱

100ExperimentsWithDGX #LocalLLM

[Day 10] Building my own personal weather officer AI, and teaching it my body's sense of cold over the next 100 days

PEPPERCORN — Mon, 01 Jun 2026 01:31:58 +0000

Intro

Day 10!

This time I'm starting a longer-running experiment. Meet the "weather officer AI" — a bot that texts me every morning saying "wear this today." The plan is to build a weather assistant that's tuned to me.

What I'm building today is just v0.1 (the very first version). From here through Day 100, I'll keep teaching it "too cold / just right / too warm" every morning, so it gradually learns my preferences. The experiment is: how smart does it get after 100 days?

What I used: my home AI machine (DGX Spark) + free weather data + a phone messaging app (Telegram)

Today's task

What I wanted

I live somewhere with a big daily temperature swing, and "what do I wear today?" is a small but real daily headache. Weather apps tell you the temperature, but whether I feel cold is a different question.

So the starting point was: can I build a clothing AI that's tuned to "how I feel," not just "the temperature"?

Approach

I kept the design dead simple.

Every morning at 7, automatically fetch today's weather
Decide "this morning's outfit" from the apparent temperature and push it to my phone
I just tap back "cold / just right / warm"
As these feelings pile up, the AI learns "this person runs cold" and corrects its suggestions

The goal this time

Not a "perfect forecast AI," but a "routine I can actually keep up every day." The smarts get grown over the next 100 days. Today is just laying the rails.

📊 How much does the temperature actually move in a day?

Before building anything, I pulled a week of apparent temperatures for where I live and graphed it.

A beautiful zigzag. Every day repeats "cold in the morning → way up by midday → down again at night." The average daily swing is 13°C, and on the biggest day it moved 20°C.

So I narrowed the suggestion down to "one outfit, at 7 a.m., matched to the apparent temperature at that hour." I record just once in the morning too. To keep something up for 100 days, simplicity matters most.

The AI doesn't learn the temperature swing itself. But by deciding to "focus on the morning," the suggestion and the feedback line up in time, so later I can cleanly check "was the morning suggestion right?"

🔧 How it works (the morning loop)

The finished weather officer runs on this loop.

At 7 a.m. it grabs the weather, decides the outfit, and pushes a notification. I tap back my feeling, and that gets recorded. Once those records pile up, step 5 — learning "you run cold / warm" — kicks in, and the suggestions gradually become mine.

Here's what the actual notification looks like.

☀️ Weather Officer AI — good morning

👕 This morning's outfit: long sleeves

This morning feels like: 13°C (highs up to 20°C today)
Rain: 4%  /  Wind: 13 km/h

How does it feel this morning? ↓ tap to tell me
   [🥶 cold]  [😊 just right]  [🥵 warm]

It suggests one outfit, but adds "highs up to 20°C today" so I can decide whether to throw on a layer myself. Tapping a button changes it to "✅ recorded," and the feeling is saved to my home AI.

The notifications go through the messaging app I already use (Telegram).

🛠️ The details

Below are the specifics.

The weather data

Weather comes from Open-Meteo, a free weather API. No API key needed, historical data available, and commercial use is OK (CC BY 4.0) — very generous.

I mostly use "apparent temperature" — not the raw air temperature, but a number adjusted for wind and humidity to reflect how it actually feels, which is better for deciding what to wear. I take the average apparent temperature from 7–9 a.m. as "this morning's feel." The coordinates stay only on my home machine; I don't write the specific location in the article or the code.

The clothing rule

A plain rule that splits apparent temperature into 7 bands, each mapped to an outfit (e.g. 13–20°C → long sleeves, 20–26°C → short sleeves).

There's one "personal offset" number baked in. It's zero for now (v0.1). Going forward, if "cold" keeps coming back, I'll push the offset negative so the same temperature suggests warmer clothes — growing it from the feedback.

The notification and buttons

The notification uses a Telegram "Bot" (a thing that sends messages automatically), built with the python-telegram-bot library to:

send a message at a fixed time every morning at 7
attach three buttons below it and record which one is pressed

This bot sits waiting inside my home AI machine and fires in the morning.

The shape of the feeling log (the 100-day foundation)

Each record is one line per day, with these fields:

date
that day's forecast (morning/daytime apparent temperature, wind, rain chance)
the outfit the AI suggested
my feeling (cold / just right / warm)

Kept in this consistent shape, I can later graph "monthly hit rate" or "my personal bias." I'll report this trend in the Day 21 / 36 / 54 / 74 / 87 check-ins.

Keeping it running

So the 7 a.m. notification reliably fires, I set the bot to launch automatically when the machine boots (systemd). Even after a reboot it comes back on its own, and the morning notification keeps going.

The feeling log and the notification settings (the Telegram token, etc.) are all stored only on my home machine — none of it goes anywhere external like GitHub.

The 100-day growth plan

I'll keep growing this weather officer across the series — it'll pop up here and there.

Milestone	What happens
Day 10 (now)	v0.1 done; the recording loop starts turning
Day 21	First "my personal bias" report from 11 days of data
Day 36	Graph the monthly hit rate and take a look
Day 54 / 74 / 87	Mid-reviews: seasonal changes in feel, how the correction works
Day 100	The 100-day accuracy trend and the finished version

Just tap a button every morning. I'm curious how "mine" it'll feel after 100 days.

Next time: Day 11

Next time it's a hard pivot back to cats 🐱 I'll convert photos of my cat into picture-book, anime, and photorealistic styles — the theme being whether I can keep "that's-our-cat-ness" while changing only the style.

LocalLLM #100ExperimentsWithDGX

[Day 9] A local Japanese sentiment AI (BERT) read 8 years of a LINE chat, and the ups and downs surfaced from numbers alone

PEPPERCORN — Fri, 29 May 2026 22:39:10 +0000

Intro

Day 9. Today is less about model internals and more of a personal experiment: have a local AI analyze the entire chat history with one LINE friend. (LINE is the dominant messaging app in Japan.)

When I exported it, 8 years were sitting there — from the very first message to today. It started, we talked a lot, it went quiet for a while, then picked up again. That whole arc is in there.

Because the content is what it is, nothing left my machine: everything ran locally on my DGX Spark.

What I used: my home AI box (DGX Spark) + a Japanese sentiment model (for tone) + a bigger local model (to guess events from numbers).

Today's setup

What I wanted to do

Re-reading 8 years of messages one by one isn't realistic. So instead of reading the content, I looked only at the "shape" of the conversation — when, how much, and in what tone we talked.

Concretely:

monthly message volume
the trend of tone (positive / negative)
then asking an AI to find "when something big happened"

Heads-up (the result)

From message counts and tone alone, the 8-year arc came out clearly on a chart. Started, went quiet, came back — the flow was visible without me re-reading a thing.

🔧 Pipeline

LINE chat export (text)
        │
        ▼
 1. Parse: split each message into {datetime, who, type, text}
        │   (from here on, message text never leaves the machine)
        ▼
 2. Aggregate: monthly counts, time-of-day, reply gaps
        │
        ▼
 3. Tone scoring: classify each of 66k messages pos/neu/neg
        │
        ▼
 4. Turning-point detection: from sudden changes in the numbers
        │   + also show ONLY the numbers to a bigger AI and ask it to guess
        ▼
 5. Answer check: compare against the real timeline

You can export a LINE chat as text from the chat screen ("send chat history").

Data size:

Item	Value
Span	~8 years 2 months
Total messages	87,621
Text messages	66,329
Stickers	15,605
Photos	3,982

15,605 stickers… that's a lot.

The two AIs

Step	Model	What it does	What it sees
3. Tone	Japanese sentiment model (`koheiduck/bert-japanese-finetuned-sentiment`)	scores each message pos/neu/neg	66k message texts (scores averaged per month)
4. Turning points	a bigger local model (`Qwen2.5` 72B)	guesses "what happened to these two?"	only the per-month table of counts + tone scores (no conversation, no words)

Both run locally on my own machine.

📊 Results

The 8-year arc of volume and tone

This chart is the highlight. Top: monthly message count. Bottom: tone (up = positive, down = negative). The x-axis is months since the conversation started. (Axis labels are in Japanese.)

Plotted, it isn't a steady climb or a flat line — it splits cleanly into "chapters": ramp-up → an 8-month silence → a second peak → a stable plateau. Four phases, at a glance.

Tone has two peaks of about +0.6, around the start and around when things resumed (overall mean ≈ 0, slightly negative in the later years). The interesting part: in the month before the silence, tone had already dropped to −0.1. The mood dimmed before the volume did.

There are two dips into negative tone. The one before the silence was an "omen." The other is the recent years — not an omen, but the effect of logistics-y messages ("what time are you home?") piling up.

💡 Mini-note: how is "tone" turned into a number?
The scoring is done by a Japanese sentiment model. Roughly:

pre-trained on lots of Japanese text labeled positive / negative

judges with context, not just by spotting keywords

returns a probability of "positive-ness" / "negative-ness" per message

I used the difference as a per-message score

What kinds of messages scored how?

A few actual judgments (short, name- and place-free one-liners):

Message	Verdict
「楽しかったね！」 (that was fun!)	Positive
「これめちゃうまい」 (this is so good)	Positive
「おはようございます」 (good morning)	Neutral
「もうお家？」 (home already?)	Neutral
「全く集中できない」 (can't focus at all)	Negative
「それは悔しいな、、」 (that's frustrating…)	Negative
(a long trip-planning message)	Neutral
(a snappy one-liner sent in a huff)	Negative

Plain happy lines score positive; logistics ("good morning", "home already?") score neutral; tiredness or irritation scores negative. Even long, businesslike planning messages lean neutral.

Mornings are when we talk

Message density by weekday × hour (brighter = more).

A clear concentration at 7–9 a.m.!

Could the AI guess the turning points?

First, the simple method: mechanically pick the points where message volume jumped or dropped, then check against the real timeline.

Real event	Auto-detected timing
When it started	exact match
When it went quiet	exact match
When it resumed	exact match
When it got lively again	a few months off
A big life milestone	hard to detect (barely shows in counts)

Sharp volume changes were nailed. But "a big life milestone" got missed. So I showed the same numbers to the bigger local model and asked "what happened?" — and got back:

"around when it started" → roughly matches
"a stretch of going silent" → matches the quiet period
"a major life change" → almost exactly before the real milestone

Rather than hunting for a single spike, it reads the whole sequence of numbers as a "flow," so it could pick up even an event that barely moves the counts.

💡 Takeaways

1. Volume + tone alone reveal the arc

Counts and tone were enough to see the 8-year shape. Silence marks the quiet stretch; a surge marks the resumption — straight off the chart.

2. A local model reads a story out of numbers

Given only monthly numbers, the model inferred even a barely-visible event ("something big around here"), and it lined up with reality. It connects scattered points into one flow.

3. A "negative" tone doesn't mean a bad relationship

The slight negative lean in later years isn't about getting along badly. Logistics messages ("what time are you home?") just don't score high. Low score ≠ trouble. It isn't that sentiment analysis is poor — the scores need to be read together with context.

🛠️ Technical details

Parsing & aggregation

LINE export format is a date header plus time<TAB>name<TAB>text. Multi-line messages (4,987 of them) are merged back into the previous message.
Speakers normalized to "A / B" by message count (no real names in anything public). Temporary group members and system lines excluded.
Messages tagged by type (text / sticker / photo / call / unsent…). Tone uses text only; volume counts use all types.
Aggregation and plotting in Python (pandas / matplotlib).

Tone (sentiment)

koheiduck/bert-japanese-finetuned-sentiment, a 3-class (pos / neu / neg) Japanese model.
66,329 texts scored on GPU in batches; per message I take P(pos) − P(neg) in [−1, +1], then average per month.

Turning-point detection

Rule-based: long near-zero stretches (silence), large month-over-month surges, and tone peaks — all from numbers only.
Plus: the per-month table of counts + tone scores fed to a bigger local model (Qwen2.5-72B via ollama) to guess events. No message text was given.
Real event dates were kept in a local note only, used for annotation and the answer check.

Privacy

Every file containing message text (raw export, parsed data, scores) stays in a non-public folder.
Only aggregate numbers and charts are published. The chart x-axis is relativized to "months since the conversation started," hiding actual dates.
Apart from a few short, name- and place-free one-liners shown as scoring examples, no conversation content, real names, specific dates, or long text appears in the article or charts.

Tomorrow: Day 10

Weather forecasts say one temperature, but everyone feels it differently. Same degrees, different "do I need a coat?" So next I'm building my own personal "weather officer" AI: from past weather data, it'll tell me each morning something like "coat + beanie today." Over the next 100 days I'll teach it my own sense of cold — the start of a longer project.

100ExperimentsWithDGX #LocalLLM

[Day 8] Pushing Looped Transformers Beyond Addition: OpenMythos on Bracket-Matching Depth

PEPPERCORN — Fri, 29 May 2026 06:27:04 +0000

[Day 8] Pushing Looped Transformers Beyond Addition: OpenMythos on Bracket-Matching Depth

Intro

Day 8!

A direct follow-up to Day 7: same OpenMythos-style mini model (3.4M params), same training pipeline, one task change — multi-digit addition swapped for nested-bracket parsing. The goal was to ask two follow-up questions Day 7 left open:

Does the "training-time loop count is the peak" finding generalize across tasks?
If we increase the structural complexity of the input (deeper nesting), does inference-time loop count start to matter?

Tools used: my home AI machine (DGX Spark, GB10) + OpenMythos (PyTorch reconstruction of the rumored Claude Mythos architecture) + synthetic bracket sequences.

Today's setup

Why bracket matching?

Day 7's task was 2-5 digit addition. Addition tests "carry propagation from low to high digit" — a fundamentally local, left-to-right state update. To probe whether looped depth helps with a different kind of structural reasoning, I wanted a task where:

The output depends on left-to-right state tracking (rules out attention-based global aggregation shortcuts).
The task admits an explicit notion of depth I can vary as a controlled difficulty knob.

Bracket matching fits both. The standard linear-time algorithm is push-on-open / pop-on-close with a stack. A model that has internalized that algorithm should scale gracefully with depth — and one that hasn't will visibly fall over.

Task: first-break-position prediction

Input: a string of ( ) [ ] { } characters, terminated by =.
Output: the left-most position at which the bracket structure breaks, as 2 digits, terminated by $. If the sequence is balanced, output --$.

Examples:

((()))=          → --$       (balanced)
([{}])=          → --$       (balanced)
([)]=            → 02$       ()` at position 2 doesn't match preceding `[`)
(()(=            → 04$       (stack non-empty at end of string, position = len)
))=              → 00$       (close on empty stack at position 0)

The "break position" is defined by a stack parser scanning left-to-right:

Close bracket whose type ≠ stack top → return that close position.
Close bracket on empty stack → return that close position.
End of string with non-empty stack → return len(s).
Otherwise balanced → return -1 (output --).

Why not just binary balanced / imbalanced?

That was the original plan. A first smoke run with T/F output saturated to 100% accuracy across all depths (up to 10) by step 4,000. There are too many shortcut signals — length parity, open/close count, etc. — for a transformer to learn the actual stack algorithm.

The first-break-position output forces the model to commit to a specific character position, which can only be answered by tracking state left-to-right. After this change, smoke results at 5,000 steps showed clean depth-dependent difficulty (d=2: 100%, d=20: 71%) and the loss had room to keep dropping. That's the signal I needed to study loop-count behavior meaningfully.

Difficulty knob: depth

I trained and evaluated across depths {2, 4, 6, 8, 10, 12, 16, 20}, with pair count capped at min(2 * depth, 50) so the 2-digit position output stays in range. Balanced and imbalanced sequences mixed 50/50; imbalanced sequences generated by deleting a close (30%), deleting an open (30%), or substituting a bracket (40%).

Architectural changes from Day 7

Minimal — only what the new vocab and longer sequences required:

	Day 7 (addition)	Day 8 (brackets)
vocab_size	16	20
max_seq_len	32	128
max_loop_iters (train)	4	4
Difficulty axis	2-5 digits	depth 2-20
Answer tokens	1-6 (digits + `$`)	2 + `$`
Total params	3.39M	3.39M

Same MythosConfig template otherwise. Same hyperparameters (AdamW, max LR 3e-4, warmup 2000, cosine decay, 30k steps, fp32, 4 seeds in parallel).

Headline finding

The Day 7 "peak at training loop count" finding generalizes. With training max_loop_iters=4, accuracy peaks at exactly T=4 again, and decays in both directions — including at every depth I tested.
But the peak height is much lower. Best accuracy was 66% at depth 2; depth 20 caps at ~36%. Day 7 hit 100% at d=5; brackets at the same parameter budget plateau dozens of points short.
Inference-time loop extrapolation does NOT improve deep-nesting performance. The hypothesis "deeper inputs benefit from more loops" did not reproduce — T>4 hurts at every depth, just as in Day 7.
Fixed-point reproduced, slightly later. Cosine similarity between consecutive hidden states reaches ~0.95 by T=3 and ~0.99 by T=4 — a step or two later than addition (which got there by T=2).

🪢 The task in pictures

Input:  ( ( [ ) ] ) =
Pos:    0 1 2 3 4 5

stack walk:
  pos 0: '(' → push '('             stack: ( 
  pos 1: '(' → push '('             stack: ( (
  pos 2: '[' → push '['             stack: ( ( [
  pos 3: ')' → top is '[', mismatch!  → first break at position 3

Expected output: 03$

The interesting thing about this task vs. addition: the answer can be anywhere from 0 to ~40 depending on the input, and the model has to commit to a specific integer. There's no global-aggregation shortcut — you have to walk left-to-right and remember what you've seen.

🔧 Pipeline

OpenMythos tiny (3.4M params, same as Day 7 modulo vocab + max_seq_len)
  ↓
Train 4 seeds in parallel, 30k steps, fp32 on DGX Spark (GB10)
  ↓
Experiment A: greedy autoregressive accuracy
              loops ∈ {1, 2, 4, 8, 16, 32}  ×  depth ∈ {2, 4, 6, 8, 10, 12, 16, 20}
  ↓
Experiment B: cosine similarity between consecutive hidden states
              ⇒ does the recurrent block reach a fixed-point?
              ⇒ does the fixed-point timing depend on depth?
  ↓
Compare against Day 7 (digits) along the same axes

Training throughput note (vs Day 7)

Day 7's 4-seed parallel training was fast because max_seq_len=32 left the GPU underutilized per process. With max_seq_len=128, a single process already saturates the GB10 — 4-seed parallel drops per-process throughput from ~60K tok/s to ~12.8K tok/s (a -79% per-process penalty). Aggregate parallel throughput is actually ~15% slower than sequential 4-seed.

I let it run in parallel anyway because it was overnight and I had no other DGX usage scheduled. Worth noting for anyone planning similar replications: longer sequences kill the "free" benefit of multi-seed parallelism on a single GPU.

GPU draw stayed at 51W / 72°C / 95% utilization throughout — comfortable enough to leave running.

📊 Results

Experiment A: accuracy heatmap

Mean exact-match accuracy across 4 seeds, 500 eval samples per condition:

Inference loops	d=2	d=4	d=6	d=8	d=10	d=12	d=16	d=20
1	0.11	0.05	0.03	0.02	0.01	0.01	0.01	0.02
2	0.32	0.20	0.13	0.08	0.08	0.08	0.07	0.07
4 (train)	0.66	0.56	0.50	0.45	0.44	0.41	0.41	0.36
8	0.58	0.56	0.51	0.47	0.46	0.44	0.39	0.34
16	0.55	0.51	0.44	0.41	0.40	0.38	0.36	0.32
32	0.55	0.48	0.42	0.40	0.39	0.37	0.36	0.31

Observations:

Peak at T=4 across every depth column. Day 7's "loops help only in a narrow window centered on training" finding generalizes: no depth I tested has its best accuracy at T≠4.
Depth scaling is graceful but the ceiling is low. Going from d=2 to d=20 at T=4, accuracy degrades smoothly (0.66 → 0.36), but the absolute numbers stay far from saturation.
The "deeper input ⇒ more loops" hypothesis does not hold. I'd hoped to see T=8 or T=16 begin to dominate at d=20, indicating inference-time scaling could rescue depth. Instead, every depth column peaks at T=4 and decays — same shape as Day 7's digit-count columns, just stretched lower.
T=8 is unusually competitive at mid-depths. At d=4 through d=10, T=8 is within ~1pt of T=4 (sometimes slightly higher). Possibly two adjacent settings of test-time depth around the training value are both near-optimal.

Experiment B: fixed-point analysis

Mean cosine similarity between consecutive hidden states cos(h_t, h_{t-1}) measured at the first-answer-token position, averaged across 4 seeds, 200 samples per depth:

t	d=2	d=4	d=8	d=12	d=16	d=20
1	0.85	0.89	0.92	0.92	0.93	0.91
2	0.91	0.91	0.94	0.95	0.95	0.97
3	0.94	0.94	0.92	0.92	0.94	0.95
4	0.95	0.97	0.96	0.95	0.93	0.93
8	0.998	0.995	0.998	0.996	0.996	0.992
16	0.9994	0.9996	0.9989	0.9985	0.9976	0.9979
32	0.9998	0.9998	0.9998	0.9997	0.9995	0.9996

Three things to note:

Fixed-point timing is slightly later than Day 7. Day 7 reached ~0.95 by T=2; brackets reach ~0.95 at T=3 and ~0.99 at T=4. About one extra loop step on this metric. Possibly the more complex left-to-right state needs a beat longer to settle.
Depth dependence is small. d=20 traces almost on top of d=2, again echoing Day 7 (where digit-count had only marginal effect on fixed-point timing). "Harder problem ⇒ slower fixed-point" did not appear.
Hidden state stops moving by T=4 (cosine ~0.99) while accuracy starts decaying. Same paradox as Day 7: extra loops are computation without information. Either the late-loop perturbations are small but logit-relevant drift away from a converged answer, or this is purely a distribution-shift artifact of training only at T=4.

Comparison with Day 7

Axis	Day 7 (addition)	Day 8 (brackets)
Loop-count peak at T=train (=4)	Yes	Yes
Best accuracy at peak	100% (all digits)	66% (d=2), 36% (d=20)
Inference-time loop extrapolation	Hurts	Hurts
Cosine fixed-point arrival	~T=2	~T=3
Depth/digit dependence on fixed-point	Small	Small
Training dynamics	Grokking (sudden phase transition)	Smooth slow climb

Day 8 reproduces all the qualitative findings of Day 7. What changes is the quantitative ceiling: at the same parameter budget and the same training compute, structure-tracking caps far below saturation while addition saturates.

💡 Tying back to the three perspectives

Day 7 tested looped transformers against three published views:

Saunshi et al. — loops can match deeper fixed-depth networks on algorithmic tasks
Geiping et al. (Huginn) — at scale, extra loops give marginal gains
Micheal Bee — loops plateau early at small scale (T=2 fixed-point)

Day 8 adds three more data points to the picture:

The "peak at training loop count" pattern persists across qualitatively different algorithmic tasks (addition vs. bracket parsing). This is consistent with Saunshi's framing but argues against naive depth-extrapolation at inference.
The fixed-point arrives at slightly different times for different tasks. Bee's "T=2" appears to be a property of the specific task and training recipe, not a universal property of looped transformers. Brackets need ~T=3-4 to plateau, addition needs ~T=2.
Task structural complexity matters more than loop count. At a fixed budget, the ceiling on accuracy is set by something else (model capacity? loss landscape? data efficiency?), not by the number of inference loops. Adding more loops can't compensate.

A useful refinement: looped transformers carry compute up to a depth bounded by the task's algorithmic complexity and the model's expressive capacity. Beyond that, the hidden state stops moving meaningfully and additional loops are computation without information. Day 7 showed this for a task within capacity (addition saturates); Day 8 shows it for a task that bumps against capacity (bracket parsing caps short).

🛠️ Technical details

Smoke history (why the task definition changed)

Initial smoke: balanced/imbalanced binary classification, depths 2-10.
Result: 100% accuracy across all depths by step 4,000.
Diagnosis: too many shortcut signals (length parity, open/close count) for the model to learn the stack algorithm — even with mutations that should defeat counting shortcuts. The 2-bit output gives the model no incentive to track position-by-position state.

Second smoke: first-break-position output, depths 2-20.
Result at 5,000 steps: d=2 100%, d=20 71%, with loss still trending down (0.32 → still falling).
Diagnosis: depth-dependent difficulty visible, room to scale training to expose loop-count effects.

Lesson worth recording: output information density matters as much as task structure for studying loop behavior. A binary classifier with global-aggregation shortcuts is a weak probe of recurrent depth.

Config and hyperparameters

MythosConfig(
    vocab_size=20,         # 6 brackets + '=' + '$' + space + '-' + '0'-'9'
    dim=256,
    n_heads=8,
    n_kv_heads=2,          # GQA
    max_seq_len=128,       # Day 7 was 32
    max_loop_iters=4,
    prelude_layers=1,
    coda_layers=1,
    attn_type="gqa",
    n_experts=4,           # MoE FFN inside recurrent block
    n_shared_experts=1,
    n_experts_per_tok=2,
    expert_dim=512,
    lora_rank=8,
    rope_theta=10000.0,
)

Total parameters: 3,394,338 (~3.4M, matches Day 7 to within rounding).

Training:

Optimizer: AdamW, betas (0.9, 0.95), wd 0.1
LR: max 3e-4, warmup 2000 steps, cosine decay to 1e-5
Grad clip: 1.0
Batch size: 128
Max steps: 30000
dtype: fp32 (same RoPE-complex-buffer reason as Day 7)
4 seeds {0, 1, 2, 3} in parallel

Data generation

On-the-fly synthetic. For each sample:

Sample depth d ∈ {2, 4, 6, 8, 10, 12, 16, 20} uniformly
Sample pair count n_pairs ~ U[max(1, d-1), min(2*d, 50)]
Generate balanced parenthesization (random bracket types, nested or sequential)
With prob 0.5, apply a mutation: delete close (30%), delete open (30%), substitute (40%)
Compute first-break position with the stack parser; format output

Loss is applied only at positions following = (i.e., on the 2-digit answer + $).

Evaluation

Experiment A: greedy autoregressive generation, exact 3-token match (position digits + $). 500 samples per (seed, n_loops, depth).
Experiment B: re-implementation of OpenMythos forward to expose per-loop hidden states. Cosine similarity at the first answer-token position. 200 samples per (seed, depth), 32 loop iterations.

What I'd want to try next

Increase training-time loop count and re-measure. Does the peak track with training depth (suggesting it's purely a distribution-shift artifact) or does extrapolation stay broken?
Scale model dim while keeping loops fixed. Does a 10x bigger model break through the ~66% / ~36% bracket ceiling, or does the structure-tracking task itself need a different inductive bias?
Mix tasks in training. Train on addition + brackets jointly and see if there's interference or transfer.
Inject explicit halting (ACT). Let the model choose how many loops per token. Does it match the empirical optimum or settle elsewhere?

References

Training and evaluation scripts: https://github.com/SAETAG/dgx-100-experiments/tree/main/days/day08-bracket-matching/scripts.

Tomorrow: Day 9

Switching gears to something much more personal — handing private chat data to a local model and seeing what it surfaces…!

100ExperimentsWithDGX #LocalLLM

[Day 7] Does Giving an AI More 'Thinking Time' Really Make It Smarter? Training an OpenMythos-Style Mini Model on DGX

PEPPERCORN — Tue, 19 May 2026 03:17:51 +0000

[Day 7] Does Giving an AI More "Thinking Time" Really Make It Smarter? Training an OpenMythos-Style Mini Model on DGX

Intro

Day 7!

Reddit kept surfacing this new project called OpenMythos in my feed with "12 days to replicate frontier AI, ASI is near" headlines, and I got curious enough to dig in.

Tools used: my home AI machine (DGX Spark) + OpenMythos (PyTorch reconstruction of the rumored Claude Mythos architecture) + synthetic multi-digit addition.

The question: does giving an AI more "thinking time" (= more recurrent loops at inference) actually make it smarter?

Today's setup

The hype

On 2026-04-07, Anthropic announced Claude Mythos. Press coverage highlights zero-day discovery capabilities — reportedly 271 zero-days in Firefox and a 27-year-old bug in OpenBSD — but the model's architecture and weights remain unreleased. Anthropic kept Mythos itself behind a limited-access coalition (Project Glasswing — AWS, Apple, Microsoft, Google, CrowdStrike, Palo Alto, ~40 organizations) rather than releasing it publicly.

Twelve days later, Kye Gomez (Swarms) released OpenMythos, a PyTorch reconstruction of the suspected architecture. The repo is explicit upfront:

"an independent, community-driven theoretical reconstruction based solely on publicly available research and speculation. It is not affiliated with, endorsed by, or connected to Anthropic"

So OpenMythos is not Mythos. It's a hypothesis-in-code: a Recurrent-Depth Transformer (RDT) with MoE FFNs and MLA/GQA attention, capable of being trained from scratch on standard text data. No leaked weights, no distillation.

Reddit's "ASI is near" framing skips this critical distinction. The interesting question, once you set the hype aside, is whether the architectural idea — recurrent depth — actually works.

Note for this article: OpenMythos is not Claude Mythos — it's a theoretical reconstruction inspired by looped-transformer research. The experiments below are not "Claude Mythos capability tests" but rather "how does a looped / recurrent-depth structure behave on a small synthetic task."

Three perspectives on looped transformers

Browsing the literature, I found three different studies giving different pictures of how looped transformers behave:

Source	Scale	Claim
Saunshi et al. 2025 (ICLR, research paper)	tens of M params, synthetic	Loops work: k layers looped L times approximately matches kL-layer fixed-depth, on addition / p-hop induction / math
Geiping et al. 2025 (Huginn, research paper)	3.5B params, 800B tokens	Task-dependent: at scale on natural-language benchmarks, gains can be marginal (T=4 → T=32 only +1.82 points on GSM8K), though effects vary by task and compute regime
Micheal Bee 2026-04 (Medium, independent experiment blog)	17M params, 12 GPU-hours on RTX 5070 Ti	Loops plateau at T=2 in this small-scale setup: hidden state reaches a fixed-point that subsequent iterations cannot escape

Theory, large-scale empirics, and an independent solo replication give different pictures. I wanted to add a fourth data point from my own DGX Spark on a clean, controlled task — multi-digit addition.

What I'd hoped to see

Does training-time accuracy phase-transition (grok) at some step? (Saunshi 3-stage prediction)
Does test-time loop count matter? At what point does it stop helping?
Does the hidden state actually keep evolving across loops, or does it hit a fixed-point early? (the Bee question)

Headline finding

Loops help, but only within a narrow window centered on the training loop count. With training-time max_loop_iters=4, accuracy peaks at exactly T=4 (100% across all digit counts) and decays in both directions — fewer loops underthink, more loops overthink.
Bee's "T=2 fixed-point" reproduced. Cosine similarity between consecutive hidden states jumps from ~0.72 to ~0.95 at T=2, then climbs slowly to ~0.99 by T=4 and stays flat through T=32.
Striking per-seed grokking variance. Same hyperparameters, four seeds: seeds 1 and 3 solve 5-digit addition by step 4,000; seed 2 takes 10,000; seed 0 stalls at <10% until step 16,000, then jumps to 100%.
No depth extrapolation in this setup. Saunshi's claim that training at T=4 should generalize to deeper T at inference does not reproduce here — instead, T>4 hurts.

🌀 What is a "looped" transformer?

A standard transformer (GPT-4, Llama, most local LLMs) routes input tokens through a stack of distinct layers, each used exactly once per forward pass. To make it "think deeper," you stack more layers — increasing parameter count.

A looped transformer reuses the same parameters across multiple iterations. The model has a Prelude → Recurrent Block × T → Coda structure: a few standard layers up front, then one block iterated T times with input injection at every step, then a few more standard layers.

Input tokens
   ↓
[Prelude P]          — standard layers, run once
   ↓
[Recurrent Block R]  — one block looped T times
   ↑_______↓          h_{t+1} = A·h_t + B·e + Transformer(h_t, e)
   ↓
[Coda C]             — standard layers, run once
   ↓
Output logits

At each loop iteration t, the hidden state updates via the LTI injection rule, and the encoded input e (Prelude output) is re-injected to keep the original signal alive across arbitrary depth. The injection parameters are constrained so that spectral radius ρ(A) < 1, which prevents divergence over many loops (Parcae stability framework).

The key claim: more loops at inference = deeper reasoning, without adding parameters. This is conceptually analogous to chain-of-thought scaling — except the "thinking" happens in continuous latent space rather than discrete token space.

🔧 Experimental setup

I trained a deliberately tiny OpenMythos variant on multi-digit addition. The model is small enough to run 4 seeds in parallel on a single GPU but large enough to exhibit the looped-transformer phenomena.

OpenMythos tiny (3.4M params)
  ↓
Train 4 seeds in parallel, 30k steps each, fp32 on DGX Spark (GB10)
  ↓
Experiment A: greedy autoregressive accuracy
              loops ∈ {1, 2, 4, 8, 16, 32}  ×  digits ∈ {2, 3, 4, 5}
  ↓
Experiment B: cosine similarity between consecutive hidden states
              ⇒ does the recurrent block reach a fixed-point?
  ↓
Compare against Saunshi / Huginn / Bee

Model config

MythosConfig(
    vocab_size=16,         # digits 0-9 + '+', '=', pad, eos
    dim=256,
    n_heads=8,
    n_kv_heads=2,          # GQA
    max_seq_len=32,
    max_loop_iters=4,      # training depth; inference varies
    prelude_layers=1,
    coda_layers=1,
    attn_type="gqa",
    n_experts=4,           # MoE FFN inside recurrent block
    n_shared_experts=1,
    n_experts_per_tok=2,
    expert_dim=512,
    lora_rank=8,           # depth-wise LoRA per loop step
)

Total parameters: 3,386,658 (~3.4M).

Data

On-the-fly synthetic addition. Operands are uniformly sampled from [10^(d-1), 10^d - 1] for digit count d ∈ {2, 3, 4, 5}. Sequence format "A+B=R$", where R = str(A+B)[::-1] (reverse-order answer, following Saunshi's convention so left-to-right autoregressive generation can carry digits naturally).

Loss is applied only at positions following the = token (i.e., on the answer tokens).

Training

Optimizer: AdamW, betas (0.9, 0.95), wd 0.1
LR: max 3e-4, warmup 2000 steps, cosine decay to 1e-5
Grad clip: 1.0
Batch size: 128
Max steps: 30000
dtype: fp32

Initially I tried bf16 to use the GB10 efficiently, but OpenMythos stores RoPE frequencies as complex64 buffers, and model.to(bfloat16) silently drops the imaginary part, breaking attention. For a 3.4M-param model on 128 GB of unified memory, fp32 is fine — the bottleneck is not memory but parallel scheduling.

Four seeds {0, 1, 2, 3} run in parallel on the same GPU. Per-seed throughput drops to ~12K tok/s (vs ~50K solo), but wall-clock time for all four is approximately equivalent to one solo run.

📊 Results

Experiment A: accuracy heatmap

Mean fully-correct rate across 4 seeds, 500 eval samples per condition:

Inference loops	d=2	d=3	d=4	d=5
1	0.38 ± 0.12	0.19 ± 0.09	0.09 ± 0.07	0.02 ± 0.02
2	0.53 ± 0.17	0.50 ± 0.12	0.16 ± 0.08	0.21 ± 0.16
4 (train)	1.00	1.00	1.00	1.00
8	0.98 ± 0.01	0.98 ± 0.01	0.94 ± 0.03	0.86 ± 0.08
16	0.91 ± 0.04	0.91 ± 0.05	0.75 ± 0.10	0.56 ± 0.16
32	0.62 ± 0.12	0.65 ± 0.13	0.45 ± 0.13	0.26 ± 0.17

Observations:

Peak is exactly at training-time loop count (T=4), 100% across all digit counts.
One step of inference-time extrapolation (T=8) is near-peak but already shows degradation at d=5 (86%).
Beyond T=8, accuracy collapses monotonically. At T=32, even 2-digit addition drops to 62%.
Under-looping (T=1, T=2) hurts more at higher digit counts, consistent with depth being needed to chain carries.

Experiment B: fixed-point analysis

Mean cosine similarity between consecutive hidden states cos(h_t, h_{t-1}) over answer positions, averaged across 4 seeds, 200 samples per digit:

t	d=2	d=3	d=4	d=5
1	0.711	0.726	0.745	0.744
2	0.961	0.967	0.957	0.946
3	0.985	0.986	0.977	0.971
4	0.993	0.992	0.986	0.983
8	0.999	0.999	0.998	0.996
16	0.9995	0.9996	0.9992	0.998
32	0.9995	0.9996	0.999	0.998

Bee's T=2 fixed-point claim is reproduced in spirit but not literally: cosine similarity jumps to ~0.95 at T=2 (vs. Bee's near-1.0), then asymptotes to ~0.99 by T=4 and stays flat through T=32.

The difference vs. accuracy is telling: hidden state is effectively static (by cosine similarity) from T=4 onwards, yet accuracy collapses at T=16-32. Two non-exclusive interpretations: (a) overthinking — late loops drift away from a converged solution; (b) distribution shift — training used T=4, so T>>4 is simply an out-of-distribution use of the model. Worth noting that cosine similarity ≈ 1 doesn't prove the hidden state is doing nothing — small logit-relevant deltas may still accumulate.

Digit-count dependence on fixed-point timing is small (d=5 lags d=2 by ~0.01 in cosine sim). "Harder problems take more loops to converge" is not observed here — they converge at the same rate but the converged state is just less accurate at higher digit counts.

Bonus: training dynamics

The most striking thing in the training curves is seed-dependent grokking timing. Four runs of identical hyperparameters:

seed 1: loss → 0 by step 3,000, all digits ≥88% by step 4,000
seed 3: loss → 0 by step 4,000, all digits ≥87% by step 4,000
seed 2: stuck at loss ~0.35 plateau until step 8,000, then collapses to 0 by step 10,000; d=4/5 jump from <10% to 99% in 2,000 steps
seed 0: stuck at loss ~0.30 plateau until step 15,000, then collapses; d=4 groks at step 12,000-14,000, d=5 groks at step 16,000

This is textbook Saunshi-style three-stage grokking (memorization → in-distribution → systematic), with the third-stage trigger varying by a factor of 4x in step count purely on random init. The largest seed gap (seed 0 vs. seed 1) is ~12,000 steps, roughly 1 hour of wall-clock on this DGX.

If you trained a single seed and stopped early, you might conclude "OpenMythos can't generalize beyond d=3" — which would be wrong. The architecture can solve all 4 digit buckets; some random seeds just need much longer to find the systematic-generalization solution.

💡 What this means for the three perspectives

Where my data point lands

My single-DGX small-scale result lands somewhere between Bee and a partial refutation of Saunshi:

Bee's fixed-point at small T is reproduced. Hidden state effectively stops evolving by T=4 (cosine sim ≥ 0.99) and certainly by T=8.
Saunshi's depth-extrapolation does NOT reproduce. Inference at T > train_T does not improve accuracy — it harms it. T=8 is already at 86% on d=5 (vs. 100% at T=4), and T=32 collapses to 26%. The "train at depth k, infer at depth k·L" recipe assumes the recurrent block has learned to keep refining; in my setup it has not.
Huginn's limited-gain finding is consistent at small scale. Extra inference loops give negative ROI rather than diminishing positive ROI.
New observation: seed-dependent grokking with up to 12K-step variance. This is an under-emphasized variable in the public looped-transformer discourse — single-seed studies (Bee's solo replication, individual rows in Saunshi's tables) may be substantially under- or over-estimating typical behavior.

Reconciliation attempt

Theory (Saunshi), large-scale empirics (Huginn), and independent replication (Bee) may not actually be in contradiction — they may be measuring different facets of the same phenomenon at different scales:

Saunshi: shows loops can work on the right kind of problem (algorithmic, depth-bounded reasoning) at the right kind of scale (small synthetic).
Huginn: shows that loops trained at 3.5B / 800B token scale on natural-language data give only marginal gains on a benchmark (GSM8K) that already favors CoT.
Bee: shows that within a particular small-scale training recipe, the recurrent block's hidden state stops evolving very early in inference.

These three findings are compatible with a unified picture: loops carry compute, but only up to a depth bounded by the task's algorithmic complexity and the model's expressive capacity. Beyond that depth, the hidden state stops moving meaningfully, and additional loops are computation without information.

What I'd watch next

Increase loop count during training (here I used 4) and see if the inference-time scaling extends further
Try ACT halting more aggressively to see how the model self-regulates loop depth per token
Add task heterogeneity (mix p-hop induction or parity) to test whether the fixed-point timing varies by problem class

🛠️ Technical details

Reproducing this experiment

git clone https://github.com/kyegomez/OpenMythos
cd OpenMythos
pip install -e .

# Data, training, evaluation scripts (this Day 7 folder):
python scripts/train.py --seed 0 --max_steps 30000
python scripts/eval_accuracy.py --seeds 0 1 2 3
python scripts/eval_fixedpoint.py --seeds 0 1 2 3
python scripts/plot.py

The training and evaluation scripts are at https://github.com/SAETAG/dgx-100-experiments/tree/main/days/day07-openmythos-loop-debate/scripts.

What went wrong (and was fixed)

bf16 broke complex RoPE buffer: switched to fp32; fine at 3.4M parameters
Initial training-time max_loop_iters too small: kept at 4 per Saunshi's recipe; future experiments could vary this
Greedy generation is slow at high loop counts: each batch repeats n_loops forward passes through the recurrent block; for loops=32 this is non-trivial

Hyperparameter choices: why these

dim=256, expert_dim=512, 1 prelude / 1 coda layer: smallest config that still exhibits looping behavior; matches Saunshi's scale
n_experts=4: enough to demonstrate MoE routing without bloating params
lora_rank=8: depth-wise LoRA lets each loop iteration adapt slightly without breaking weight-sharing
max_seq_len=32: tight bound — d=5 addition fits in ~18 chars

References

Tomorrow: Day 8

A follow-up to Day 7, pushing looped thinking one step further into something harder…!

100ExperimentsWithDGX #LocalLLM

[Day 6] I Had an AI Look at 25,000 iPhone Photos and It Decided My Mom and I Are the Same Person

PEPPERCORN — Tue, 12 May 2026 18:10:12 +0000

[Day 6] I Had an AI Look at 25,000 iPhone Photos and It Decided My Mom and I Are the Same Person

Intro

Day 6!

On Day 4, I had a local AI sort through 25,000 photos on my iPhone (Day 4 article). Today is the follow-up — I wanted to go one level deeper and have AI look at my behavioral patterns over time.

Tools used: my home AI machine (DGX Spark) + a face recognition AI (FaceNet) + a summarization LLM (Qwen2.5 72B running on Ollama).

Today's setup

What I actually did

Take 5 years of photos (25,000) and have an AI summarize my day-to-day life from them. Two phases:

Phase 1: aggregate by capture date + camera model + photo category (cat, food, scenery, etc.), then ask the LLM to read it
Phase 2: add face recognition AI to answer "who is in each photo," then ask the LLM again

The key bit of today

The face recognition AI treated me and my mom as the same person — but the interesting part is that all the other misclassifications were "different people with the same expression," whereas in our case it was "different expressions, same person" despite my mom being straight-faced and me grinning with teeth showing.

The AI gets fooled by expressions, but it also seems to pick up on something beyond expressions (bone structure? face shape?). That's today's headline.

🔧 How I went about it

25,000 photos (already categorized on Day 4)
   ↓
Phase 1: aggregate "capture date / camera model / category" only
   → ask the LLM to summarize year by year
   ↓
Phase 2: add face recognition AI to label "who is in each photo"
   → ask the LLM to summarize again

GPS data ("where") had been stripped during the iCloud export, so I substituted camera model as a proxy (iPhone = daily life, Olympus TG = travel, DJI handheld = video shoots, etc.).

(The tools and detailed steps are in the "Technical details" section at the end.)

📊 Results

Phase 1: date / camera / category only

I pulled "capture date + camera model + category (sorted on Day 4)" out of the 25,000 photos and turned it into four heatmaps showing year-over-year patterns. Then handed those to the LLM.

What's a heatmap? = A table where the rows × columns are filled with color intensity based on count. Dense color = a hotspot of activity, visible at a glance.

Photo count per year

2019 was a clear outlier at 4,931 photos — 2-3x the other years.

Year × Category

Cat photos exploded starting 2021 → matches the year my cat joined the household.

Year × Month

August 2019 was the single highest month at 1,082 photos.

Year × Camera model

Olympus TG dropped off sharply in 2020 (matches the COVID period). The DJI handheld shows up starting 2025.

When I handed this to the LLM and asked for a yearly summary, the output was along the lines of "this might have been a busy year" or "looks like an active year." Well, of course — the only info I gave the LLM was "when, what camera, what kind of subject." That's the ceiling for what it can say.

So the next question: what happens if you add who is in each photo? That's Phase 2.

Phase 2: adding "who's in the photo"

I ran face recognition AI over the 25,000 photos, detected 21,000 faces, and grouped similar-looking ones into 209 groups (C1, C2, …, C209). Plotting those over time:

What's a "similar-face group"? = a group the face recognition AI thinks contains "the same person" (technically called a "cluster"). The AI only manages them as numbered IDs, so a human still has to look at each group and label "this is person X."

Person cluster × Year heatmap

This heatmap turned out to be interesting:

Long-spanning groups (C1, C2, C3) → likely family or myself
Short-spanning groups → likely acquaintances from a specific period

…which gives you a working guess. When I fed this back to the LLM, the summary turned much more concrete: "C3 is a new appearance," "C2 is decreasing in frequency," etc.

💡 Today's biggest finding

I went through the face clusters one by one and saw that the AI's groupings landed in a mix of "worked great," "fair enough," and "failed":

✅ Worked great: grouped the same person across different angles and expressions (one group had all 4 photos of the same family member)
🤔 Fair enough: burst shots end up grouped (multiple groups were just consecutive frames of the same moment)
⚠️ Failure pattern A: grouped different people who happened to share a similar smile (happened in several groups)
😳 Failure pattern B: grouped me and my mom despite our totally different expressions

The most striking one was failure pattern B.

Failure patterns A and B are misclassifications for opposite reasons

Failure pattern A: different people, same expression

Different people grouped together because of a similar smile.

Three different people — but when smiles are similar, the AI calls them "same person."

※The actual experiment used real photos. The illustrations here are AI-generated stand-ins for privacy.

Failure pattern B: parent and child, different expressions

My mom and I in the same group — despite the expression difference (I'm grinning with teeth showing, she's neutral).

Parent and child with clearly different expressions — but the AI still says "same person."

※The actual experiment used real photos. The illustrations here are AI-generated stand-ins for privacy.

In the groups I eyeballed, "different expressions but same person" only happened in our case. Every other misclassification was "same expression, different people."

So my mom and I are a different kind of mistake. Either the AI is picking up on genetic facial similarity, or there's some other mechanism at work (I'll touch on this in the technical details). Hard to be definitive, but a fascinating case.

Summary: how the AI "sees" faces

Situation	AI judgment	Likely reason
Same person, different angle & expression	◯ to △	Bone structure matches well
Different people, same expression	✕ (often grouped)	Pulled in by expression noise
Parent & child, different expressions	✕ (sometimes grouped)	Bone structure similarity outweighs expression difference

The AI gets fooled by expressions, but seems to actually pick up on something beyond expressions (bone structure? face shape?) — that was the most interesting observation of the day.

🛠️ Technical details

:::details Tools used

EXIF extraction: Python pillow_heif + PIL.Image.getexif() (HEIC-aware)
Face recognition: facenet-pytorch (InceptionResnetV1, vggface2-pretrained)
Clustering: scikit-learn DBSCAN
LLM summarization: Qwen2.5 72B via Ollama
Compute: DGX Spark (lots of GPU memory)

What's EXIF? = the camera metadata embedded in each photo file (capture time, camera model, GPS, etc.).

What's FaceNet? = an AI that converts a face photo into a 512-dimensional vector. Same person's faces are close vectors, different people are far apart.

What's DBSCAN? = a classic ML clustering method that automatically groups similar items. You don't need to specify the number of groups in advance.

:::

:::details EXIF extraction script (parallelized, 6 seconds total)

pillow_heif to support HEIC, PIL.Image.getexif() to read EXIF. Parallelized with concurrent.futures.ProcessPoolExecutor (12 processes).

import pillow_heif
pillow_heif.register_heif_opener()

from PIL import Image, ExifTags

def extract_one(path):
    with Image.open(path) as img:
        exif = img.getexif()
        inner = exif.get_ifd(34665)  # ExifIFD
        # DateTimeOriginal lives inside ExifIFD
        if 36867 in inner:
            dt = parse_exif_datetime(inner[36867])
        make = exif.get(271)
        model = exif.get(272)
        gps = exif.get_ifd(34853)

Photos with no EXIF date (screenshots, etc.) fall back to file mtime, but that's just "the day I copied the file," so I excluded those from year-level aggregation.

:::

:::details Tuning DBSCAN's eps

Distance between embeddings is cosine distance (1 - dot product).

eps	Clusters	Largest cluster size	Verdict
0.4 (loose)	3	21,310	Everyone in one group
0.3	73	17,234	Still big lumps
0.25	146	12,905	Still too big
0.2	209	4,582	◎ chosen
0.18	216	3,131	Too tight — single people split into multiple clusters

from sklearn.cluster import DBSCAN
from sklearn.preprocessing import normalize

embeds_n = normalize(embeds, norm="l2")
db = DBSCAN(eps=0.2, min_samples=5, metric="cosine", n_jobs=-1)
labels = db.fit_predict(embeds_n)

min_samples=5 means only people who show up 5+ times across photos get clustered.

:::

:::details Why parent and child tend to land in the same cluster

facenet-pytorch's InceptionResnetV1 (vggface2-pretrained) produces 512-dim embeddings that are designed to capture geometric (bone structure) features. Lighting, angle, and expression noise also leak in.

Parent and child share genetic bone structure, so their embeddings can be closer than you'd get between random different people. This is a known phenomenon in face recognition research — several papers have demonstrated it.

DBSCAN is density-based: if "A→B is close" and "B→C is close," then A and C end up in the same cluster even if A and C aren't directly close. If there's one photo of me looking especially like my mom that sits in between, that single bridge photo can connect us into one cluster.

:::

:::details Generating representative face thumbnails for manual labeling

Clusters are just IDs (C0, C1, …), so a human has to look at them and label "this is person X."

I wrote a script that crops the largest face from each cluster's representative photos and lays them out as a diagnostic image:

from facenet_pytorch import MTCNN

mtcnn = MTCNN(keep_all=True, min_face_size=40, device='cuda')
boxes, _ = mtcnn.detect(img)
if boxes is not None:
    biggest = max(boxes, key=lambda b: (b[2]-b[0]) * (b[3]-b[1]))
    crop = img.crop(biggest)

This image contains real faces of family and friends, so I kept it strictly local in private-data/day06-timeline/ (gitignored). Opened it via VS Code Remote-SSH to label by eye.

:::

📝 Today's takeaways

Handing the LLM only "when / what camera / what category" yields a blurry overview
Adding "who is in the photo" jumps the resolution of the analysis up several notches
Face recognition AI is sensitive to expression noise but does pick up something beyond expressions (bone structure / face shape)
Because of that, parent-child being grouped "despite different expressions" became the one unique case in my dataset
Keeping sensitive face data off the cloud is a big advantage of running this locally
Processing 25,000 photos in one go is also realistic on a local setup — no API costs to worry about

Tomorrow's preview: Day 7

Day 7 plan: local AI vs cloud AI, 5-round showdown.

Going to take the tasks I usually do with local AI (photo classification, credit card analysis, code completion, etc.), run them on both sides, and build a head-to-head matrix.

To be continued >>>

100ExperimentsWithDGX #LocalLLM #ImageAnalysis #FaceNet

[Day 5] I Trained My Cat-LoRA on 22 vs 213 Photos and the Results Were Basically Identical

PEPPERCORN — Mon, 11 May 2026 01:23:26 +0000

[Day 5] I Trained My Cat-LoRA on 22 vs 213 Photos and the Results Were Basically Identical

Intro

Day 5!

Today was originally going to be "have AI analyze a year of my Amazon order history," but downloading the Amazon purchase history just wouldn't work no matter what I tried. So that was a bust.

Pivoted.

On Day 2, I trained an AI to memorize my cat from 22 photos (Day 2 article). That thing is called a "LoRA."

What's a LoRA? = A small add-on that teaches an AI to recognize a specific subject. Pair photos with a trigger word like ohwx cat, train, and then writing ohwx cat in any prompt makes the AI draw my cat.

On Day 4, I had AI sort through 25,000 photos on my iPhone (Day 4 article). It found 999 photos it identified as cats.

Today's experiment: Will using those 999 photos make my cat-LoRA stronger?

A simple expectation, really. 22 photos → 999 photos is 45x more data. Surely the LoRA gets stronger, right?

TL;DR

Spoiler-free version:

Training with 999 photos made things worse, not better
After removing "other people's cats" from the dataset (down to 213 photos), I got LoRA quality matching my original 22-photo version
22 photos and 213 photos produced basically the same quality

I came in thinking "more photos = stronger LoRA." Turns out that's not really how it works, and today I learned why.

What I actually did

Trained on 999 photos → got worse (v2)

Same base model and trigger word (ohwx cat) as Day 2. Just bumped the photo count from 22 to 999. Kohya_ss training, 14 minutes. Calling this v2.

Generated test images and…

Photorealistic scene (left: no LoRA, center: v1=22 photos, right: v2=999 photos). v2 looks barely different from no-LoRA. 45x more data, but the cat identity is gone.

Creative prompts were worse:

Prompt: "ohwx cat as a cute chef." v2 produced a human woman as the chef, with the cat reduced to a tiny illustration on her apron.

Prompt: "ohwx cat as an astronaut." v2 produced a tabby (orange-striped) cat — the fur color is straight up wrong. My cat is black and white.

→ More data made the LoRA broadly worse, across both photorealistic and creative prompts.

Cause: "other cats" had snuck into the dataset

Once I thought about it, it was obvious.

Day 4's classifier labels images as "contains a cat or not" — it does NOT verify "is this MY cat." So the 999-photo "cat" folder included:

My cat
Friends' and family's cats
Stray cats from around town
Cats at pet stores

All mixed together. When I trained with the label ohwx cat = my cat, the model basically learned ohwx cat ≈ generic cat-shape.

Pulled out just my cat → 213 photos (v3)

To curate, I borrowed another AI — CLIP.

What's CLIP? = An OpenAI image-understanding model. Show it two images and it returns a similarity score.

I used the 22 confirmed-my-cat photos from Day 2 as a reference set, then asked CLIP to score how similar each of the 999 candidates was. Sorted by score, threw the thumbnails into a single HTML page, and went through visually — checking "this one's a different cat", "this has a person in it", and so on, marking exclusions as I went.

Final cut: 213 photos, all confirmed to be my cat. Re-trained → v3.

Result:

v3 is as sharp as v1. Tuxedo pattern, white chest, the works.

Creative prompts came back too:

The human chef from v2 is gone, replaced by my cat. The astronaut and forest cat similarly snapped back (more comparisons in the collapsible section below).

→ Cleaning the data was enough to fix everything.

Bonus: also tried natural-language captions (v4)

One more thing I wanted to test.

v1 (Day 2) and v3 (today) differ in their captions — the text labels paired with each training photo:

v1: hand-written natural sentences (ohwx cat, walking on a metal kitchen counter, side profile, indoor kitchen with spice bottles...)
v3: just the trigger word (ohwx cat) repeated for every image

What's a caption? = A short English text describing what's in each photo, paired with that photo during training.

Would adding richer captions on top of clean data push v3 further? Hand-writing 213 captions wasn't realistic, so I had another AI (Qwen2-VL) auto-generate them. Calling this v4.

Result: v4 looked basically identical to v3. Small differences here and there but nothing substantial.

→ Caption granularity barely matters once the data is clean.

The actual question: does more data make a stronger LoRA?

Now for the real comparison. v1 (22 photos) vs v4 (213 photos):

	Photos	Data purity	Captions
v1	22	My cat only	Hand-written natural language
v4	213 (10x!)	My cat only	VLM natural language (same style)

The only meaningful difference is photo count.

Five-way comparison:

Left to right: no LoRA, v1 (22), v2 (999, contaminated), v3 (213, trigger-only), v4 (213, natural captions).

v1 and v4 are essentially the same quality. To my eye, v1 has a slightly more painterly feel on the chef prompt, but otherwise — same.

Same pattern across all the other prompts:

→ 10x more photos. No visible improvement. This was today's main finding.

After the fact, I looked it up. Turns out this is common knowledge.

I found "more photos doesn't help" interesting enough to look up afterward, and:

Character LoRAs are typically trained on 25–40 images, with 40–80 as a soft cap
"Over 30 images shows diminishing returns; dataset quality matters more than dataset size"
"15–20 well-curated images beat 50 mediocre ones"
Too many images can actually overfit and degrade the result
DreamBooth (a closely related technique) was designed around 3–5 images

→ It's established consensus in the field: photo count saturates fast, and dataset purity is the real lever.

Day 2's 22 photos? Turns out that was already a healthy amount.

What I learned today

Quality > Quantity, apparently

22 photos (v1) ≈ 213 photos (v4): photo count doesn't push quality much
999 photos (v2): contamination made things worse
213 photos (v3): cleaning brought everything back

"More photos = better LoRA" runs out of road fast. What actually moves the needle is the right photos, not more photos.

A working playbook (so far)

From today's experiments:

Source photos that match the goal (photos of MY cat, not "any cat")
Aim for 20–30 photos — past that, diminishing returns
Captions help, but don't sweat the wording — auto-generated is fine
If you must use a big dataset, curate aggressively first — contamination is brutal

💡 Tip: when you want to use a big dataset anyway

If you're starting from a large unfiltered pile and want to keep it that way, pre-curation is essential. The approach that worked today:

Pick a small "ground truth" set (~20 confirmed examples)
Use CLIP image similarity to score the big pile against the ground truth
Browse thumbnails sorted by score, eyeball-exclude the misses
Train on what's left

Details in the collapsible section below.

Technical details (the AI explains)

The implementation details, walked through by Claude.

:::details 1. More v2 failure examples

Skipped from the main body for length, but worth seeing:

Prompt: "ohwx cat in a magical forest." v2 produced a black-bear-style illustration — the cat identity is completely gone.

The one photorealistic-ish prompt where v2 sort-of held it together.

:::

:::details 2. Data prep and CLIP similarity ranking

Day 4's _review/cat/ had 1,009 symlinks (503 HEIC, 505 JPG, 1 other). Resized to short-side 512px:

python3 shared/utils/resize-shortside.py \
  --src private-data/iphone-photos-classified/_review/cat \
  --dst private-data/cat-lora-v2/images-512 \
  --short-side 512

1,009 → 999 after collisions (9 stem collisions where IMG_XXXX.HEIC and IMG_XXXX.JPG produced the same .jpg name) and 1 resize failure.

CLIP similarity scoring with openai/clip-vit-base-patch32:

from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32").to("cuda")
ref_feats = embed(model, processor, ref_paths)     # 22 refs
cand_feats = embed(model, processor, cand_paths)   # 999 candidates
sim = cand_feats @ ref_feats.T                     # (999, 22)
score = sim.mean(dim=1)                            # (999,)

Score distribution:

Score band	Contents
≥ 0.85	Almost all solo shots of my cat
0.76 – 0.85	Mostly my cat, with occasional other-cat or human contamination
< 0.76	Mostly other cats or photos with people

Cut at 0.76 and reviewed everything above visually. 312 manual exclusions later: 213 photos.

:::

:::details 3. Browser-based curation UI

A single HTML page laying out all 999 thumbnails in score order, served via python3 -m http.server. Each thumbnail has a checkbox:

<div class="cell" data-name="IMG_2906.jpg">
  <img src="thumbs-256/IMG_2906.jpg">
  <div class="meta">#1 0.871</div>
  <input type="checkbox" onchange="toggleExclude(this)">
</div>
<script>
function exportExcluded(){
  const names = [...document.querySelectorAll('.cell.excluded')]
    .map(c => c.dataset.name);
  download('excluded.txt', names.join('\n'));
}
</script>

Click "Export excluded list" to download excluded.txt, then use that to filter the training dir.

:::

:::details 4. Training configs (Kohya_ss / TOML)

The training config is identical across v1/v2/v3/v4 — only the dataset and output name change:

output_name = "ohwx_cat_v3"   # or v4
max_train_epochs = 2
network_dim = 32
network_alpha = 16
unet_lr = 1e-4
text_encoder_lr = 5e-5

Step count is also matched:

	Math	Steps
v1	22 × 10 × 10 ÷ 2	1,100
v2	999 × 1 × 2 ÷ 2	999
v3 / v4	213 × 5 × 2 ÷ 2	1,065

All within ~1,000 steps, so the only variables in play are photo count and caption granularity.

cd ~/Kohya_ss && source venv/bin/activate
accelerate launch --num_cpu_threads_per_process 8 train_network.py \
  --config_file configs/train_v3.toml \
  --dataset_config configs/dataset_v3.toml

DGX Spark, 1.4 it/s, ~14 minutes per training run.

:::

:::details 5. Qwen2-VL caption auto-generation

Reusing Day 4's Qwen2-VL 7B Instruct setup. The prompt:

Describe what is happening in this cat photo using short comma-separated
phrases. Cover: (1) the cat's pose or action, (2) the view angle,
(3) the setting and notable background details. Keep it under 25 words.
Do NOT describe the cat's appearance (color, breed, fur, markings) — focus
only on the scene. Output the description directly without any preamble.
Example: walking on a metal kitchen counter, side profile, indoor kitchen
with spice bottles and shelves in the background

The "do not describe the cat's appearance" line is intentional: identity is supposed to come from the trigger word ohwx cat, so captions should only describe context.

desc = vlm_caption(model, processor, img)
caption = f"ohwx cat, {desc}"
txt_path.write_text(caption + "\n", encoding="utf-8")

213 captions in 6 minutes. Sample output:

ohwx cat, sitting, side view, indoor setting, wooden floor,
folding chair, curtain, air conditioner

Stylistically very close to Day 2's hand-written captions.

:::

:::details 6. Version summary

	v1 (Day 2)	v2	v3	v4
Photos	22	999	213	213
Cat content	My cat only	My cat + many others	My cat only	My cat only
Captions	Hand-written natural	`ohwx cat` only	`ohwx cat` only	VLM natural
Total steps	1,100	999	1,065	1,065
Training time	13m 3s	14m 0s	14m 0s	14m 0s

What each pair isolates:

v2 vs v3 → effect of data purity (same captions, only purity differs)
v3 vs v4 → effect of caption granularity (same data, only captions differ)
v1 vs v4 → effect of photo count (clean data, natural captions, only count differs)

:::

:::details 7. References on LoRA training dataset size

The "diminishing returns past ~30 photos" claim has multiple sources:

20–30 photos saturates; dataset quality > dataset size (Civitai: Large Dataset LoRA Tips)
15–20 well-curated images beat 50 mediocre ones (same)
Over-training and "overcooked" LoRAs from too much data (Hugging Face Blog: After 500+ LoRAs)
DreamBooth (the original subject-finetuning technique) was designed around 3–5 images (DreamBooth project page)

:::

Tomorrow's preview: Day 6

Day 6: still undecided. Decision tomorrow morning.

100ExperimentsWithDGX #LocalLLM #LoRA #StableDiffusion

[Day 4] I Had a Local AI Sort Through 25,000 Photos on My iPhone

PEPPERCORN — Thu, 07 May 2026 23:58:45 +0000

[Day 4] I Had a Local AI Sort Through 25,000 Photos on My iPhone

Intro

Day 4: I'm going to hand the 25,000 photos sitting on my iPhone over to a local AI for sorting.

This is experiment #4.

What I'm using today: DGX Spark + CLIP (image-understanding AI from OpenAI) + Qwen2-VL (a vision-language model that can chat about images, from Alibaba).

Today's setup

Data: 25,382 photos and videos sitting on my iPhone (96 GB).
Goal: Have AI find unnecessary photos so I can drop my phone storage subscription.
Approach:
- Stage 1: Quickly classify all 25K with CLIP.
- Stage 2: Have Qwen2-VL (a VLM) grade CLIP's classifications.
Comparison axis: Lightweight + fast classifier (CLIP) vs. heavyweight + smart conversational AI (VLM).

Bottom line: Overall agreement of 84.5% when the VLM grades CLIP's classifications. People detection: 99.2% — only 59 misses out of 7,195 photos. Documents and screenshots ended up wrong about half the time. Oh, and I gave up midway and just dumped everything into Amazon Photos because I'd just learned Prime members get unlimited photo storage. Five years a Prime member, never knew.

🔧 Steps

Big picture flow:

iPhone
   ↓ ① Sync via iCloud for Windows
myPC1 (Windows)
   ↓ ② scp transfer to DGX (96 GB)
DGX (Linux)
   ├─ ③ Split photos and videos by extension
   │     └ Photos 24,497 / Videos 884
   ├─ ④ Classify with CLIP (~20 min)
   │     └ Sorted into 8 categories
   └─ ⑤ Have VLM grade "is this category right?" (~3 hours)
         └ Overall agreement: 84.5%

Let's walk through each step.

Getting photos onto the DGX (the biggest hurdle)

iPhone → myPC1 (a Windows laptop I use day-to-day) → DGX, a two-leg relay.

The first leg started at 0.5 MB/s, with the ETA showing "6 days." After realizing my Wi-Fi was the bottleneck, I switched to wired LAN, fixed the hostname-resolution path, and got it up to 80 MB/s (~160x faster). Burned half a day. More technical details in the collapsible section below.

Splitting photos and videos

The 25,382 transferred files broke down like this:

Extension	Count	Type
HEIC	13,107	Photo (Apple's format)
JPG / JPEG	10,721	Photo
PNG	660	Photo (mostly screenshots)
WEBP	9	Photo
MOV	799	Video
MP4	85	Video
ini	1	System file (ignored)

I had Claude write a small script that splits photos and videos into separate folders by extension (one command, takes a few minutes — details in the collapsible section).

Result:

Photos: 24,497
Videos: 884
Photos are the focus from here.

What is CLIP?

CLIP ＝ an image-understanding AI from OpenAI, apparently. You hand it a photo and ask "is this a cat? a landscape? a screenshot?" with multiple labels, and it returns a similarity score for each. Lightweight and fast is its specialty, supposedly.

Stage 1: Classifying all 25K photos with CLIP

I set up 8 categories:

Trash candidates: screenshot / document / blank
Keep: food / landscape / other
Review: people / cat

For each category, I prepared multiple English captions (e.g., "a screenshot of an app", "a photo of a cat") and used the maximum similarity. Also: anything below 0.5 confidence goes into the uncertain bucket for manual review.

Batch size 64, ~20 minutes of GPU time, all done. Results in the next section!

The "How accurate is it?" question

CLIP did the classification, but how accurate is it really?

Normally you'd verify by manual inspection, but eyeballing 25,000 photos is not realistic.

So I decided to have a smarter AI grade CLIP's classifications.

What is a VLM?

A VLM (Vision-Language Model) is an AI that can hold a conversation about images, apparently.

How it differs from CLIP:

	CLIP	VLM
What it does	Category classification (returns probabilities)	Can describe image content in natural language
Smartness	Lightweight, fast, coarse	Heavy, slow, smart
Size	~400 MB	~16 GB

I picked Qwen2-VL 7B Instruct (Alibaba). Apache 2.0 licensed for commercial use, no Hugging Face authentication required for download — those were the selection criteria.

The plan: ask the VLM "is this a screenshot? answer yes or no" for each image and record the answer.

Stage 2: Grading all 25K photos with VLM

Started at 16 seconds per image (~5 days for the full set). The cause was image size — resizing to 448px on the short side dropped it to 0.3 sec/image (~54x faster). Even with one-image-at-a-time inference, the full set takes ~2-3 hours.

Started before bed, woke up to 24,496 graded results.

📊 Results

CLIP's classification results

After CLIP processed 24,496 photos, the distribution looked like this:

private-data/iphone-photos-classified/
├── _trash-candidate/      Trash candidates
│   ├── screenshots/    (981)
│   ├── documents/    (1,804)
│   └── blank/           (59)
├── _review/                Review
│   ├── people/       (7,195)
│   ├── cat/          (1,009)
│   └── uncertain/    (7,700)
└── _keep/                  Keep
    ├── food/         (1,682)
    ├── landscape/    (1,991)
    └── other/        (2,075)

Category	Count	Share
people	7,195	29.4%
uncertain (low confidence)	7,700	31.4%
other	2,075	8.5%
landscape	1,991	8.1%
document	1,804	7.4%
food	1,682	6.9%
cat	1,009	4.1%
screenshot	981	4.0%
blank	59	0.2%

That's a lot of cat photos...

Let's see how CLIP actually judged some of these.

🎯 Big wins


My cat	A meal	App screenshot	Mountain (landscape)
cat 0.97	food 0.999	screenshot 0.74	landscape 0.98

CLIP nailed the cat without hesitation, food at 0.999, screenshots and landscapes too. Reliable.

✨ Subtly impressive recognition


Cat keychain	Close-up of coffee beans
cat 0.64	food 0.53

Even the keychain got recognized as "cat." And coffee beans up close as "food." Quietly impressive.

🤔 Funny misclassifications (CLIP's quirks)

Browsing thumbnails by category, some interesting patterns emerged.

Food edition: "Trash sorting chart" beats "homemade cake" for being food-like


My homemade cake	Trash sorting chart
food 0.57	food 0.83

Both ended up in the "food" category. Apparently the trash sorting chart looks more food-like to CLIP than my homemade cake. Reacting to the text? The table layout? Mystery.

People edition: "A doodle" beats "Mona Lisa" for being people-like


The Mona Lisa	A face I doodled myself
people 0.50	people 0.52

Both in the "people" category. My crappy doodle edges out da Vinci's Mona Lisa for being more "people-like" (just barely).

CLIP's quirks — kind of charming.

VLM's grading results

I asked the VLM, one photo at a time, whether CLIP's category was correct. For example, photos in the cat folder got "is this a cat?", food folder got "is this food?" — yes/no answers.

Summary by final destination bucket:

Final bucket	Count	VLM agreement
people	7,195	99.2% 🎯
food	1,682	95.3% 🎯
cat	1,009	95.0% 🎯
other	2,075	93.6% 🎯
landscape	1,991	83.5% ⚠️
screenshot	981	75.2% ⚠️
document	1,804	67.4% ⚠️
blank	59	52.5% ⚠️
OVERALL	24,496	84.5%

People detection at 99.2% is quietly amazing. Out of 7,195 photos, the VLM said "no" to only 59.

Documents and screenshots, on the other hand, came back "no" about half the time. CLIP-only confidence isn't enough for those. Out of 24,496 photos, 3,808 got a "no" from the VLM — that's the part CLIP alone wouldn't have caught.

💡 Today's discoveries

Multimodal AI runs at home

Both CLIP (400 MB, classifier) and Qwen2-VL (16 GB, conversational) ran fine on my home machine. Reassuring.

CLIP's confidence is a reliable signal

VLM agreement broken down by CLIP confidence:

CLIP confidence	Count	VLM agreement
0.9+ (super confident)	3,555	96.5%
0.7–0.9	6,285	93.5%
0.5–0.7	6,956	86.0%
<0.5 (uncertain)	7,700	70.1%

Boring but important: when an AI says it's confident, you can trust it.

CLIP's weak spots

Things that clearly appear in photos — people, food, cats, objects — score 95%+. Abstract or compound subjects — documents, screenshots, landscapes — drop to 60-80%.

Documents at 67.4% in particular. That's where VLM re-grading earns its keep.

Role split: lightweight model × smart model

Use CLIP to triage everything quickly, VLM to grade the suspicious cases — a two-layer setup. Best of both worlds in speed and accuracy.

Day 3 had the same pattern: "aggregation = tools, interpretation = AI." Today's variant: "rough sorting = CLIP, accuracy check = VLM." Picking the right AI for the right task pays off in both performance and cost.

"Input quality matters more than model size" struck again

In Day 3 (credit card analysis), I learned "input quality > model size." The same pattern showed up today:

VLM with original-resolution images: 16 sec/image (5 days for full run)
VLM with resized 448px images: 0.3 sec/image (2 hours)

Just by tidying up the input, 54x speedup — small change, huge impact.

Not "biggest model possible" or "raw original" — clean up the input before sending it to the AI. This worked in Day 3 and Day 4 in a row.

Heart broken, switched to Amazon Photos

I tried to verify the trash candidate folder, then realized I'd need to cross-reference VLM scores too, then realized I never set clear criteria for "what to delete" in the first place. Couldn't finalize the cleanup, and morale broke.

Right then I learned that Amazon Prime members get unlimited photo storage, so I just dumped everything into Amazon Photos. Lol.

That said, I really should have defined the deletion criteria before starting.

The classified data on the DGX is a useful resource for future Day experiments.

🛠️ How I actually did this

:::details Wi-Fi 0.5 MB/s → wired LAN 80 MB/s journey

myPC1 → DGX over 96 GB started at 236 KB/s via WinSCP (ETA: 6 days). The cause was myPC1 being on Wi-Fi.

I plugged the PC into the router with a LAN cable → ping dropped close to 0 ms. But WinSCP was still stuck at 500 KB/s.

PowerShell ping spark-XXXX.local revealed the address resolved to DGX's Wi-Fi-side IP. The DGX was dual-homed (wired + Wi-Fi), and mDNS was returning the old route.

# Failure (routes through Wi-Fi)
scp -r "C:\Users\[user]\Pictures\iCloud Photos\Photos" [user]@spark-XXXX.local:...

# Success (direct IP over wired LAN)
scp -r "C:\Users\[user]\Pictures\iCloud Photos\Photos" [user]@10.0.0.205:...

Switched from hostname to explicit IP and watched it scream:

IMG_0190.HEIC                100% 1812KB  84.3MB/s   00:00
IMG_0190.MOV                 100%   17MB 102.4MB/s   00:00
IMG_0192.HEIC                100% 2256KB  81.6MB/s   00:00

Also discovered WinSCP (SFTP-based) struggles with many small files, while scp (stream transfer) is much faster. With 25,382 files, scp won by a landslide.

:::

:::details Splitting photos and videos by extension

PHOTO_EXTS = {".jpg", ".jpeg", ".heic", ".heif", ".png", ".webp"}
VIDEO_EXTS = {".mov", ".mp4", ".m4v"}

for src in input_dir.rglob("*"):
    if not src.is_file():
        continue
    ext = src.suffix.lower()
    if ext in PHOTO_EXTS:
        shutil.move(str(src), str(photos_out / src.name))
    elif ext in VIDEO_EXTS:
        shutil.move(str(src), str(videos_out / src.name))

Simple. Caught one snag: right after transfer, the directory permission was dr-x------ (read-only), so the first shutil.move died with PermissionError. chmod u+w fixed it.

:::

:::details CLIP classification script

Used transformers to load openai/clip-vit-base-patch32. For each category, multiple captions are prepared, and the max softmax score is used:

CATEGORIES = {
    "screenshot": [
        "a screenshot of an app",
        "a phone screenshot",
        "a screenshot of a website or chat",
    ],
    "document": [
        "a photo of a document or paper",
        "a photo of a receipt",
        "a QR code or barcode",
        "a photo of an ID card or driver's license",
    ],
    "people": [
        "a photo of a person",
        "a photo of people",
        "a portrait of someone",
    ],
    "cat": ["a photo of a cat"],
    "food": ["a photo of food or a meal"],
    "landscape": [
        "a photo of a landscape or scenery",
        "a photo of a building or city",
    ],
    "other": ["a photo of an object or item"],
}

inputs = processor(text=text_prompts, images=images,
                   return_tensors="pt", padding=True).to(device)
with torch.no_grad():
    outputs = model(**inputs)
probs = outputs.logits_per_image.softmax(dim=1)

Anything below 0.5 confidence goes into _review/uncertain/. Near-black/near-white images get caught by a brightness check and routed to _trash-candidate/blank/ before they reach CLIP.

All per-image category scores are also saved to JSON. That JSON is what the VLM evaluation step consumes later.

:::

:::details The 54x speedup from image resizing for VLM

Qwen2-VL's vision token count scales with input resolution. Original-size images (several thousand pixels) consume hundreds to thousands of tokens, slowing inference dramatically.

processor = AutoProcessor.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    min_pixels=224 * 224,
    max_pixels=448 * 448,  # ← cap here
)

# Belt and suspenders — also pre-resize the image
img = Image.open(path).convert("RGB")
img = ImageOps.exif_transpose(img)
img.thumbnail((448, 448))

That took 16 sec/image → 0.3 sec/image.

The verification prompt is dead simple:

CATEGORY_PROMPTS = {
    "screenshot": "Is this image a screenshot of a phone screen, an app, or a website? Answer with one word: yes or no.",
    "document":   "Is this image primarily a document, receipt, ID card, or QR code? Answer with one word: yes or no.",
    "people":     "Does this image clearly show one or more human persons? Answer with one word: yes or no.",
    # ...
}

max_new_tokens=5 means only yes/no comes back. Minimal design.

:::

:::details Resumable checkpointing

Running 24,000 images for 3 hours straight, you really want recovery if something hiccups:

CHECKPOINT_INTERVAL = 100

for i, (name, r) in enumerate(todo):
    # ... inference ...
    if (i + 1) % CHECKPOINT_INTERVAL == 0:
        save_checkpoint(results, output_path)

And a --resume flag that picks up where the JSON left off:

if args.resume and args.output.is_file():
    with args.output.open() as f:
        results = json.load(f)
    print(f"Resumed from {len(results)} existing entries")
todo = [(name, r) for name, r in clip_data.items() if name not in results]

Essential for any overnight job.

:::

Next up: Day 5

Tomorrow: have an AI analyze a year of my Amazon purchase history.

Switching to Amazon Photos for storage made me realize Amazon also has my entire purchase history. What if I asked AI "what kind of person am I, based on this?" — see what patterns emerge that I never noticed myself.

To be continued ＞＞＞

100ExperimentsWithDGX #LocalLLM #ImageClassification #CLIP

[Day 3] I Had a Local LLM Analyze a Year of My Credit Card Statements

PEPPERCORN — Tue, 05 May 2026 22:52:50 +0000

[Day 3] I Had a Local LLM Analyze a Year of My Credit Card Statements

Intro

Day 3: I'm going to hand a year of credit card statements over to a local LLM and see what it can do.

This is experiment #3.

What I'm using today: DGX Spark + Ollama + Qwen2.5 (comparing 7B vs 72B). Ollama is the de-facto local-LLM runtime, and Qwen2.5 is a multilingual model from Alibaba (China) that handles Japanese reasonably well, apparently.

Today's setup

Data: 12 months of credit card statements from a single card.
Volume: 383 transactions, ¥2,761,555 in total spend.
Goal: get the AI to spot waste patterns and propose savings.
Comparison axes:
- Model size: 7B (light) vs 72B (heavy)
- Input format: raw CSV vs pandas-aggregated summary
- → 4 patterns total

Takeaway: "If you ask an AI to aggregate raw data, the numbers come out way off." / "If you pre-aggregate with a spreadsheet tool first and then feed the AI, you get fast and accurate results." A small but practical finding.

1. Get the CSVs onto the DGX

Log into the credit card company's web statements page on myPC1 (my Windows laptop), download 12 months of CSVs, then push them to the DGX.

I deliberately skipped GitHub for the transfer this time — once you push something, it's in the history forever, and credit card data shouldn't be there even briefly. Instead, I used direct PC-to-PC transfer over SSH (one command, finishes in seconds; details in the collapsibles at the end). The .gitignore excludes private-data/ too, so accidental commits are ruled out.

2. Install Ollama

Ollama is the de-facto runtime for local LLMs. One command should be enough.

There was a small password hiccup during install (details below), but eventually it was up and running.

The DGX Spark specs really show through:

Memory: 121 GB
Default context window: ~262,144 tokens

In other words: "throw a whole book at it, no problem" territory. Reassuring.

3. Two model sizes: Qwen2.5 7B vs 72B

The strategy: same model family, different sizes. That way the differences come from size, not architecture.

7B (light): ~4.7 GB, downloads in 5 minutes. Fast.
72B (heavy): ~47 GB, 25 minutes to download. Slow but smart.

What does "B" mean? Short for Billion. It's the number of "weights" inside the AI — more weights, more it remembers, basically. So 7B has 7 billion weights, 72B has 72 billion.

Loading both onto the DGX simultaneously, memory usage looks like:

AI model	Memory occupied
qwen2.5:72b	61 GB
qwen2.5:7b	8.2 GB
Total	69 GB

69 GB. Spacious!

4. Prepping the CSVs

Once I had the CSVs in hand, three small headaches before they were ready for the AI:

Headache 1: An older encoding (Windows Japanese flavor) → needs converting to modern UTF-8
Headache 2: Some merchant names contain commas, which breaks naive CSV parsing
Headache 3: Each file has a "monthly total" line at the end that isn't actually data

Details in the collapsible. After cleanup, the 12 files merge into a single dataset:

Item	Value
Transactions	383
Period	12 months (1 year)
Total spend	¥2,761,555
Avg per tx	¥7,210
Median per tx	¥3,000
Largest single tx	¥209,283 (overseas flight)
Smallest	¥-3,980 (refund)

Now to feed this to 7B and 72B and see what each of them says.

5. Experiment 1: Throw the raw CSV at the AI

No tricks: all 383 rows, straight at the AI. Prompt is the full ask: "As a household budget consultant, output category breakdown / monthly trend / waste patterns / savings suggestions / lifestyle hypothesis."

7B's answer (75 seconds)

...this is where the numbers go wildly off.

Item	What 7B said	Real data	Match?
Amazon total	¥2,014,386 (257 tx)	¥693,663 (166 tx)	❌
Amazon Downloads	¥2,014,386 (257 tx)	¥80,323 (50 tx)	❌
Outdoor brand	¥495,740	¥154,820	❌
A local recreation venue	"¥49,574" cited	(a different small charge actually exists)	❌

None of the numbers line up. Amazon total is roughly 3× off, Amazon Downloads about 25× off, and the cited venue context is a different charge entirely.

Reading 383 rows of CSV and computing totals turned out to be a heavy lift for the 7B model.

72B's answer (12m 9s)

What if we throw size at the problem? After 12 minutes of patience:

Item	What 72B said	Real data	Match?
Amazon total	¥635,792 (104 tx)	¥693,663 (166 tx)	❌
AI/dev tools	¥193,629 (21 tx)	¥176,850 (24 tx)	❌
Travel	¥487,555 (43 tx)	¥416,268 (8 tx)	❌

Not exact, but the off-by amounts are within ~10%, and there are no fabricated venues. A real improvement.

However — when asked about the monthly trend, here's what 72B said:

Month 1: ¥316,789 → Month 2: ¥229,600 → Month 3: ¥237,500 → ... → Month 12: ¥291,500
(Gradually increasing.)

The actual range is ¥69,961 (low) to ¥493,072 (high) — a chaotic up-and-down waveform. "Gradually increasing" isn't quite right. Even 72B isn't great at aggregating distributed data over a long CSV.

6. Experiment 2: Aggregate first, then feed the AI

If the AI struggles with aggregation, do the aggregation in a different tool first and only hand the AI the result.

The flow:

📥 Raw CSV (22,132 chars, 383 rows)
       ↓
🔧 Pre-aggregate with a spreadsheet tool (Python's pandas)
       ↓
📋 Aggregate summary (1,884 chars, ~90% smaller)
       ↓
🤖 Hand it to the AI (let it interpret and propose)

Python's pandas = a spreadsheet-like library, but ~10,000× more powerful than Excel functions, used for tabular data analysis.

7B + pre-aggregated input (50 seconds)

Numbers are fully accurate now.

Item	What 7B said	Real data	Match?
Amazon total	¥693,663	¥693,663	✅
AI/dev tools	¥176,850	¥176,850	✅
Monthly max	¥493,072	¥493,072	✅
Monthly min	¥69,961	¥69,961	✅

Quoting straight from the pre-aggregated numbers, the hallucinations vanished.

And 7B did this in 50 seconds — better quality than the 72B + raw CSV at 12 minutes. Quietly remarkable.

	Before (raw CSV)	After (aggregated)
Time	75s	50s
Numbers	wildly off	exact
Verdict	not usable as-is	quote directly

72B + pre-aggregated input (12m 13s)

72B's numbers also match exactly (well, since they're being quoted from pre-aggregated data, that's expected). The proposal quality was the strongest of the four patterns:

Reduce Amazon dependency

Current: online shopping (Amazon family) is 25.1% of total (¥693,663).

Suggestion: stick to essentials only, regular review, avoid impulse buys.

Expected savings: ¥57,805/month average (25% reduction) → ¥693,660/year

...wait, hold on. Annual Amazon spend was ¥693,663. The "savings" 72B suggests is ¥693,660. That's basically the same number. So the proposal is effectively "stop buying on Amazon entirely (100%)" — definitely not 25%. Apparently 72B's percentage arithmetic isn't bulletproof either.

That aside, the lifestyle hypothesis section was kind of striking. Here's what 72B observed:

Heavy reliance on apps and subscriptions: "App/subscription" category is 10.5% of total

Frequent international travel: "Travel/airline" is 15.1%, with notable overseas charges

Frequent online shopping: "Online (Amazon)" is 25.1% of total

It's just one card's data, so this isn't a complete picture — but if I fed an AI my full household financials, the analysis and advice would probably go a lot deeper.

Summary: 4 patterns

#	Model	Input	Time	Numerical accuracy	Proposal quality
1	7B	Raw CSV	75s	❌ Numbers way off	△
2	72B	Raw CSV	12m 9s	△ Misread monthly trend	○
3	7B	Aggregated	50s	✅ Exact	○ Some repetition
4	72B	Aggregated	12m 13s	✅ Exact	◎ Best (mind the % math)

Quietly notable: 72B takes ~12 minutes regardless of input size (shrinking the prompt didn't change wall-clock time much). Output generation is the bottleneck. Which strengthens the case for "small model + pre-aggregate" as the cost-effective default.

7. Cross-check: the actual graphs

Before trusting any of the AI output, let me put the real numbers on charts using the spreadsheet tool (pandas).

Monthly spending

Average ¥230,130/month, but the range is ¥69,961 (lowest) to ¥493,072 (highest) — about a 7× spread. The 72B's "gradually increasing" claim was a bit off the mark; the reality is bouncy.

Category share

"Other" being 32% is because my categorization rule is sloppy. I just wrote a simple "if the merchant name contains keyword X, bucket Y" rule, and lots of merchants didn't match any keyword and ended up in "Other." Reading meaning from a merchant name is exactly the kind of thing AI is good at, so next time I'll let the AI do the categorization itself.

Top 15 merchants

Amazon at ¥421,978 (105 tx) is far and away #1. Amazon really is too convenient...

Weekday rhythm

Tuesday alone is ¥692,549 — way above the rest. Probably because that's when most of the subscription auto-charges land.

8. Today's takeaways

Separate "aggregation" from "interpretation"

AI is bad at	AI is good at
Multi-row sum/average (numbers go wildly off)	Categorization (interpreting fuzzy meaning)
Percentage math (saw "25% off → 100% off")	Pattern recognition / hypothesis generation
Distributed aggregation like monthly totals	Narrative interpretation, savings proposals

→ Aggregation is the spreadsheet tool's job; interpretation is the AI's. When you split the work, things go fast and accurate. "Data prep matters before analysis" — yeah, that old saying really is true. Note to self.

Sometimes input quality beats raw size

"7B + pre-aggregated input in 50 seconds" outperformed "72B + raw CSV in 12 minutes". Sometimes you don't need a bigger model — you need cleaner input. Felt that one today.

The local-LLM angle

Feeding 12 months of raw credit card data to an AI without a single byte going to the cloud — it was surprisingly stress-free. This is one of the spots local LLMs really shine. Got personal info, or anything cloud-uncomfortable? This is the place for them.

9. Tech details (Claude explains)

The technical bits, written up by my AI pair.

SCP transfer to the DGX (mDNS, no IP needed)

NVIDIA Sync auto-configures a Host alias in ~/AppData/Local/NVIDIA Corporation/Sync/config/ssh_config:

Host spark-XXXX.local
  Hostname spark-XXXX.local
  User [user]
  Port 22
  IdentityFile "...\\nvsync.key"

Which means I can SSH/SCP using spark-XXXX.local without ever looking up an IP. The .local suffix uses mDNS (Multicast DNS) for hostname resolution within the LAN.

Transfer command (one line, from PowerShell on the Windows side):

scp -r "C:\Users\[user]\Desktop\docs\dgx\csv" spark-XXXX.local:/home/[user]/personal/dgx-100-experiments/private-data/credit-card-csv

Ollama install + the sudo-TTY catch + GPU detection log

Ollama install:

curl -fsSL https://ollama.com/install.sh | sh

Running this through Claude Code's Bash, it errored at the sudo password prompt — an interactive TTY is required there:

sudo: a terminal is required to read the password

Reopened a separate SSH session, ran the same command manually, and it went through.

Once installed, systemd auto-starts the service. The GPU detection log via journalctl -u ollama:

inference compute id=GPU-986c194b... name=CUDA0 description="NVIDIA GB10"
total="121.7 GiB" available="79.0 GiB"
default_num_ctx=262144

VRAM (DGX Spark unified memory): 121.7 GiB
Default context: 262,144 tokens

Compared with a typical RTX 4090 (24 GB VRAM, 8K–32K default context), the gap is significant.

Loading both models simultaneously

ollama pull qwen2.5:7b   # 4.7 GB
ollama pull qwen2.5:72b  # 47 GB

After loading both, ollama ps shows:

NAME           SIZE      PROCESSOR    CONTEXT    
qwen2.5:72b    61 GB     100% GPU     32768
qwen2.5:7b     8.2 GB    100% GPU     32768

Total ~69 GB used out of 79 GB available. Both models stay resident, switching between them is instant.

Custom CSV parser for the credit card data

Three quirks needed handling: CP932 encoding, no quotes (commas in some merchant names break parsing), and a trailing summary row in each file.

def parse_line(line: str) -&gt; list[str] | None:
    fields = line.rstrip("\r\n").split(",")
    if len(fields) &lt; 7 or not fields[0]:
        return None  # skip blank/summary rows
    if len(fields) &gt; 7:
        merchant = ",".join(fields[1:-5])
        fields = [fields[0], merchant] + fields[-5:]
    return fields


def load_one(path: Path) -&gt; pd.DataFrame:
    rows = []
    with path.open(encoding="cp932") as f:
        next(f)  # skip header (cardholder metadata)
        for line in f:
            parsed = parse_line(line)
            if parsed is not None:
                rows.append(parsed)
    df = pd.DataFrame(rows, columns=COLUMNS)
    df["利用日"] = pd.to_datetime(df["利用日"], format="%Y/%m/%d")
    df["利用金額"] = df["利用金額"].astype(int)
    return df

Japanese fonts in matplotlib

japanize-matplotlib doesn't work on Python 3.12 — it imports distutils, which was removed from the standard library.

The modern replacement is matplotlib-fontja:

pip install matplotlib-fontja

import matplotlib_fontja  # noqa: F401  ← just importing it sets up IPAexGothic

Calling Ollama from Python

The official ollama Python client is straightforward:

import ollama

client = ollama.Client()
stream = client.chat(
    model="qwen2.5:72b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_prompt},
    ],
    options={"temperature": 0.3},
    stream=True,
)
for chunk in stream:
    print(chunk["message"]["content"], end="", flush=True)

Streaming makes long generation easier to watch unfold.

Tomorrow: Day 4

Day 4 plan: let a local AI sort 20,000 iPhone photos.

The actual goal is to have a local image-recognition model (CLIP family?) clean up my photo library so I can stop paying iCloud for storage upgrades...!

100ExperimentsWithDGX #LocalLLM #Ollama

[Day 2] I Trained an AI on 22 Photos of My Cat — Now It Draws Her in Any Scene

PEPPERCORN — Tue, 05 May 2026 00:06:00 +0000

[Day 2] I Trained an AI on 22 Photos of My Cat — Now It Draws Her in Any Scene

So, yesterday I generated "some cat"

Day 1 ended with "I made my DGX draw a cat" — but the cat that came out was just "a cat from somewhere". Today, the goal is to teach the AI about my actual cat (who's currently being looked after at my parents' place back in Japan).

This is what people call LoRA training.

LoRA: A technique that teaches an AI model "specific features" using a small set of images, without touching the base model itself. Apparently. The output is a small "diff" file (tens of MB).

This is experiment #2.

The training data

Source material: 22 photos of my cat.

I picked a mix of angles — front-facing, full body, sleepy poses, varying lighting — to give the AI a fair shot at recognizing the cat's defining features (tuxedo black-and-white pattern, white socks, the black smudge on the nose).

Training pipeline

1. Pre-processing

iPhone HEIC files don't work directly with most AI tools, so first conversion to JPG. 10 of the 22 were HEIC.

Then resize to 512px on the short side for training. This is where I tripped over a sneaky bug — details in the collapsible section below.

2. Captions

Every image gets a text description like "ohwx cat, sitting on a wooden floor, indoor, soft lighting". The four-letter ohwx is a meaningless token that becomes the trigger word for "my specific cat" after training.

Drafting 22 captions by hand would be tedious — but Claude can read images directly, so it drafted them while I just reviewed. The accuracy was uncanny. For example:

ohwx cat, walking on a metal kitchen counter, side profile, indoor kitchen with spice bottles and shelves in the background

ohwx cat, in a loaf pose on a gray carpet, mouth open showing teeth, mid-yawn, indoor with shelves and warm lights in the background

ohwx cat, sitting on a wooden floor by a balcony window, viewed from behind, sharp sunlight casting long shadows, indoor

SUGOI.

3. Kohya_ss training

Kohya_ss is the de-facto LoRA training tool. Set up a TOML config, run one command:

$ accelerate launch train_network.py \
    --config_file configs/train.toml \
    --dataset_config configs/dataset.toml

Training logs scroll by, and the loss value gradually drops. Lower loss = the model is learning, apparently.

4. Done

1100 steps in 13 minutes 3 seconds on the DGX Spark.

Result 1: just typing "ohwx cat" gives me my cat

The first thing I tried was a "without LoRA vs with LoRA" comparison. Same prompt — "ohwx cat as a chef in a kitchen, ..." — first without the LoRA, then with it:

Left: no LoRA. Right: with LoRA.

Without LoRA, ohwx is gibberish to the model, so it's ignored and only "a chef in a kitchen" carries weight. Result: a human chef. A nice woman cooking in a pink kitchen.

With LoRA, ohwx becomes a real token that points at my cat. Same prompt, but now my cat is the chef.

This was the moment that hit.

Result 2: novel scene reproduction

The training set has no photo of the cat sitting on a wooden floor in this exact composition. So I tried it:

White socks: present. Nose smudge: present.

My cat, in places she's never been

ohwx cat in various scenes.

Sunny balcony

Cozy.

Chef (reprise)

The chef hat fits suspiciously well. Cooking ability unverified.

Autumn forest

A painterly take.

Astronaut

A doppelgänger via the helmet glass — but sci-fi all the same.

Today's takeaway

"Build your own AI from your own data" turned out to be way more accessible than I'd assumed.

Tech details (Claude explains)

The technical bits, written up by my AI pair.

HEIC → JPG conversion and the EXIF orientation trap

Reading iPhone HEIC files in Python is straightforward with pillow-heif. JPG conversion is a few lines:

from PIL import Image, ImageOps
from pillow_heif import register_heif_opener
register_heif_opener()

with Image.open("IMG_1234.HEIC") as img:
    oriented = ImageOps.exif_transpose(img)  # ← critical line
    rgb = oriented.convert("RGB")
    rgb.save("IMG_1234.jpg", quality=95)

What I tripped on

My first version skipped ImageOps.exif_transpose(). Result: 8 of 22 photos came out rotated 90° in the resized output.

iPhones save portrait shots with the actual pixels stored landscape-ways, plus an EXIF Orientation tag saying "rotate 90° on display". Pillow's default Image.open() ignores that tag — you have to call exif_transpose() explicitly.

Caught it before training started. If I hadn't, the LoRA would have learned "sideways cat" and generation would be weird.

Kohya_ss setup on ARM64 (DGX Spark)

There are two repos commonly referred to as "Kohya_ss":

bmaltais/kohya_ss — GUI wrapper, xformers dependency (clashes with ARM64)
kohya-ss/sd-scripts — the actual training engine, CLI/TOML driven

DGX Spark is ARM64, so I went with the latter:

git clone --depth 1 https://github.com/kohya-ss/sd-scripts.git ~/Kohya_ss
cd ~/Kohya_ss
python3 -m venv venv &amp;&amp; source venv/bin/activate
pip install --upgrade pip
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

DGX Spark uses CUDA 12.8 + ARM64 (sbsa), so the PyTorch cu128 channel works directly. Surprisingly painless.

Training config (TOML)

# train.toml (excerpt)
pretrained_model_name_or_path = ".../Realistic_Vision_V6.0_NV_B1.safetensors"
vae = ".../vae-ft-mse-840000-ema-pruned.safetensors"

network_module = "networks.lora"
network_dim = 32
network_alpha = 16

optimizer_type = "AdamW8bit"
unet_lr = 1e-4
text_encoder_lr = 5e-5
lr_scheduler = "cosine_with_restarts"

max_train_epochs = 10
save_every_n_epochs = 2

mixed_precision = "bf16"
sdpa = true
cache_latents = true

# dataset.toml
[general]
shuffle_caption = false
caption_extension = ".txt"
keep_tokens = 1

[[datasets]]
resolution = 512
batch_size = 2
enable_bucket = true

  [[datasets.subsets]]
  image_dir = "/path/to/cat-photos-512"
  num_repeats = 10

22 photos × 10 repeats × 10 epochs ÷ batch 2 = 1100 steps. 13 minutes.

Base model: Realistic Vision V6.0 B1 noVAE (a photo-realistic SD 1.5 derivative). External VAE: sd-vae-ft-mse-original. The combination is good at fur detail.

Hitting the ComfyUI HTTP API for batch generation

Clicking through the GUI for one image at a time gets old fast. ComfyUI exposes an HTTP API that's easy to drive from Python — urllib.request from the standard library is enough (no extra deps).

import json, urllib.request, time

COMFY_URL = "http://127.0.0.1:8188"

def queue_prompt(workflow):
    payload = json.dumps({"prompt": workflow}).encode()
    req = urllib.request.Request(
        f"{COMFY_URL}/prompt",
        data=payload,
        headers={"Content-Type": "application/json"},
    )
    return json.loads(urllib.request.urlopen(req).read())["prompt_id"]

def wait_for_history(prompt_id, timeout=180):
    start = time.time()
    while time.time() - start &lt; timeout:
        with urllib.request.urlopen(f"{COMFY_URL}/history/{prompt_id}") as resp:
            data = json.loads(resp.read())
            if prompt_id in data:
                return data[prompt_id]
        time.sleep(0.5)

The workflow is ComfyUI's API format (a dict of node IDs with their connections). To use a LoRA, insert a LoraLoader node between the checkpoint loader and KSampler.

DGX Spark generates one 512×768 image in about 3 seconds. With seed/strength/prompt parametrized in a script, all 12 grid images came out in under a minute.

Tomorrow: Day 3

Day 3 plan: have a local AI analyze my credit card history.

The kind of data I'd rather not send to a cloud AI, but absolutely want to understand. Quintessential local-AI territory.

100ExperimentsWithDGX #LocalLLM

[Day 1] DGX Spark Came Home — I Made It Draw a Cat

PEPPERCORN — Mon, 04 May 2026 03:20:48 +0000

[Day 1] DGX Spark Came Home — I Made It Draw a Cat

So... what is "local LLM" again?

Honestly, I'm still figuring out what "local LLM" even means. But somehow, through a series of decisions I won't fully justify here, I ended up buying an NVIDIA DGX Spark — and now it's sitting in my house.

DGX Spark: NVIDIA's "supercomputer for the home" — a small but seriously expensive box with the latest-gen AI chip inside. Apparently.

What I really want to figure out is: when should I use local AI vs. cloud AI? Reading articles about it doesn't seem to help, so I'm going full hands-on. Goal: 100 experiments, one per day-ish, until I have an evidence-based answer.

This is experiment#1.

First, the hardware

So this is what showed up at my door — solidly packed in a sturdy cardboard box.

When I opened it, I was surprised at how small it actually is. "This is the AI machine?" kind of small.

Boot up → Initial OS setup

Power on, and an Ubuntu-based DGX OS 7.5.0 boots up.

Welcome screen

"Get started" — yes, please.

Language and timezone

Standard Linux installer territory — same as Ubuntu?

Privacy settings

Diagnostic data sharing prompts.

System update

The moment I plugged it in, it started updating itself. Modern Linux being Linux.

Setup complete

I picked a username and let the hostname auto-assign. DGX-side prep done.

Connecting from my Windows PC

Plugging a monitor into the DGX every time would be tedious, so I want to SSH in from my regular Windows machine (which I've nicknamed "myPC1").

NVIDIA provides a desktop app called NVIDIA Sync that's supposed to make SSH setup painless. So I install it.

…and that's where I fell into a trap big-time. Windows OpenSSH refused to connect with a "your SSH config has weird permissions, can't trust it" error.

Full troubleshooting steps are in the collapsible "Tech details" section below.

Inside the DGX, finally

After much wrestling, I made it inside. Here's the rough lay of the land:

Item	Spec
GPU	NVIDIA GB10 Grace Blackwell
Memory	128GB (unified between CPU and GPU)
Storage	4TB SSD (basically empty)
CPU	20 cores (perf + efficiency combo)
Idle power	4W (yes, four)

128GB of memory is apparently 8–16x what's in a typical laptop.

Setting up image generation → 🐱

This is the main event. I'm setting up ComfyUI to generate the first cat from this DGX.

The ComfyUI interface looks like this:

The "boxes connected by cables" view is intimidating at first, but the default workflow is pre-wired. You just type a prompt and hit Queue Prompt.

So:

a cute fluffy cat sitting on a sunny windowsill, photorealistic, high detail, beautiful lighting, soft fur, cinematic, masterpiece, best quality

A few seconds later...

🐱 There it is — the very first cat my DGX has ever drawn!

Tweaked the prompt and made some more.

Eyes a bit unsettling but yeah, fluffy cat.

Going a touch dark there.

…is this a cat? It feels artistic though.

Distinctive composition.

Each masterpiece takes a few to a dozen seconds. That speed means I can iterate on prompts without thinking about cost — which turned out to be quite addictive.

Tech details (let the AI explain it)

The rest is the technical stuff. Read on if you're curious.

I'm a non-engineer poking at this stuff for the first time, so I had Claude (my AI pair programmer for this challenge) write up the technical details. Hopefully useful for anyone walking the same path.

How to actually get SSH working on Windows

NVIDIA Sync should generate an SSH keypair, register the public key on the DGX side at ~/.ssh/authorized_keys, and let you connect without a password.

If it doesn't work, the cause is usually permissions on Windows SSH config files.

Symptom

$ ssh spark-XXXX.local
Bad permissions. Try removing permissions for user: [PC]\CodexSandboxUsers
on file C:/Users/[user]/.ssh/config.

If you've installed Codex CLI or similar sandboxing tools in the past, the [PC]\CodexSandboxUsers group may have inherited permissions on ~/.ssh/.

Fix (run from an elevated PowerShell)

Use environment variables to avoid hard-coding your username/PC name.

# Take ownership
takeown /f "$env:USERPROFILE\.ssh\config"
icacls "$env:USERPROFILE\.ssh\config" /grant:r "$($env:USERNAME):F"

# Disable inheritance and remove the bad user
icacls "$env:USERPROFILE\.ssh\config" /inheritance:d
icacls "$env:USERPROFILE\.ssh\config" /remove "$env:COMPUTERNAME\CodexSandboxUsers"

Use /inheritance:d rather than /inheritance:r — :r strips all permissions, locking yourself out.

NVIDIA Sync's internal config files need the same treatment

~/.ssh/config Includes an NVIDIA Sync config file, and that one inherits the same problem.

$cfg = "$env:LOCALAPPDATA\NVIDIA Corporation\Sync\config\ssh_config"
icacls $cfg /inheritance:d
icacls $cfg /remove "$env:COMPUTERNAME\CodexSandboxUsers"

$key = "$env:LOCALAPPDATA\NVIDIA Corporation\Sync\config\nvsync.key"
icacls $key /inheritance:d
icacls $key /remove "$env:COMPUTERNAME\CodexSandboxUsers"

Ghost SIDs that icacls can't remove

If you have SIDs from deleted user accounts lingering, icacls /remove won't touch them. You need PowerShell ACL manipulation:

$cfg = "$env:LOCALAPPDATA\NVIDIA Corporation\Sync\config\ssh_config"
$acl = Get-Acl $cfg
$badRules = $acl.Access | Where-Object {
    $_.IdentityReference.Value -like "S-1-5-*" -and
    $_.IdentityReference.Translate([System.Security.Principal.NTAccount]) -isnot [System.Security.Principal.NTAccount]
}
$badRules | ForEach-Object { $acl.RemoveAccessRule($_) | Out-Null }
Set-Acl -Path $cfg -AclObject $acl

After this, ssh spark-XXXX.local connects on the first try (replace XXXX with your hostname).

Commands to check DGX specs

# GPU
$ nvidia-smi
NVIDIA-SMI 580.142    Driver Version: 580.142    CUDA Version: 13.0
GPU 0: NVIDIA GB10    36C    P8    4W / N/A

# OS
$ uname -a
Linux spark-XXXX 6.17.0-1014-nvidia ... aarch64 GNU/Linux

# Memory
$ free -h
Mem: 121Gi  2.6Gi  118Gi

# Storage
$ df -h
/dev/nvme0n1p2  3.7T  47G  3.5T  2%  /

# CPU
$ lscpu
Architecture:  aarch64
CPU(s):        20
Model name:    Cortex-X925 + Cortex-A725

Notable bits:

CUDA 13.0 (latest)
aarch64 (ARM64) architecture — yes, the DGX is ARM
121Gi (≈128GB) unified memory
20 cores in big.LITTLE layout (10 perf + 10 efficient)

ComfyUI installation steps

Following the official NVIDIA Comfy UI playbook.

# Virtual environment
cd ~
python3 -m venv comfyui-env
source comfyui-env/bin/activate

# PyTorch with CUDA 13.0
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu130

# ComfyUI itself
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt

# Model (SD 1.5, ~2GB)
cd models/checkpoints/
wget https://huggingface.co/Comfy-Org/stable-diffusion-v1-5-archive/resolve/main/v1-5-pruned-emaonly-fp16.safetensors

# Launch server
cd ~/ComfyUI
python main.py --listen 0.0.0.0

Key packages installed:

torch 2.11.0+cu130
cuDNN 9.19
NCCL 2.28
transformers 5.7.0
comfyui-frontend-package 1.42.15

Open http://spark-XXXX.local:8188 from your Windows PC's browser to access ComfyUI (XXXX is your hostname).

Download speed

The 2GB model came down at 40.6 MB/s in 50 seconds from HuggingFace's CDN. About half of my home 1Gbps LAN.

Tomorrow: Day 2

Day 2 plan: Train a LoRA on photos of my actual cat.

Today's SD 1.5 only knows "some cat from somewhere". With LoRA fine-tuning, I should be able to teach it about my specific cat. That kind of personalization feels like the killer feature of running locally.

100ExperimentsWithDGX #LocalLLM