Intro
Day 17!
Today I collected everything I've ever said to an AI and had a local model read it back to me. Seventeen months of history — then a self-audit of how I actually use these tools, and where the easy wins are.
| What I collected | Messages | Mine only |
|---|---|---|
| ChatGPT (browser) | 10,002 | 4,964 |
| Claude (browser) | 10,937 | 5,482 |
| Terminal (Claude Code, Codex CLI) | 12,530 | 3,518 |
| Total | 33,469 | 13,964 |
What I used: DGX Spark (my home AI machine) / qwen2.5 (the local model doing the analysis).
Usage drifted from browser to terminal over time.
Where the history lives
ChatGPT (browser)
- Open Settings from the account icon (top right)
- Go to Data controls
- Hit Export data → confirm
- A link arrives by email; download the zip from there
Claude (browser)
- Open Settings from your name (bottom left)
- Go to Privacy
- Hit Export data
- A link arrives by email; download the zip from there
Neither one is instant — you wait a bit for the email.
Terminal (Claude Code, Codex CLI)
No request needed. The logs are already on your machine — just copy them.
# Claude Code
~/.claude/projects/<per-project>/*.jsonl
# Codex CLI
~/.codex/sessions/<year>/<month>/<day>/*.jsonl
What you get from each
| Browser export | Terminal logs | |
|---|---|---|
| How to get it | Request, wait for email | Just copy the files |
| Conversation text | Yes | Yes |
| Image / attachment files | Yes (563 for me) | No |
| Shared-link conversations | Yes | — |
| Project settings & docs | Yes | Yes |
| Record of what the AI actually did | No | Yes |
Note that the browser export does not contain your terminal history, and vice versa. If you work across two machines, collect from both.
Audit 1: what I use it for
First, what was I actually using AI for? I had qwen2.5 read each conversation and tag it one by one ("this is research," "this is coding").
The big ones were research & learning (291), writing & editing (236), coding (193), and setup & troubleshooting (151) — two-thirds of everything.
Split by tool, it separates cleanly:
| Browser 1,267 convos |
Terminal 75 convos |
|
|---|---|---|
| Coding | 12.0% | 54.7% |
| Setup & troubleshooting | 10.9% | 17.3% |
| Research & learning | 22.7% | 5.3% |
| Writing & editing | 18.1% | 9.3% |
Terminal is 70% code-related; browser is 40% research and writing. The topics themselves split neatly between the two tools.
One caveat: the terminal side is only 75 conversations, so read it as a rough tendency, not a precise number. (Why so few? See "Not keeping work data in the first place" below.)
Audit 2: how I ask
Next, how I phrase requests. I counted phrasing patterns as a proxy for care. The yardstick is Anthropic's AI Fluency Framework.
| Phrasing I counted | Browser | Terminal |
|---|---|---|
| Specify steps or format | 21% | 40% |
| Reject the output | 11% | 22% |
| Ask for the reasoning | 8% | 24% |
| Ask it to verify | 6% | 35% |
In the terminal, my instructions are much more detailed.
(A fair reading: the terminal is where I do code work, and code work naturally invites step-by-step instructions and verification — so this may be a difference in task, not in skill.)
Audit 3: which features I use
The terminal logs record what the AI actually did. Here's the operation history:
| Most-used operations | Count |
|---|---|
| Bash (run a command) | 4,053 |
| Edit (change a file) | 2,844 |
| Read (read a file) | 2,443 |
| PowerShell | 1,001 |
| Write (create a file) | 612 |
| WebSearch | 460 |
So the reality is: read, edit, run a command. Nothing fancy.
And here's how often I reached for the fancier stuff. Turns out I barely write my own skills...!
| Feature | Usage |
|---|---|
| Agents (the AI spawns helpers) | 78 times, 5,542 helper messages |
| MCP (connect to outside services) | 80 times |
| Skills (built-in) | 6 times |
| Skills (my own) | 0 |
| Plan mode (approve a plan before it runs) | 1 time |
/model (switch model) |
82 times |
Agents and MCP I already lean on. Skills and plan mode? Almost untouched.
The easy wins
Three obvious places to improve:
1. Turn routine work into a Skill.
Zero of my own. My article pre-publish checklist runs the same way every time — I'd rather call it by name than re-ask by hand each time.
2. Use plan mode before big jobs.
Once in 17 months. I keep course-correcting mid-run; I'd rather see the plan first.
3. Move all coding into the terminal.
12.0% of it still happens in the browser, where the AI can't touch files directly — so I waste round-trips copy-pasting.
The details
What I counted, and what I left out
I counted only human↔AI text exchanges. I dropped:
-
agent-*.jsonl/journal.jsonl— internal work logs the AI keeps - Sub-agent (AI helper) messages, 5,542 of them — not typed by a human
- Auto-inserted text like
<system-reminder>— same reason - Image/attachment contents (563
.datfiles) — text only this time
One thing I missed: Claude's design_chats (3 conversations) got skipped because my loader only reads conversations.json. It's just one message, so the numbers don't move — but I can't claim I read the entire export.
What counts as a "phrasing"
I picked these up with regular expressions:
| What I counted | Example words (Japanese source) |
|---|---|
| Specify steps or format | first / next / finally / "in a table" / bullet points / JSON |
| Reject the output | wrong / not that / that's off / not working |
| Ask for the reasoning | really? / is that right / evidence / source / why |
| Ask it to verify | test / run it / verify / double-check / reproduce |
It's counting surface phrasing, not actual competence — treat it as a rough proxy. The comparison used conversations from May 2026 onward, where I use both tools: 348 browser vs 102 terminal.
Not keeping work data in the first place
Nearly half the history was work. That isn't even really my data — it belongs to the other party.
At load time I classify by folder name; for work, I throw away the body and keep only a character count.
cat = classify(project)
# keep the body only when it's confirmed personal
body = (text or None) if cat in ("personal", "web") else None
The classification keywords are literally client/partner names, so they don't live in the code — they're read from a config file that never enters Git. The classifier itself is more sensitive than the data it classifies — an obvious point I only noticed during the pre-publish check.
Because of this design, I could only read 75 of the 12,530 terminal conversations. A design that protects me also trimmed my own analysis.
The measurement flipped three times
My first pass showed every metric dropping. Read naively: "I got sloppy." But it was the denominator's fault.
| How I measured | What it showed | The problem |
|---|---|---|
| Rate per message | All metrics fall | Shorter messages drag it down automatically |
| Rate per conversation | Still falls | Picking up the shifting browser/terminal mix |
| Split browser vs terminal | Terminal higher on everything | ← adopted this |
I couldn't compare past-me to present-me directly: the tools swapped underneath me, so a skill difference and a place difference get tangled together.
A junked experiment: blind-judging old vs new requests
I hid the dates and had qwen2.5 compare an old request against a new one. Result: old won 10, new won 3. But it was browser-vs-browser, so I was really comparing "browser when I used it seriously" against "browser after it became an afterthought." It says nothing about whether I improved. (I judged each pair twice with the order flipped, counting a win only when both agreed — a fix for the position bias I found in the Day 14 cat-meow quiz.)
Speed notes, since I got stuck:
| Snag | What happened |
|---|---|
| Big model won't fit | qwen2.5 72B: only 39.2 of 62.9 GB on the GPU → 7 min per judgment |
| Model switching | The previous model lingers; timeouts right after a switch |
| Prompt length | 621 chars = 8s, 918 chars = 22s, 1,500 chars didn't finish in 60s |
Switching to 32B (19 GB) fit entirely on the GPU → a few seconds each. Fitting or not fitting changes the order of magnitude. And the scoring isn't very reliable: of 25 browser-vs-browser pairs only 16 were usable, and for browser-old vs terminal-new, 17 of 25 timed out. No cloud API was used, so this analysis cost $0 (electricity aside).
Sources
- The AI Fluency Framework — Rick Dakan, Joseph Feller, and Anthropic (CC BY-NC-SA 4.0): https://aifluencyframework.org/
Outro
Line up 17 months and you can see exactly what you were doing each month. Re-uploading all of that somewhere to analyze it feels like a bit much. Keeping it on my own machine is well suited to moments like this.
Soon, I'd like to build a Skill of my own.
Thanks for reading!
Top comments (1)
This is the kind of audit I wish more people ran on themselves instead of on the model.
I'm a non-developer who ended up building a pile of small internal tools with AI, and almost every improvement to how I work came from noticing a pattern in my own usage, not from a new feature. Two of your findings match mine exactly:
Auditing your own transcripts beats guessing. The unused features are usually sitting right there in the gap between what you meant and what you actually typed.