I finally started my startup.
For the last 5 months I have been building it, on an i3 4th gen with 8GB RAM. It is called MegaBit.
And we just hit our first real milestone - 45 coding datasets created. Let me explain what it is, why I am building it, and why it has been hell.
The problem no one talks about
AI coding models are trained on the wrong thing. They see millions of git diffs. They see the final answer. But they never see how a real developer got there.
They do not see the 20 files you opened, the 3 commands that failed, the test you broke, the fix, the retry(34th times). A diff shows you WHAT changed. It never shows you HOW. AND YOU WANT TO TEACH THE MODEL THE "How", so it can do it by itself later.
That process is everything. That is how humans learn to code. Meanwhile public human data is running out. Stanford calls it peak data. The dataset market is already worth $3.19B and synthetic data is the only way forward.
So I thought - what if we reconstruct that missing process?
What MegaBit actually is
MegaBit is a teacher factory for AI coding assistants. One sentence: It takes a real code change from a real open source project and makes an AI agent recreate that change from scratch inside a real sandbox. And we record everything(including failures - and that is crucial).
Input: Two diffs from a public GitHub repo. Diff 1 is the before state. Diff 2 is the final human change.
Process: An AI agent gets a ticket, failing tests, and has to code its way from Diff 1 to Diff 2 inside a real Docker sandbox.
Output: A full trajectory. Every reasoning step, every tool call, every file edit, every real terminal output, every error and retry. That becomes pretraining data.
No faked outputs, NEVER EVER.
How it works under the hood
This is the flow I built:
mining -> tasks -> harness (agent in sandbox) -> verify -> export
1. The Miner
It clones any public repo and scans the last 500 commits. It filters for good tasks:
- 1 parent only, no merges
- 1 to 8 files changed
- 15 to 400 lines changed
- no binaries, no lockfiles, no vendor folders
Every task saves base commit, target commit, license, file list, everything as
data/tasks/<id>.json
2. The Sandbox
Real execution. I have two backends - Docker and local. The Docker image is Python 3.11 slim with git, ripgrep, pytest. Every command the agent runs actually runs. If pytest is missing, you get a real No module named pytest error. And we capture that. Because that is real data.
3. The Agent
I forked an open source Go agent harness and added a headless run command. It streams NDJSON events. It can do file reads, bash, edits. The prompt system explicitly tells it never to fabricate tool outputs.
4. The Simulation
This is what makes it different. It is not just "here is a task, solve it".
- A ticket generator LLM creates a real Jira style ticket from the change, without ever seeing the solution
- We seed failing tests and commit them as baseline, then lock them with chmod 444 so the agent literally cannot edit them
- It goes through code review rounds with another LLM reviewer(at the end)
5. Honest Process Guarantees
I was obsessed with not letting the agent cheat(deepseek mocked me almost everytime, but now it cannot):
- Workspace history is erased. Only an initial state commit exists. Agent cannot git log the answer
- Ticket and reviewer LLMs never see the target diff
- Anti shortcut gates - if an episode has less than 3 tool calls or less than 1 bash call or similarity under 0.45, it gets rejected
- Verification re-derives everything from the git state, not from what the model claims
6. Verification and Export
We score the agent's produced diff against the real human diff. Our first live E2E test on karpathy/micrograd commit 315a2cb passed with 0.9512 similarity, 26 tool calls, 9 real bash calls. The first bash call failed with a real error and we captured it verbatim. BUT, this first one had some negative and CRITICAL issues, so I deleted it from github.
Then we export to pretraining JSONL with full provenance.
CLI is simple:
megabit mine https://github.com/karpathy/micrograd
megabit run data/tasks/xyz.json
megabit verify data/episodes/xyz.episode.jsonl
Where we are right now
45 datasets done. That took a lot of grinding.
Our biggest bottlenecks are stupid but real - free API limits. Groq free tier has 8000 TPM which is too small for full episodes. And my machine. I3 4th gen, 8GB RAM. Everything takes 10 times longer.
Right now I am fine tuning a small Qwen model on Kaggle just to prove this works. I want to come back with real eval numbers that impress people enough that companies actually want to buy this.
The part I feel shame to post about
This is not sexy.
I am not sitting in a flat in San Francisco with a Mac Studio and VC money and free coffee.
I am sitting in front of this old PC for tens of hours a day, alone. Debugging sandboxes, watching API rate limit errors, restarting episodes.
No funding. And being under 18 makes everything annoying. You cannot get a proper payment setup, you cannot get credits, every form has an age check. If I figure out any solution, there is a limitation waiting for me.
But everything is good. We are moving.
What is next
I will post here again when the Qwen fine tune is done. I want to show you the performance upgrade after being trained on just those 45 datasets.
If you are working on coding models, trajectory data, or you are just curious, the repo is open:
https://github.com/Sumama-Jameel/megabit-trajectories
If you want to mine tasks yourself, YOOO??? YOU CANNOT! Megabit is private, and it had ALOT of upgrades in some previous weeks that I didn't explained. But I WILL explain its architecture, how it works, etc. ONLY IF PUBLIC DEMANDS(I mean yeah bro, I can't spend 30mins for 25 views).
Love you all. Thanks for reading this far.
Top comments (0)