SnapCode 🦥: I stopped retyping code from lecture slides. One hotkey, offline, Gemma does the rest

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 3

What I Built

In our machine learning practicals, the lecturer shows code on the slides: imputers, pipelines, encoders, one cell after another. We're supposed to run it in Colab ourselves. Until this week, my routine for every single cell was:

1. Take a screenshot
2. Open it in Google Lens
3. Copy the text
4. Paste it into Colab
5. Fix the indentation Lens threw away, the ‘curly quotes’ that break Python, and the [[ that turned into [
6. Run it, get a SyntaxError, and go back to step 5

That's about a minute per cell, plus an internet connection the lab Wi-Fi doesn't always give you. A sloth would never.

So I built SnapCode. Press a hotkey, drag a box around the code, and press Cmd+V. The clean, correctly indented code is already on your clipboard. Everything runs on my laptop, with no internet and no account, using open-source OCR and an open-weight model.

Demo

Rendered from a real SnapCode run on a slide from my practical. The output and the 12.0 s timing are real; the timer is sped up in the video. MP4 version

Hotkey → drag → Cmd+V. A 🦥 notification tells you when it's ready, for example "24 lines copied via gemma4:E4B in 12.0s".

Before / after: why plain OCR isn't enough

I ran SnapCode on real screenshots from my practical. Here is a slide with three Colab cells (mean, median and mode imputation).

Tesseract on its own (open-source OCR, raw output):

Mean replacement
12]                  .      ;      ;
ne      from sklearn.impute import SimpleImputer
mean_imputer = SimpleImputer(strategy="mean")
X_train[ ‘Preparation Time _min’] = mean_imputer.fit_transform(
X_train[[‘Preparation_Time_min’]]
)
X_test[‘Preparation_Time_min’] = mean_imputer.transform(
X_test[[ ‘Preparation _Time_min*]]
)
Median replacement
3)         a         a                      7

Cell numbers and headings are mixed into the code. The column name is misread two different ways (Preparation Time _min, Preparation _Time_min*). The quotes are curly, and all indentation is gone. Pasting this into Colab gives you errors to fix, not working code.

SnapCode (Tesseract + Gemma):

from sklearn.impute import SimpleImputer
mean_imputer = SimpleImputer(strategy='mean')
X_train['Preparation_Time_min'] = mean_imputer.fit_transform(
    X_train[['Preparation_Time_min']]
)
X_test['Preparation_Time_min'] = mean_imputer.transform(
    X_test[['Preparation_Time_min']]
)
median_imputer = SimpleImputer(strategy='median')
X_train[['Distance_km']] = median_imputer.fit_transform(
    X_train[['Distance_km']]
)
# ...mode imputer cell, also exact

It matches the slide exactly. Here's a second one, a ColumnTransformer pipeline:

Tesseract:

numeric features = X_train.select_dtypes(include=[‘int64', ‘float64']).columns
numeric pipeline = Pipeline([
(‘imputer*, SimpleImputer(strategy="median")),
(‘scaler’, StandardScaler())
])

SnapCode:

numeric_features = X_train.select_dtypes(include=['int64', 'float64']).columns
numeric_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

Results on 4 real lecture screenshots

Screenshot	Lines	Exact match?	Time (Apple M5, offline)
3 imputer cells	22	✅	15.6 s
ColumnTransformer pipeline	24	✅	12.0 s
OneHotEncoder cell	11	✅	4.1 s
One-line print	1	✅	1.3 s

The lazy payoff

My old Google Lens routine took about 60–90 seconds per cell, counting the fixes. A practical has around 15–20 cells, so that's 15–25 minutes per session spent copying code instead of understanding it. With SnapCode it's a hotkey and a 5–15 second wait, during which I don't have to do anything. That's real sloth energy. 🦥

Code

Sachindu-Nethmin / snapcode

Hotkey → drag over code on screen → Cmd+V. Offline OCR with Tesseract + Gemma via Ollama 🦥

SnapCode 🦥

Press a hotkey, drag a box around code on a lecture slide or video, then press Cmd+V. The clean, correctly indented code is already on your clipboard.

Everything runs locally and works without internet:

Step	Tool	Why
Select region	macOS screencapture -i	built in
Rough OCR	Tesseract (Apache-2.0)	fast first draft
Fix + re-indent	Gemma (open weights) via Ollama	reads the image, fixes [[ ]], quotes, underscores, indentation
Verify	Python ast	on a syntax error, Gemma gets one retry with the error message
Paste	pbcopy	Cmd+V into Colab or your IDE

Setup

brew install tesseract ollama
ollama pull gemma4:E4B

Hotkey (Raycast): Settings → Extensions → Script Commands → Add Directory → choose snapcode/raycast, then give SnapCode a hotkey (for example ⌥⇧C). The first run asks for Screen Recording permission for Raycast.

CLI

python3 snapcode.py                 # select a region
python3 snapcode.py --image a.png   # use an existing

…

View on GitHub

The whole thing is one Python file (standard library only, no pip install) plus a Raycast script command for the hotkey. Try it:

brew install tesseract ollama
ollama pull gemma4:E4B
python3 snapcode.py   # select a region; the code lands on your clipboard

For the hotkey, add the repo's raycast/ folder under Raycast → Script Commands and give SnapCode a shortcut.

How I Built It

hotkey (Raycast) → screencapture -i → Tesseract → Gemma (Ollama) → ast check → pbcopy

1. Select a region. macOS's built-in screencapture -i gives the drag-a-box picker.
2. Rough draft with Tesseract (--psm 6, preserve_interword_spaces=1). It's fast, but as you saw above, it doesn't understand code.
3. Gemma fixes it. I send both the screenshot and Tesseract's draft to Gemma 4 E4B running locally in Ollama. The prompt says the image is the source of truth: output only the code, restore 4-space indentation, fix OCR confusions ([[ ]], quotes, _, 0/O, 1/l), and don't add, rename or "improve" anything. Thinking is off and temperature is 0, because this is transcription, not creativity.
4. Self-check. The output goes through Python's ast.parse (ignoring notebook lines like !pip and %time). If it doesn't parse, Gemma gets one retry with the exact error message.
5. Clipboard. pbcopy, plus a macOS notification.

If Ollama isn't running, SnapCode still copies Tesseract's draft, so the hotkey never just fails silently.

Why both Tesseract and a vision model? Gemma can read the image alone, but giving it the draft too works like a second pair of eyes. It anchors long identifiers like Preparation_Time_min and leaves Gemma to do what it's good at: understanding that this is Python and putting the structure back.

Why Open Innovation Matters Here

• It works in the lab with no internet. Practical rooms have bad Wi-Fi. SnapCode runs entirely on my laptop. Google Lens needs a connection.
• Lecture material stays on my machine. Course slides aren't mine to upload to someone else's server. With a local model, nothing leaves the laptop.
• It costs nothing per snap. I'll press this hotkey hundreds of times a semester. With a cloud vision API, that's a bill or a rate limit. With Gemma on Ollama it's free forever after one ollama pull.
• I can swap the model. SNAPCODE_MODEL=qwen2.5vl switches models in one line, so I can pick whichever model reads my lecturer's slide theme best instead of accepting whatever a closed API gives me.
• Every piece is open: Tesseract (Apache-2.0), Ollama (MIT) and Gemma (open weights). A friend can run the same setup without signing up for anything.

Prize Categories

Best Use of Gemma. Gemma 4 is the step that turns broken OCR into code you can run. Its vision input lets it check every character against the screenshot, and it runs fully offline through Ollama.