Three models aced my Korean prize benchmark. That is the lesson.

ohyoseok92 — Sat, 10 Oct 2026 07:56:07 +0000

This is a submission for the Kaggle Benchmarking Challenge.

What I Benchmarked

I wanted to test a narrow mistake in contest discovery: reading a Korean prize notice and reporting the headline prize pool or a gift voucher as the first-place cash award. A contest listing that says “total prizes: 10 million won” may pay only 3 million won to the winner.

I wrote ten original, synthetic Korean notices with hand-checked answers. The task asks each model to return only the KRW cash amount for 대상 or 1등, for the whole winning team. It returns 0 when that award is non-cash or undisclosed. Cases cover 만 and 억 notation, commas, distractor awards, a gift voucher, an unknown amount, and a team payout. Exact integer matches earn one point; the reported score is the fraction correct. No personal data or scraped contest notices are in the test.

Models Tested

I ran the same Kaggle task against Gemini 3.7 Flash, Gemma 4 26B A4B, and GPT-5.4 nano. I wanted a mix of a widely used hosted model, an open model, and a small hosted model. Each saw the same ten cases and scoring code.

Findings

Model	Exact matches	Score
Gemini 3.7 Flash	10/10	1.00
Gemma 4 26B A4B	10/10	1.00
GPT-5.4 nano	10/10	1.00

The result surprised me less as a ranking than as a warning about the test: three perfect scores do not distinguish these models. The cases confirm that all three follow this explicit instruction on clean, short notices. They do not show how any model handles real contest pages.

The zero-answer cases are useful checks: “50만원 상당의 여행 상품권” must not become 500,000 KRW of cash, and an undisclosed first-place amount must not be inferred from the total pool. All three models got those cases right. The team example also asks for the whole team's award rather than a per-person division; all three returned 1,000,000 KRW.

What would I measure next? Longer notices with prize tables and footnotes, OCR errors, mixed currencies, ranges and conditional awards, plus paraphrases of the same source so the answer is not tied to one wording. I would add reviewed real notices only with clear reuse rights. I would also report error types alongside accuracy: wrong award level, unit conversion, non-cash confusion, and unsupported inference.

This first version is a small, transparent probe. Its strongest finding is the ceiling effect, and it gives a reproducible baseline for a harder follow-up. Codex helped build and run the notebook; the cases, code, and evaluations are open for review.

My Benchmark

Korean Contest Prize Reading leaderboard on Kaggle. The task and linked notebook show every prompt, expected integer, scoring rule, and run.

Pocket Field Notes: a local AI card that gets you outside

ohyoseok92 — Sat, 10 Oct 2026 07:55:48 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

What I Built

Pocket Field Notes turns four small choices—available time, setting, pace, and what you want to notice—into a printable three-step outdoor observation card. The point is to make the screen the shortest part of the experience. Read or print the card, choose a safe route yourself, put the device away, and look around.

I built it for people who want a low-pressure reason to step outside without tracking their location or following an app's route. The 10-, 20-, and 30-minute options work for a short walk or for sitting and observing from a doorstep.

Demo

Watch the 20-second demo. The field card in the video came from a real local run and its JSON response is included. The video uses rendered slides to show the flow; it is not a screen recording.

That run used 20 minutes · park · gentle walk · color. The card allotted 5 minutes to walk on a chosen route, 10 minutes to pause and observe, then 5 minutes to continue or return. Its reflection asks what colors and sounds stood out.

Code

Source, setup instructions, and MIT license. Install Ollama and the qwen2.5-coder:7b-instruct-q4_K_M model, run python server.py, then open http://127.0.0.1:8765. Python 3.11+ is the only app runtime dependency.

How I Built It

The browser sends four choices to a tiny Python standard-library server. The server calls Ollama on localhost, asking an open-weight Qwen2.5-Coder model for JSON with a title, three observation questions, and a reflection. The app checks the response shape. Deterministic code sets the step timing and movement, and replaces questions that assume a landmark the app has never verified. The browser renders and prints the card.

I tested the live app with the installed local model and adjusted the guardrails after an early output assumed a park entrance and pond. The final example avoids those claims. Codex assisted with the new code and this write-up; I reviewed and tested the running result.

Why Does Open Innovation Matter?

A local open-weight model can tailor an observation card without sending a user's location or preferences to a hosted inference API. The model is replaceable via OLLAMA_MODEL; the timing and safety rules remain visible in the repository. Once the model is installed, inference can run without an internet connection. There is no account, API key, usage charge, GPS, or tracking.

The model cannot know current conditions, opening hours, or whether a path is safe. The user chooses the place and route. That limitation shaped the product: it suggests what to notice, then gets out of the way.

DEV Community: ohyoseok92