DEV Community: Ikkun

Oracle Cloud free tier: the always-free ARM instance

Ikkun — Fri, 06 Nov 2026 00:15:44 +0000

Yes, Oracle Cloud's always-free tier really does include a permanently free ARM instance — I run a meaningful part of my agent infrastructure on one. The catch people miss is not the price, it is the availability and the mindset. The free-tier ARM instance hosts agent services and a long-running site in my stack, and it has been genuinely useful. But I treat it as a nearly-free compute resource with a recall clause, not as a permanent home for anything I cannot afford to lose.

What the always-free tier gives you

Oracle Cloud's free tier covers a small set of always-free resources, distinct from the temporary trial credit you start with. The part that stands out for people running agents is the ARM instance: it is always-free rather than a time-boxed trial, which means you can leave a service running on it indefinitely without it expiring. For an always-on workload — an agent service that needs to answer around the clock, or a site that is expected to stay up — that long-running nature is the whole point. A time-limited free trial that vanishes after a month is useless for a service; an always-free instance is a different category of thing entirely.

I deliberately keep the platform's monitoring agents on it as well. Some installations strip those out to save resources or reduce noise, but I leave them in place. They are the console's eyes — the signal I rely on to notice when something on the instance stops behaving. On a free resource with no support contract to call, that visibility matters more than the modest overhead it costs.

Where I use it, and where I do not

Generic terms here, because the specifics matter less than the lesson. The ARM instance runs agent services and a long-running site in my infrastructure. It handles sustained, always-on work that does not need the headroom of my paid hardware. That split is deliberate: free-tier compute is excellent for workloads that are steady and predictable, and awkward for anything that suddenly needs a burst of cycles.

What I do not do is treat it as my only copy of anything. If a service on the free tier dies, I want the recovery to be a re-provision and a restore, not a reconstruction. That is the discipline the free-tier stack post lays out, and it applies here more than anywhere.

The caveat that changes the design

The honest caveat: instance availability on the free tier can be reclaimed. Oracle can take an always-free instance back, and when it does, the instance is gone — you do not get a negotiation, you get a notice. I have not been bitten, but I design as if I will be. That constraint is what forces the important habit: keep state somewhere else.

The principle is simple. The instance is a cattle resource, not a pet. Its job is to run the compute. The data it produces and the state its services depend on live elsewhere, backed up and recoverable. That way, if the instance is reclaimed, it is an inconvenience — a re-provision, a restore, a few hours of my time — not a disaster where I lose the accumulated state my agents depend on. My agent memory layer and its vector store live outside the free instance for exactly this reason.

The mental model

The always-free ARM instance is real, capable compute at zero monthly cost, and that makes it easy to over-trust it. The useful frame is: free compute, guaranteed only until it is not. Run the always-on services on it, keep the platform's monitoring eyes attached, and keep everything that matters recoverable somewhere that a reclaim cannot touch. Done that way, it is one of the best cost-per-mileage pieces of my whole stack. Done the other way, it is a single point of failure wrapped in a great price.

What would you be willing to re-provision from scratch in exchange for a permanently free ARM instance?

Originally published at agentjournal.dev.

Alpha Vantage limits and the Japanese stock gap

Ikkun — Wed, 04 Nov 2026 00:15:45 +0000

When I first went to wire up a stock data feed, Alpha Vantage looked like the obvious default: it's free, it's a real API, and it covers a huge swath of assets. So I wrote my pipeline against it — and then discovered two things that changed how I think about any free financial data feed. First, the free tier is rate-limited so tightly (I measured roughly 5 requests per minute) that any loop needs a sleep between calls. Second, and bigger for me: it doesn't cover Japanese stocks at all, which is the market I actually needed. Checked and confirmed in my own setup.

The free tier is a scheduling constraint, not a firehose

The headline "free" hides a real operational constraint. On the free tier I observed roughly 5 requests per minute, which is fine if your job makes a handful of calls. But if you write a loop that iterates over a list of tickers without throttling, you'll blow through the limit in the first few seconds and spend the rest of the run backing off.

My fix was boring and effective: a sleep between requests. I pace the loop to stay under the observed rate, and I treat the limit as part of the contract I'm designing against rather than a surprise. This is the same discipline I apply to any upstream that isn't mine to hammer — throttle, back off, and make the job idempotent so a partial run can resume cleanly.

What Alpha Vantage does cover well

To be fair, the coverage where it exists is broad. The API handles US and other global stocks, forex, crypto, commodities, and economic indicators. If your universe is US equities, a free-tier Alpha Vantage feed with a throttled loop is a perfectly reasonable place to start a research pipeline. The rate limit is manageable when your request volume is small.

That's exactly why the gap stung: the tool is good at the things it does, and the missing piece is precisely the market I needed. For Japanese equity data, Alpha Vantage simply wasn't the answer, and no amount of throttling fixes a coverage hole.

The Japanese stock gap

The hard lesson wasn't about rate limits — it was about coverage. Japanese stocks aren't on the free global feed, and that's a category problem rather than a config problem. You can't sleep your way to data the API doesn't have.

So for Japanese tickers I use a Japan-specific source instead, one that covers daily OHLCV going back years plus valuation and dividend data. Different endpoint, different rate budget, same throttling discipline. The point is that the two feeds serve different asset classes, and choosing between them is a coverage decision first and a pricing decision second.

The decision rule I now run with

Before I build any pipeline on a free financial feed, I check three things in order. Coverage: does it actually serve my market and asset class? Rate limits: can the free tier sustain my request volume once throttled, or will I be waiting all day? Officialness: is there an official source for this data that's more reliable and documented? Alpha Vantage fails me on the first question for Japan, which is enough to disqualify it for that job no matter how good the rest is — the same reasoning I applied when I replaced scraping with proper Japanese data feeds.

The rule generalizes beyond stocks. Any free API with a tight free tier and partial coverage is a candidate for the "check before you build" list. Read the limits, confirm the coverage actually includes your universe, and only then write the pipeline.

What's your go-to when a free API covers most of the world but not the one market you actually trade — do you tolerate the gap or switch sources entirely?

Originally published at agentjournal.dev.

Enterprise Agent Selection: A Field Report (Series)

Ikkun — Mon, 02 Nov 2026 00:15:44 +0000

TL;DR: When an organization handles information that must not leak, audit coverage decides the agent platform — not model quality. This series walks through a real selection process I supported, why a desktop app excluded from Compliance API lost the lead role, and what would change the conclusion.

One-line summary: Audit coverage > model benchmarks for regulated environments; pick the platform whose entire surface set is auditable.

Most articles about choosing an AI platform are written by and for engineering organizations. This one is not. The organization I recently helped has no dedicated engineering staff, handles pre-release information whose exposure carries material consequences, and runs ordinary managed Windows laptops. If that sounds unusual: per 2026 surveys, only ~12% of companies have deployed AI agents at all (under half among large enterprises). The constraint set in this series is not exotic — it is the default starting point for most first-time adopters.

I've anonymized everything the NDA covers. What remains is the decision process, which is the part worth recording.

Why this series exists

The existing guide on this site, Choosing Your Agent, answers "which agent should I run." This series answers a harder question: "which platform can my organization live with for the length of a contract?"

That reframing matters because three things are expensive to change after signing:

Audit coverage — what the vendor records about user activity, and whether you can pull it into your own systems.
Management plane shape — how many consoles, toggles, and config files it takes to keep the deployment consistent.
Cost structure — where the bill is flat (seats) and where it is variable (usage).

Model quality, feature velocity, benchmark deltas — all of that churns monthly. The three items above are structural. You inherit them for the contract term.

The parts

Each part stands alone: direct answer in the first paragraph, tables over prose, facts dated "as of August 2026," and a one-line takeaway you can lift into your own decision memo.

Part	Title	Core question
1	The Audit Gap Decides the Platform	Which surfaces actually land in audit logs?
2	One Admin Console Beats Three	How much management complexity can a staff-less org absorb?
3	What Seat Pricing Actually Covers	Where does the flat rate end and usage billing begin?
4	Why Command Allowlists Fail Alone	What actually constrains an agent on a managed laptop?
5	Research Tasks on a Locked-Down Network	Can agents do web research under default-deny egress?
6	The Decision, and What Would Change It	What was chosen, and which facts would reverse it?

(Links go live as each part publishes.)

Who this is for

Compliance / IT owners at companies adopting AI agents without a dedicated engineering team
Vendors' docs readers who want to know which admin controls matter first
Anyone who has been asked "can we just buy licenses?" by their board

FAQ

Q: Is this a recommendation to avoid Anthropic?
No. It's a record of one selection process where audit coverage outweighed model preference. The reversal conditions are documented in Part 6.

Q: Are the prices in this series official?
All pricing is quoted as of August 2026 from public plan pages; Enterprise tiers are custom quotes, so treat numbers as directional.

Q: Why write about a client engagement?
Because the failure mode — picking a platform nobody can audit or operate later — is common enough that the process deserves documentation.

Next: Part 1 — The Audit Gap Decides the Platform

Originally published at agentjournal.dev.

Running AI Coding Agents in Production: My 2026 Stack

Ikkun — Fri, 30 Oct 2026 00:15:38 +0000

Everyone has an opinion about AI agents. Almost nobody publishes their actual production setup — the gateways, the MCP servers, the credential handling, the monthly bill. This is mine, as of August 2026, with real numbers and the failures included.

I run coding and automation agents daily for real work: this site, open-source tooling, and the automation that runs my infrastructure. Here's the stack, layer by layer.

The agents (division of labor)

I don't use one agent for everything. Using the right tool per task is the cheapest optimization available:

opencode — my open-source coding agent for repository-scale work. Runs with a local provider config, executes code, verifies results. It's the workhorse for feature work where I want full control of the loop.
Claude Code / Codex — used for sessions where their specific strengths apply (long-context refactors, or tasks tuned to their model families). I treat them as specialists, not a default.
Hermes (Nous Research) — the automation layer. Cron jobs, scheduled pipelines, monitoring: things that run while I sleep and message me when they break. This is the agent that writes the drafts for this site, runs my infrastructure checks, and handles repetitive ops.

The rule I've landed on: a coding agent that runs unattended is an ops problem, not a coding problem. Which is why the layers below matter more than which agent you pick.

The gateway layer

Routing through gateways instead of calling providers directly is the single biggest sanity improvement. I use three:

OpenRouter — the default router for experimentation. The free tier covers low-stakes calls, and the model catalog means I can A/B test providers without touching config. OpenRouter - Merge — my main paid gateway. One API for the models I use daily, with cost tracking across agents. At $10/mo it pays for itself in what it stops me from overspending on direct API calls. Merge - Vercel AI Gateway — for the projects already on Vercel's stack. Its free tier ($5 of credits per 30 days) handles the low-volume path fine; I only reach for it when the project is already there. Vercel Why gateways and not direct API keys? Because every agent that holds a direct provider key is a credential you're one prompt injection away from leaking. Gateways centralize the key, give you a kill switch, and make the bill auditable. This matters more than any model choice.

Credential handling (the part nobody writes about)

This is where most agent setups are a disaster waiting to happen. My rules:

Secrets live in pass / Bitwarden, never in agent context. Agents get credentials via an injection layer at runtime, not by reading a .env that sits in the repo.
I maintain trustless — an open-source credential broker that serves secrets to agents through a local proxy, with DLP scrubbing and per-call authorization. It exists because I watched agents echo API keys into logs, chat history, and debug output. Once per week was too often.
Long-lived keys are the enemy. Refresh tokens that rotate, short-lived sessions, and per-service keys scoped to exactly one tool. When a key leaks (it will), the blast radius is one service, one function.

MCP servers and tooling

The MCP layer is where the real leverage is. Currently running:

Filesystem + git servers — for agent access to repos without shelling out into the filesystem blindly
Web search / fetch servers — the ones that make agents useful for research without hallucinating URLs
Home-grown servers for my own infra — including a server that wraps my credential broker so agents can request secrets through a structured interface instead of me pasting them

The lesson: a well-scoped MCP server beats a prompt telling the agent to "be careful" every time. If the agent can't reach it, it can't break it.

Networking and self-hosted pieces

Tailscale — all agent access to my infra goes over the tailnet; SSH is not exposed publicly. Agents reach services by tailnet name, which also means the public attack surface is ~zero. Tailscale - Cloudflare — DNS, Pages (this site), Workers, and the edge layer. The free tier covers everything this site needs. Cloudflare - Qdrant + Mem0 — self-hosted memory/vector layer for the automation agent. Running my own instance means the agent's memory never leaves my server. Qdrant - vLLM — for the local models I run (a small Japanese-optimized model does embedding and light extraction work). Keeping boring inference local cuts gateway spend measurably.

The bill (real numbers)

Monthly, as of this writing:

Layer	Cost
Gateways (Merge Pro + Vercel free tier + OpenRouter free)	~$10
Cloudflare (DNS + Pages + Workers)	$0
Self-hosted infra (cloud free tiers, one cheap VPS)	~$5
Total	~$15/mo

The trick is boring: route the expensive calls through the free tiers, self-host the repetitive work, and let the gateways absorb the variance. I've seen the same setup run $200+/mo when the routing is lazy.

What I'd change / what's broken

Context management is still the bottleneck. Long-running agents degrade; I still restart sessions more than I'd like.
The agents occasionally fight each other — two automation runs touching the same state. File locking and job ownership markers helped, but it's not solved.
Model quality is uneven across providers. The gateway makes switching cheap, but "cheap to switch" means I spend more time evaluating than I'd like.

The agent you pick matters less than the plumbing around it: gateway routing, credential hygiene, scoped MCP access, and a tailnet. Get those right and you can swap models and agents freely — which is exactly what you'll want to do, because this space moves monthly.

This is the first post in what will be a series on the details: MCP server design, gateway cost engineering, and credential handling for agents. If you're running agents in production too, I'd like to hear what your stack looks like.

Disclosure: some links in this post are affiliate links — see disclosure.

Originally published at agentjournal.dev.

vLLM for one person: serving a small model

Ikkun — Wed, 28 Oct 2026 00:15:37 +0000

vLLM is the serving engine I use for the local models in my stack, and the honest summary is this: for one person running a small Japanese-optimized model for embeddings and light inference, vLLM gives you high-throughput, continuously batched serving that you can point at and forget — once you treat the model as a service instead of a script. The hard part is not starting vLLM. It is remembering that a served model now needs monitoring, restarts, and version pinning, just like anything else you run.

A lot of one-person setups never make that leap. They run a model inline in a Jupyter process, or start a server by hand when they need it. That works until it does not. Here is the workflow I landed on.

Why vLLM for a small model

The pitch of vLLM is high-throughput inference with continuous batching. What that means in practice: rather than processing one request at a time or blocking while a batch fills, the engine keeps a queue of pending requests and packs new work into the gaps as each generated token completes. For my workload — repetitive inference, embeddings, and light extraction that runs constantly in the background — that throughput matters more than raw latency on any single request.

It is suited to exactly the kind of jobs I run: many small requests that arrive steadily rather than one large burst. Continuous batching turns that stream of work into near-constant utilization of the GPU.

Picking the right model

I serve a small Japanese-optimized model. The choice of Japanese optimization is not cosmetic: for Japanese text, generic English models degrade noticeably on embeddings and generation alike. Matching the model to the language of your documents is the difference between retrieval that works and retrieval that quietly loses precision.

For a one-person setup the model should be small enough that you are never fighting your hardware. A small model on modest hardware beats a large model that you cannot keep running. The volume of embeddings in my memory stack is high, so throughput per token is the metric that matters, not benchmark scores.

Serving makes it a service

The mental shift is the part nobody warns you about. Before serving, a model is a thing you invoke. After serving, it is a service with an address, and services fail in new ways. Three disciplines made the difference for me:

Monitoring. A silent serving process is a liability. Watch request latency, error rate, and queue depth, because those tell you when the model has degraded before a user notices.
Restarts. The process will eventually die or hang. Plan for restarting it cleanly, the same way you would any other long-running service.
Version pinning. Pin the exact model and engine versions you serve. A silent upgrade can change output behavior and corrupt embeddings without you realizing it.

This is the same discipline I apply to the rest of the self-hosted stack — assume things break and make recovery boring. A served model is production infrastructure, so it gets treated like it.

Where it fits in the stack

The embeddings this model produces feed the memory layer, which is the highest-volume inference in my stack, so keeping it self-hosted and fast is a deliberate cost decision. Running it locally keeps that repetitive work off the paid path entirely.

Serving a model properly is the difference between a working demo and infrastructure you can rely on. The engine is the easy part; the operational habits around it are what make it dependable.

What does your smallest, most repetitive inference job look like — and is it still running as an unmanaged script?

Originally published at agentjournal.dev.

Japan public APIs for agents: e-Stat, EDINET

Ikkun — Mon, 26 Oct 2026 00:15:40 +0000

The problem: Japanese data has no single doorway

The first reflex for a Japanese company is to search the name and read whatever page comes back. That works for one company and collapses on the second: the pieces you need sit in four registers with four access rules, none of them a page you want to automate against. This is the map of which official source answers which question, and where each stops.

The identity layer: corporate numbers

Company names are not unique and get transcribed inconsistently, so a name match is not an identity. The corporate number (法人番号) is the stable key: a 13-digit identifier from the National Tax Agency, reachable through the corporate number API with a free application ID, plus a CSV publication for bulk work. Use it as the join key for everything below. The corporate number API: looking up any Japanese company.

Filings and financial statements: EDINET API v2

For anything financial, the official route is EDINET, the FSA's disclosure system. Filings arrive as XBRL, and API v2 gives you a subscription key, date-range search and a flattened CSV form that most people prefer once they try parsing raw taxonomies. The sharp edges matter: the CSV is UTF-16, tab-delimited and quotes every field, so fix encoding and delimiter up front or the pipeline mangles numbers silently. EDINET API v2: XBRL filings as CSV without the pain.

Statistics: e-Stat

Population, GDP, CPI, employment: the official numbers come from the Statistics Bureau through the e-Stat API with an application ID. The part that trips people up is the two-step flow — ask the metadata endpoint what tables and categories exist, then request the data for the table you picked. Skip the first step and you are guessing at identifiers. Getting Japanese government statistics via the e-Stat API.

Market data: an IRBANK-compatible endpoint

Equities are where the global free feeds fail you: US and European tickers are covered, Japanese ones are missing, and the coverage wall appears only after you have built on them. The IRBANK-compatible endpoint fills that role — daily OHLCV, valuation metrics, dividends, margin balance and segment disclosure through one code per company, with no scraping of rendered pages. IRBANK API: Japanese stock data without scraping.

The glue: where the application IDs live

Application IDs and subscription keys are credentials, so they belong where the calling code cannot read them. A Google Apps Script web app holds a key server-side and exposes a thin endpoint at zero cost, with one trap: use a deployment ID, not the /dev URL, or it only works for you (GAS web apps as API proxies). In my setup those IDs never sit in an agent's environment — they live in a credential store and arrive through a proxy, the pattern behind trustless. For lightweight record stores rather than acquisition, the same CLI access pays off (kintone from the CLI).

The shape they share

Official first. An official feed has a stated contract and a maintainer paid to publish the numbers; scraping is the fallback.
Identity before metrics. Resolve the entity key, then attach filings, statistics and prices. Merging without a stable key is how double counting starts.
Read the format spec, not the sample. UTF-16 and tab delimiters look fine in a preview and break a loader.
Treat rate limits as a schedule — they tell you the cadence the pipeline should run at.

None of these is turnkey, and each is documented better in Japanese than in English — a real cost when you work in English. The alternative is scrapers against four rendered sites, and that bill arrives every time a layout changes.

Which country's official data have you had to wire in for the same reason — and did an official API exist, or did you end up maintaining scrapers?

Originally published at agentjournal.dev.

OpenCode's leak was Console, not your CLI

Ikkun — Sat, 24 Oct 2026 02:00:50 +0000

The OpenCode story that went around on 2026-10-17 was reported as "OpenCode hacked, user credentials leaked", and the first thing most people did was look at the CLI they run every day. That is not where the leak was. The exposed system was OpenCode Console, an optional hosted service where you connect provider accounts so a team can share them. Console stores those provider credentials on its side. The local coding agent stores your keys on your disk and talks to whichever endpoint your config names.

I use the CLI daily and I have never connected a provider to Console. I still spent twenty minutes proving that to myself, because the two products share a name and nobody's first reaction to a breach notice is "let me map the architecture". This is the check I ran, with the numbers, because "I think I'm fine" is not an answer you can act on.

What the vendor actually said

The team's statement, posted in the discussion thread, says unauthorized access to "a legacy system that we were in the process of decommissioning", and that "third party provider keys (eg OpenAI, Anthropic) saved in your workspace may have been exposed". The instruction was to revoke and replace those keys at the provider, update anything using them, and review recent usage.

An OpenAI customer posted the email they received, which describes the same event from the other side: unauthorized access to a system that stored customer API keys, an affected key belonging to their organization, and confirmation that the key had already been disabled. That is the shape of a real incident: not a model doing something strange, but a credential store losing custody of credentials.

The thread hit 975 points with 37 top-level comments, and most of the argument was not about the breach. It was about which product had been breached and why the announcement did not say so in the first line.

Why the name cost everyone a day

The documentation for Console is explicit about the design: connect provider accounts you already pay for, "Console stores the credential and serves each provider through" the gateway, and the models are shared across a workspace. Storing the credential is the feature. Sharing it is the point.

The local agent does something different. It keeps provider credentials in a file on your machine and sends requests to the base URL configured for that provider. Whether those requests reach the vendor's hosted gateway, a local proxy, or a third-party API depends entirely on your config. That single sentence resolves most of the confusion, and it is missing from the announcement.

The four checks I ran

Check	Command shape	Result on my machine
Which endpoint do the stored keys authenticate to	read `baseURL` per provider in the agent config	all three point at `127.0.0.1:8787`, a local gateway, not the hosted service
How many credentials are on disk	list the auth file entries	3 provider entries, file mode 600
Extra copies of the same keys	scan projects, skills, dotfiles for the exact key strings	0 hits across 38,734 files, 0 in shell history
Keys committed to the config repository	`git ls-files` in the config directory	49 tracked files, auth file not among them

The endpoint check is the one that settles it. My three entries are for a gateway, a model lab, and a hosting provider, and every one of them is routed through a process on loopback, which means the key is presented to code I run, not to a hosted workspace that stores it for me. If your config points at the vendor's gateway URL and you went through their connect flow, you are in the other category and you should be revoking keys today.

The copy scan is the check most people skip. A key that leaked into a repository, a .env committed once and deleted later, or a shell history file is a credential with a second life. Scanning 38,734 files takes about a minute and returns a number you can compare later.

What the checks do not prove

Local mode is not "safe". Every one of those three keys is stored as plaintext JSON, readable by anything running as my user. A process I install tomorrow can read them without asking. The 600 file mode stops other users on the machine, not me, and not a compromised package running under my uid.

One detail from the same scan makes that concrete: there are two auth files, and the backup copy holds a different value for one of the three providers. A rotation happened on 2026-10-20, and the retired value is still sitting on disk in a second file. I have not deleted it yet, because I want to confirm with the gateway that the old value is actually dead, and "probably dead" is how these things turn into incidents later.

The config directory is also a public repository. The auth file is not tracked, which is correct, but a public repo with 49 tracked files is one careless git add -A away from publishing something that should not be published. That is a control I should tighten, not a fact I should feel good about.

Five steps if you use any hosted key service

Decide which product you actually used. A CLI is not a hosted workspace, and the difference is in your config, not in your memory.
If you connected a provider to a hosted service, revoke the key at the provider and issue a new one. Disabling it in the service is not enough; the provider's dashboard is the authority.
Read the usage graphs at each provider for the last two weeks. A leaked key usually shows up as a spike before it shows up as an email.
Look for second copies: repositories, backups, shell history, CI secrets, that one .env on the old branch.
For anything an agent uses, prefer a path where the agent never holds the key. A local proxy that injects the credential at the network layer means the blast radius of a compromised agent is requests, not the credential itself.

That last one is the lesson I keep relearning. Sandboxing the agent does not help when the key lives in a service the agent can reach. Removing the key from the agent's reach does.

If you went through this on the 17th: did you find out from the vendor, from your provider, or from the bill?

Originally published at agentjournal.dev.

Cloudflare's audit skill needs a different harness

Ikkun — Fri, 23 Oct 2026 18:26:28 +0000

Cloudflare published the skill that seeded their vulnerability discovery harness, and it is a good skill: six phases, adversarial validation, structured verdicts, and an explicit instruction to report what you could not verify instead of guessing. I ran it on my own repository and it found real bugs.

It did not run out of the box. Not because the skill is wrong, but because it assumes three things about the environment that a normal developer setup does not have: an agent that can spawn isolated parallel subagents, an OS-enforced sandbox for executing target code, and Node available on the parent side to validate the artifacts. I had a laptop-grade agent CLI and no sandbox. Most of the work was building those three things, and that work is reusable.

What the skill actually requires

Straight from the repository, the requirements are:

a coding agent whose model supports tool use and parallel sub-agents
Node.js, for the two zero-dependency validators that check the coverage ledger and the findings file against their schemas
an OS-enforced sandbox that disables external networking, uses a sanitized allowlisted environment, enforces resource limits, and permits writes only to assigned scratch paths

The last one is not optional advice. The skill says that without all of those controls it will not execute target code, and the affected findings stay at needs_validation. That is the correct behaviour: an audit that runs a target's test suite unsandboxed is a supply chain incident with extra steps.

The pipeline itself is six phases: reconnaissance into an architecture map and a coverage ledger, coverage-led hunting with isolated hunters, candidate validation by a fresh verifier, structured output to findings.json, independent verification of the final source claims, and reporting derived from the verified records. Verdicts are confirmed, needs_validation, and rejected, and the skill is explicit that a rejected candidate is a disproved candidate, not a harmless one.

Gap 1: the agent runtime

The skill delegates. A single agent doing all the hunting sequentially would take days, and worse, it would review its own work. So the first question was which CLI could actually spawn isolated subagents with a skill attached and permission restrictions.

Runtime	Result
pi	no task tool, so no subagents. Cannot run this skill
Command Code	has a task tool, but the Go quota throttled runs into stopping partway
opencode (V1)	skill tool, task tool and permission maps all present. Completed all three runs

The detail that made opencode work is the role split. I defined two subagent roles as files: security-hunter and security-verifier. Both are read-only on the code, and the verifier is configured with task: false so it cannot spawn children of its own. That is what makes verification independent: the agent that checks a finding has no way to delegate the checking back to the agent that produced it.

Gap 2: the sandbox

I tried bubblewrap first, because it is the lightest option and it needs no daemon. It failed immediately on Ubuntu with loopback: Failed RTM_NEWADDR. The cause is a kernel setting that restricts unprivileged user namespaces (kernel.apparmor_restrict_unprivileged_userns=1), which turns bwrap into a tool that cannot set up its own network namespace.

Docker worked, with three corrections that cost me a round each:

--tmpfs mounts are noexec by default. The build tools inside the scratch directory then fail with permission errors that look like compilation problems. It needs exec in the mount options.
debian:bookworm-slim does not ship sqlite3. The target's test suite had database tests, and about a third of them failed for a reason unrelated to security. Adding the package fixed the noise.
GOTOOLCHAIN=auto tries to download a toolchain, which cannot work when the container has no network. Pinning GOTOOLCHAIN=local and bind-mounting a GOROOT that matches the module's Go directive was the fix. Dependencies were vendored on the host before the snapshot, so the container never needs to fetch.

The resulting invocation, trimmed to the parts that matter:

docker run --rm --network none --read-only \
  --tmpfs /scratch:rw,exec,size=2g,mode=1777 \
  --mount type=bind,src=$SNAPSHOT,dst=/target,readonly \
  --mount type=bind,src=$TOOLCHAIN,dst=/toolchain,readonly \
  -e GOROOT=/toolchain -e PATH=/toolchain/bin:/usr/local/bin:/usr/bin:/bin \
  -e HOME=/scratch -e GOTOOLCHAIN=local -e GOFLAGS=-mod=vendor -e CGO_ENABLED=0 \
  --user 1000:1000 --cap-drop ALL --security-opt no-new-privileges \
  --memory 2g --cpus 2 --pids-limit 256 \
  local/audit-runner:1 /bin/sh -c 'cp -a /target/. /scratch/t/ && cd /scratch/t && <command>'

The target is mounted read-only and copied into a writable scratch directory per run. The snapshot comes from git archive, so the audit sees one commit and not my working tree. Because .git is absent from the snapshot, the prompt has to state the source ref explicitly, otherwise the agent cannot say what it audited.

Gap 3: who validates the validators

The two .cjs validators are the only objective gate in the pipeline. If they run, a malformed findings.json cannot be reported as a result. If they do not, you get a report that looks structured and means nothing. So the parent runs them, not the agent, and both must exit zero before a run is done.

The same logic applies to the findings themselves. A confirmed verdict from a delegated agent is a claim. For each confirmed item I wrote a small reproduction test as a separate file against the original commit, then ran the same test against the fixed commit. Red before, green after is the only evidence I accept for a security fix.

What it costs in time

Measured on a Go codebase of moderate size:

Profile	Wall clock	Agents
quick, scoped	about 40 minutes	fewer
quick, full	about 52 minutes	19
standard, full	about 2.9 hours	up to 60

The repository notes that a single run finds roughly half of what repeated runs find. My own numbers agree: the second and third runs produced findings the first did not, and one of them reversed a rejection from the first.

Things I would do differently

Do not touch git while a delegated agent is working. I ran a checkout during a run and the agent kept reading a stale HEAD, which produced findings against code that no longer existed. Fetch and reset before or after a run, never during.

Keep the agent's output directory outside the target repository. The skill defaults to a path under your home directory for a reason.

Treat the profile as a coverage decision, not a speed setting. A quick run that returns few findings may have simply not looked.

The findings from the three runs, including the one verdict that reversed between runs, are in a separate post.

If you have tried to run an agent-oriented security skill on your own projects: which requirement stopped you first, the subagents or the sandbox?

Originally published at agentjournal.dev.

EDINET API v2: XBRL filings as CSV without the pain

Ikkun — Fri, 23 Oct 2026 17:27:43 +0000

The problem: Japanese financial filings in a usable shape

If you want annual or quarterly financials for Japanese companies, the filings are real and public, but they sit in XBRL. XBRL is great for regulators and lousy for a quick pipeline: it is verbose, namespaced, and tied to taxonomy dictionaries. My first attempts to pull company financials meant wrestling XBRL documents and their taxonomies into something I could actually compute on. It was painful enough that I kept looking for a shortcut, and the right shortcut is the official one.

The direct answer: EDINET API v2 is Japan's official financial filings API, and it exposes XBRL metadata and documents for essentially all filers going back about ten years, free. For bulk work, the type=5 form delivers the data as CSV so I can skip the full XBRL parse when all I need is the numbers.

Where the CSV simplicity hides

The type=5 CSV form is what makes EDINET usable as a data source rather than a document dump. It flattens financial data into a CSV-like payload that I can load into analysis directly. But "CSV-like" deserves the qualifier, because the format has quirks that will break a naive reader.

Three things bit me. The payload is UTF-16 encoded, not UTF-8. The delimiter is a tab, not a comma. And every field is quoted, like a fully escaped TSV. My normal CSV parser choked on the encoding on the first try, and even after fixing that, the all-fields-quoted layout needs the right handling to split lines cleanly. Once I put the splitlines handling in place, the data comes through reliably, but it is precisely the kind of edge case that makes a canned "just use csv.reader" approach fail silently.

Auth lives in a header, not a query param

A detail that cost me a round-trip: the API authenticates with an Ocp-Apim-Subscription-Key header, not a query parameter. If you are used to appending a key to the URL, you will get a rejection before you ever reach the data. It is a small thing, but it is the kind of contract detail that determines whether an integration is smooth or a morning of debugging. Because the credential goes in a header, it stays out of the URL, which also keeps it out of the obvious places where query strings end up logged. I still keep the key behind my normal key-handling practices rather than embedding it in anything that ships. I describe it as the subscription key; the actual value never leaves my controlled environment.

Bulk ingestion and date-range search

The API supports searching by date range, and that is the lever for doing this at scale. I ingest financial data for hundreds of companies programmatically, which would be miserable one filing at a time. Searching by range and pulling the type=5 CSV form means I can batch the work instead of scripting per-company lookups. Searching by date also maps cleanly onto a recurring job: I know what moved since the last run, and I fetch only the new material.

Why official EDINET wins

The case for EDINET over an unofficial feed is the same one I apply across data sources: when an official API exists with a stated contract and a subscription-key auth model, it is more reliable and it avoids the upkeep of chasing a feed that might change without notice. An unofficial EDINET mirror can work, but it is a source of truth I do not control. My preference is the official API first, unofficial only when there is no other choice.

None of this is to say the official API is turnkey — the UTF-16, tab-delimited, every-field-quoted CSV is a reminder that "official" and "ergonomic" are different things. But it is official, it is free, and it gives me a ten-year window of filings. That combination is hard to beat.

The rest of the Japanese-API cluster

The other pieces I use with this one: Getting Japanese government statistics via the e-Stat API, the corporate number API, IRBANK API, GAS web app proxies and kintone from the CLI. Full map: Japan's public APIs for agents.

For anyone pulling Japanese financials, my advice is: start from EDINET API v2, use the type=5 CSV form, and fix the encoding and delimiter up front. What has your experience been with XBRL taxonomies versus the flattened CSV — is the raw XBRL worth the extra parsing for you?

Originally published at agentjournal.dev.

The Audit Gap Decides the Platform

Ikkun — Fri, 23 Oct 2026 17:13:16 +0000

TL;DR: In August 2026, Claude Cowork's activity is excluded from Compliance API and audit logs even on Anthropic Enterprise. ChatGPT Work — the equivalent non-engineer desktop surface — is covered. If your organization handles information that must not leak, this single fact outweighs every model-quality argument.

One-line summary: A surface you cannot audit cannot be your primary deployment — regardless of which vendor makes it.

This is Part 1 of a series on selecting an agent platform for an organization with no dedicated engineering staff (see the series overview for context). The client details are anonymized; the vendor facts are not, and each one below is dated and checkable.

What "audit coverage" means here

Three distinct things get called "audit," and confusing them causes bad selections:

Audit logs / Compliance API — organization-level records of user activity, pullable into your SIEM via API. This is what a regulator or your own incident review needs.
OpenTelemetry events — developer-oriented telemetry. Useful, but not a substitute: coverage differs, retention is yours to build, and it is not designed as an evidence trail.
Local history — conversations stored on the user's machine. Admins can't query it centrally at all.

When I say "auditable" in this series, I mean #1. The other two have real value — OTel feeds dashboards and alerts, local history serves the individual user — but neither answers the question "show me everything that happened in this workspace last month."

Why staff-less organizations feel this hardest

In an engineering org, audit gaps can be compensated: someone wires up OTel pipelines, builds retention, writes detection rules. It's work, but it's work their people can do.

In an organization without dedicated engineering staff, there is no compensation path. The vendor's audit coverage is your audit coverage. Whatever the admin console shows you is the whole picture, forever. That asymmetry is why audit coverage sits above model quality in this series' evaluation order:

Model benchmarks change monthly; audit scope is structural and survives contract renewals.
You can switch models inside a platform; you cannot self-build compliance coverage for a surface where the vendor records nothing.
When something goes wrong — and at some point something goes wrong everywhere — the question "what exactly happened?" needs an answer from records, not from users' memories.

The facts on the table (as of August 2026)

Surface	Anthropic	OpenAI
Chat (web)	Compliance API covered	Compliance API covered
Non-engineer desktop app	Claude Cowork: NOT covered (OTel only; local history is device-local)	ChatGPT Work desktop: covered (workspace policies apply)
Coding CLI	Claude Code: managed settings + OTel	Codex: requirements.toml + Compliance API (`CODEX_LOG`, `CODEX_SECURITY_LOG`, ~30-day retention)
Cloud agents	Cowork cloud (beta): not covered	Codex cloud: covered

Three observations:

Cowork's exclusion survives the top plan tier. Anthropic Enterprise adds SCIM, custom roles, and Analytics API — and still documents Cowork activity as out of scope for Compliance API. This is a structural gap, not a pricing gap.
OpenAI keeps both audiences inside one auditable container. Non-engineers (Work) and developers (Codex) both land in Compliance API, with Codex events arriving as typed event streams (CODEX_LOG, security events separately).
Cowork does have OpenTelemetry, and it's the only telemetry route even on cheaper plans. It's real, and worth wiring into a collector if you allow Cowork at all — but treat it as monitoring, not evidence.

Retention changes the math

Coverage is only half of it; retention is the other half. An audit log that expires in 30 days supports incident triage but not annual review or a formal inquiry. When comparing platforms, ask three questions as one bundle: What is recorded? Per surface? For how long, and in what export format? A platform that answers all three consistently is worth more than one with deeper features and vaguer logging.

Verify it yourself before signing

None of the facts above require trusting this article. The verification takes under an hour with trial or demo tenants:

Open the admin console for each plan tier you're considering. List every surface (web chat, desktop app, CLI, cloud agents) visible to a regular user.
For each surface, generate activity — one conversation, one agent task — then search for it in the audit log / Compliance API export.
Note what's missing. Anything absent from step 2's results is a surface you cannot audit. Write the list down; this becomes a policy input, not a footnote.
Pull one real export via the API and check field completeness (actor, timestamp, action, object). Screenshots in a console UI don't count until the export works.
Check retention terms in writing, in the contract paperwork — not just marketing pages.

Step 3's list is the deliverable that matters. In the selection process this series documents, that list contained Cowork — and that single line item reordered the whole evaluation.

What to do with excluded surfaces

Excluding a surface entirely isn't always possible; users arrive with preferences. The mitigation ladder, weakest to strongest:

Exclude from regulated work by policy — written rule, enforced by review, technically unenforced. Fragile, but honest about its weakness.
Read-only limitation — allowed for summarizing public material, banned for anything sensitive. Requires information-classification discipline (see Part 6 for why that's hard without staff).
Telemetry-first conditional allowance — OTel wired into your collector before first use, alerting on volume anomalies, plus periodic manual review. This is the strongest available posture for Cowork today, and it still isn't evidence-grade.

Whatever rung you choose, the decision should be recorded with a date and a re-check trigger. Vendor terms move; a policy frozen in August 2026 shouldn't silently govern August 2027.

Caveats

Vendor terms change without notice; re-check the admin docs before signing anything.
This article reflects documentation review and sandbox-tier verification, not a full Enterprise deployment. Pilot results follow in later parts.
Nothing here claims Claude models are weaker, nor that Cowork is unsafe. It is a record that its audit story did not match this particular constraint set. If Anthropic brings Cowork into Compliance API coverage, Part 6 lists exactly how the conclusion gets re-evaluated.

Coming August 27: Part 2 — One Admin Console Beats Three

Takeaway: Check Compliance API coverage per surface, per plan tier, before comparing anything else. One excluded surface disqualifies it from hosting regulated work — and exclusions don't always disappear at higher tiers.

Originally published at agentjournal.dev.

Vercel AI Gateway free tier: $5 of credits, real limits

Ikkun — Fri, 23 Oct 2026 17:05:12 +0000

"Free tier" on a gateway usually means one of two things: a meter you discover after the bill, or a teaser that evaporates the moment you enter a card. The Vercel AI Gateway free tier is neither — it is $5 of credits per 30 days, real and observed, with no card required. The catch is that the free credits stop once a card is registered. I use it for the projects already on Vercel's stack, and it works fine as long as I respect that one rule and keep the gateway off the default path.

What the $5 credit tier actually is

Every 30 days, the account gets $5 in gateway credits. For a low-volume path — a dashboard endpoint, a small agent script attached to a Vercel deployment — that covers the month without drama. The key word is low-volume. I treat the credit balance as the spec: if a project's gateway traffic fits inside $5 of credits a month, it can live on the free tier. If it cannot, it belongs on the paid gateway, not here. There is no billing surprise, because there is no card attached.

The card-on-file trap

The behavior I observed: the free tier ends once a card is registered. Not "the credits keep coming and the card catches the overflow" — the free credits simply stop. That is the single most important detail, because every signup flow nudges you toward adding a card for overage protection. On this gateway, that safety net cancels the free tier. I keep the free-tier projects card-free, and the moment a project needs overage protection, it graduates to the paid gateway instead of adding the card.

Where the key lives

The gateway key goes straight into my password manager — never into the repository, never into agent context. Agents that need it fetch it from the credential broker at runtime, so a checked-in file or a pasted env var is not how the key travels. That is the same discipline I described in my post on credential management for AI agents, and it matters more with gateways than with direct providers, because one gateway key opens every model behind it.

What happens on a 429

Rate limits are the routine failure mode, and the handling is configured, not improvised. When the gateway returns HTTP 429, the fallback chain routes the call to a different model provider and the work continues. The gateway is not my default gateway; it is used only for projects already on the Vercel stack, and only for the low-volume path within those projects. A 429 on that path is an inconvenience handled in the chain, not a stop-the-world event. That is why I keep the free tier on the chain instead of in it as the only option.

Why it is not my default gateway

Workflow preference: the projects on Vercel's stack get the gateway because it sits next to the deployment, not because it is the best router I have. Everything else routes through my main gateway, with its own cost tracking and fallback chain. Keeping the Vercel gateway scoped to its own projects means the $5 credit tier never has to carry more than it is sized for, and the card-on-file rule never gets tested by accident.

The $5 credit tier is honest about what it is — a small monthly allowance with a sharp edge. Respect the edge, and it quietly covers a whole class of low-volume work. Have you checked whether your gateway's "free tier" actually survives adding a card?

Disclosure: some links in this post are affiliate links — see disclosure.

Originally published at agentjournal.dev.

Jev vs a 310M encoder I trained myself: 750 rows, three tasks, two different winners

Ikkun — Mon, 21 Sep 2026 02:20:55 +0000

TL;DR: I ran three Japanese classification tasks (250 rows each, same gold labels) through six systems. A 310M encoder I fine-tuned on 250 labels beat TypeSafe's Jev on topic classification by +12.0 points (McNemar p=0.00007) and was 4–20× faster — but on the two polarity tasks it only tied Jev. The zero-shot open-source models everyone is writing about lost everywhere: a GLiNER-family classifier scored 62.8 / 87.2 / 44.8 against Jev's 76.8 / 94.4 / 74.0. The same architecture family, trained on my labels, scored 88.8 / 92.8 / 75.2.

One-line summary: A decision API wins when you have no labels; a small trained encoder wins when you have a few hundred — and which one wins on a given task is decided by the shape of the task, not by which model is trending.

The question behind the hype

Jev (TypeSafe AI's System One) is genuinely good, and it is cheap: $0.042 per 1M input tokens with output free, roughly 32k of context, a confidence number in the response, and no training data required. If you read the last two weeks of posts about it, the conclusion is "use Jev."

I use Jev. But I had a pile of results from my own tasks that did not fit that conclusion, and one of them was embarrassing enough that I re-ran it twice:

A char n-gram + logistic regression baseline beat Jev on long-document topic classification by 11.6 points (0.884 vs 0.768).
The same baseline lost to Jev by 20.8 points on short-text polarity (0.736 vs 0.944).

A 5-minute linear model beating a decision API on one task and losing badly on the next is not a story about either model. It is a story about task shape. So I designed the experiment I should have run first: hold the answers fixed, hold the rows fixed, and separate the two axes that actually differ — labels or no labels, and topic or meaning.

What I measured, and how

Three Japanese datasets, 250 rows each, frozen:

Task	Shape	Classes	Chance
livedoor news corpus	long-document topic (mean 1,174 chars)	9	11.1%
rakuten reviews	short-text polarity (mean 138 chars)	2	50.0%
chabsa	financial-sentence polarity (mean 92 chars)	3	33.3%

Six systems, two conditions:

Zero-shot (0 labels): Jev, a local 26B MoE (Gemma 4 26B-A4B, Q4_K_M, resident on the same mini-PC), SemIf (Qwen3.5-4B), GLiClass multilang-mini (a GLiNER-family classifier), Laya (typed decisions).
Trained (250 labels): sbintuitions/modernbert-ja-310m with a classification head, and a char n-gram + logistic regression baseline.

The trained numbers are 5-fold cross-validated out-of-fold predictions: train on 200, predict the held-out 50, five times, so all 250 reported predictions come from a model that never saw that row. That is the same protocol I use for the linear baseline, which means the trained row and the baseline row are directly comparable.

Everything is compared on identical row ids with McNemar's test. This matters: at n=250 the independent-confidence interval is about ±6 points, so "0.888 vs 0.944" cannot be called a difference without a paired test.

Results

System	Condition	livedoor (9)	rakuten (2)	chabsa (3)	p50 latency
ModernBERT-ja-310m	trained, 250 labels	88.8%	92.8%	75.2%	0.10–0.45s
Jev 1.13.0	zero-shot	76.8%	94.4%	74.0%	1.9–2.4s
Gemma 4 26B-A4B (local)	zero-shot	65.2%	94.4%	75.2%	1.24s
SemIf (Qwen3.5-4B)	zero-shot	68.8%	92.0%	64.0%	1.3–6.8s
GLiClass multilang-mini	zero-shot	62.8%	87.2%	44.8%	0.04–0.42s
Laya (typed decisions)	zero-shot	29.2%	60.0%	40.0%	—
char n-gram + LogReg	trained, 250 labels	88.4%	73.6%	64.0%	~0s

McNemar, same ids:

Comparison	livedoor	rakuten	chabsa
trained encoder − Jev	+12.0pt (p=0.00007)	−1.6pt (p=0.50)	+1.2pt (p=0.83)
trained encoder − local 26B MoE	+23.6pt (p<1e-15)	−1.6pt (p=0.45)	±0.0pt (p=1.00)
trained encoder − SemIf	+20.0pt (p<1e-15)	+0.8pt (p=0.83)	+11.2pt (p=0.007)
trained encoder − GLiClass (zero-shot)	+26.0pt (p<1e-15)	+5.6pt (p=0.016)	+30.4pt (p<1e-15)
trained encoder − char n-gram LogReg	+0.4pt	+19.2pt	+11.2pt

Finding 1: the same family wins or loses depending on whether you trained it

This is the part I did not expect, and it is the most useful thing in the table.

GLiClass is a GLiNER-family classifier — same lineage as the models currently being pitched as open-source Jev alternatives. Zero-shot, it scored 62.8 / 87.2 / 44.8: it lost to Jev on all three tasks, and on the 3-class polarity task it was barely above chance (44.8% against 33.3%).

Trained on 250 of my labels, the same generation of model scored 88.8 / 92.8 / 75.2. The gap between zero-shot and trained is +26.0 / +5.6 / +30.4 points, all significant.

So "is there an open-source model that replaces Jev?" was the wrong question. The models are not the variable — the labels are. A 310M encoder with 250 labels beats a decision API on one task and ties it on two; the same architecture with zero labels loses to everything.

I also paid for this lesson twice. Before this experiment I ran a proper evaluation of the two models most often named as Jev replacements — SemIf and Laya — on these same rows. Both failed to replace anything. Laya scored 29.2% on a 9-class task (chance 11.1%) with a median confidence of 0.029, i.e. it was visibly not deciding. Its zero-shot English number in its own documentation is 0.362, so this is not a Japanese-specific defect — it is a zero-shot defect. SemIf was closer but lost on all three tasks, two of them significantly.

Independent benchmarks showed the same thing. On a frozen-question benchmark, Laya's "hard" score was 34.1% while its model card implied a win over Jev, because the model had been fine-tuned on that benchmark's training split. Self-reported comparisons of fine-tuned models are not evidence about zero-shot behaviour.

Finding 2: training pays on topic tasks and does not on meaning tasks

Look at where the 12-point win came from and where it did not.

livedoor (9-class topic, 1,174 chars): trained 88.8% vs Jev 76.8%. The labels are in the text — if you have a few hundred examples, "what topic is this" is a vocabulary problem, and a trained encoder learns the vocabulary. It also beat the char n-gram baseline by a hair (+0.4pt), which tells you the topic signal itself was already largely lexical.
rakuten (2-class polarity, 138 chars): trained 92.8% vs Jev 94.4% — a statistical tie (p=0.50). Same for the local 26B MoE (p=0.45). Adding data does not fix this, because the problem is not "I don't know the vocabulary", it is "I need to judge sentiment".
chabsa (3-class financial polarity, 92 chars): trained 75.2%, Jev 74.0%, local MoE 75.2%. A three-way tie. On this task no one is good — 74–75% is the ceiling for all three approaches, and that is the real finding. If you need better here, you need a third stage (a stronger model on the low-confidence tail), not a different single model.

The practical rule I now use: if the label is visible in the words, train something small; if the label is a judgement about the words, use a decision API; if both plateau around 75%, stop shopping for models and fix the data or add an escalation stage.

Finding 3: the trained encoder is dramatically cheaper to run

Latency, single-threaded, same machine:

	p50 per item
trained encoder (310M, CPU)	0.10–0.45s
Jev	1.9–2.4s
local 26B MoE	1.24s
SemIf (4B)	1.3–6.8s

That is 4–20× faster than the API, with zero marginal cost, no network dependency, and no data leaving the machine. The confidence score is a softmax you can calibrate yourself rather than a vendor's number.

This is the part that changes architecture. Once the local model is both faster and not worse, the API becomes the fallback rather than the primary path: trained encoder first, Jev only on the low-confidence tail, the novel-looking rows, and as a drift detector.

Because you will ask: can I train it on the GPU in the box?

I tried that too, on the same mini-PC (Ryzen 9 7940HS, Radeon 780M iGPU, gfx1103). Short answer: it works, and it is not worth switching to.

Getting PyTorch onto that iGPU is now possible through AMD's device-specific wheels, but there are three traps in a row:

Installing with the AMD index as the only index makes pip fail to resolve ordinary dependencies, because you have just replaced PyPI.
Adding PyPI back as an extra index lets PyPI's newer torch win the version comparison — you silently get a CUDA build and a pile of nvidia-* packages.
The nightly index's newest torch requires a rocm-bootstrap version that the same index does not ship.

Once pinned properly, the iGPU is real: torch 2.8.0+rocm7.14.0a20260519, arch=gfx1103, fp16 matmul fine. The microbenchmark is impressive — 1.21 s/step against 16.41 s/step on CPU, 13.6×.

The real workload does not deliver that:

Task (5-fold CV, 310M encoder)	CPU fp32	iGPU fp16
long documents (512 tokens)	112 min	44.5 min (2.5×)
short sentences	21 min	23 min (0.9×)

Two reasons the microbenchmark lies. Short sequences leave the GPU idle while the CPU is comparatively efficient at small matrices, and per-fold fixed costs (model load, tokenisation, one-at-a-time evaluation) can exceed the training itself. Only long-document training showed a real gain.

And it is not stable by default. Roughly every few minutes the driver resets the GPU — amdgpu: ring gfx_0.0.0 timeout → GPU reset(8) → then the process dies with HIP error: unspecified launch failure. The GPU edge temperature was 49°C, so this is a driver problem, not a thermal one. Serialising kernel dispatches and pinning the performance level plus running one fold per process with retries did get a full 5-fold run through cleanly, but fp16 also cost 2.8–4.4 points of accuracy (not significant at n=250, but negative on both tasks).

So: CPU for training, GPU for what already works — Vulkan inference. If you are training on your own private data, a documented 30 minutes per dataset on CPU is better than a 2.5× speedup that crashes.

Where the labels come from is the whole game

If trained beats zero-shot this decisively, the obvious move is to generate labels with Jev and train on those. That works, with one trap I want to be explicit about, because it is the reason I did not just ship the first version:

If Jev alone produced the labels, Jev's accuracy is the ceiling of the student. You are teaching a model to reproduce another model's mistakes. The fix is to spend the labels where they matter: run Jev over everything, then take the low-confidence rows and have a stronger model or a human relabel them. That is where the ceiling comes off, and it is also where the marginal cost is, because high-confidence rows can be accepted as-is.

At 250 labels the trained encoder is already at parity with Jev on the tasks Jev is strong at, and ahead where Jev is weak. Labels are the lever, and the second and third hundred labels are worth more than the first.

What I run now

Situation	Use
No labels, short text, judge-shaped task	Jev (74–94% zero-shot, no setup)
No labels, must stay offline or in-machine	local 26B MoE (ties Jev on both polarity tasks)
A few hundred labels, topic-shaped task	train a small encoder (beats Jev by 12pt, 4–20× faster)
A few hundred labels, meaning task	trained encoder, but accept parity with Jev
Everything plateaus at ~75%	stop model-shopping; escalate the low-confidence tail
Labels to be generated	Jev for volume, a stronger model or a human on the low-confidence rows

Caveats

The trained numbers are 5-fold CV over 250 rows, so 200 rows of training data per fold. That is a floor, not a ceiling — more labels should widen the topic-task margin.
livedoor was truncated at 512 tokens, so the longest documents are not fully represented.
The trained side reports raw argmax; I have not temperature-scaled it, so its confidence is not yet comparable to Jev's for thresholding.
The char n-gram baseline is a single configuration. It may not be the strongest non-neural baseline available.
The iGPU accuracy comparison is at n=250 per task, which is not enough to make a 3-point effect significant. "Not significant" is not "no effect."

Takeaway: Zero-shot and trained are different products, not better and worse versions of the same one. If you have no labels, a decision API is the only thing that works today. If you have a few hundred labels and the label is visible in the text, a 310M encoder will beat it, cost nothing to run, and answer in a tenth of a second. Measure your task's shape before you pick a winner — and be suspicious of any "open-source Jev replacement" benchmark where the model was trained on the benchmark.

The API used here: TypeSafe AI's Jev (System One, jev-latest) — text plus a list of questions in, a 0–1 score per question out, own confidence included, roughly 32k of context, $0.042 per 1M input tokens with output free. Both controlled and randomised question sets were frozen and committed before the runs, and every comparison above is on identical row ids.

If you have run the same shape of experiment on your own data, I want to know where a trained small model beat the API and where it did not — especially on tasks where both plateaued.