DEV Community: Lawrence Liu

Searching for Orbi's harness, part 1: what our reviewer missed

Lawrence Liu — Mon, 05 Oct 2026 15:19:59 +0000

Orbi takes a GitHub issue and hands back a reviewed, merged pull request. One agent writes the fix, another reviews it, and the loop runs until the review passes. Around the models sits the harness: the prompts each role gets, the skills loaded next to them, the tools on the runner's PATH, and the rules for when a review may pass.

When a delivery goes wrong, one run can't tell you whether the model or the harness is to blame. So we stopped guessing and spent a night and the following morning running Orbi's full loop over and over: first on three open-source issues where Orbi had shipped a regression, then on nine ordinary bugs it had never seen, then on three whose first real-world fix had broken something. We swapped prompts, skills, tools, implementation models and review models, and graded every merged result with a hidden grader Orbi never saw.

This is part 1: how we found the failures and built a harness that fixes them. Part 2 asks whether that harness is worth what it costs. The results and scripts are on the benchmark page and in orbi-build/orbi-bench. Here's the short version.

Ten takeaways

On the issues we tuned on, the harness moved the numbers more than any model swap we tried. With deepseek-flash in both roles, the original harness passed 6 of 15 runs on the three tuning issues. v13 to v15 passed 28 of 30. That's a training score. Takeaway 7 covers issues we didn't tune on.
"Compare before and after" is not enough. Ask an agent to check for regressions and it checks examples it already believes in. One run tested U+0301, which was never broken, and skipped U+3099, which was. Inputs have to be generated by a program, across whole classes of values.
An oracle written from the fix proves nothing. A gpt-5.6-sol run generated 1,200 tables, compared them with a "reference" that implemented the rule it had just written, reported head wrong: 0, and shipped the regression.
Reference libraries are wrong too. wcwidth 0.2.14 counts 👍🏽 as four columns. Trust it and you'll prove a regression correct. Cross-check two references and say which one you believe.
The characters you strip are data. The password leak needed a secret that itself ended in \n. Agents always put the backslash after the secret, never inside it. Naming that input class fixed the leak.
Your own harness text can be the bug. An example we wrote ("don't edit the generated changelog") was wrong for one project and steered Orbi wrong. Rules about a repository should come from the repository's own tools.
On ordinary bugs, the stronger harness costs about twice as much and buys nothing we could measure. On nine unseen issues, the original harness, v15 and v15 with a sol reviewer all passed 9 of 9. v15 used a median 2.0× the tokens and 1.45× the wall-clock time.
Agents don't repeat the regressions humans made. On three bugs whose first upstream fix had broken something (numpy, the OpenTelemetry Collector, aiohttp), the original harness passed 4 of 4 scored runs and v15 3 of 3 (it never finished a numpy run), graded on both the bug and the regression. The regression to guard against is the agent's own habit, rather than a general list of ways code breaks.
Review models matter where judgement matters. Of the two chainloop failures in v13–v14, one was a deepseek-flash review misjudging whether \n was data or an escape; the other was the collision that v15 then fixed. A sol reviewer passed 4 of 4 there, as did deepseek-flash on v15, so treat this as a lead.
A benchmark is software, and ours had bugs. Three of our graders were wrong at some point, one snapshot silently dropped tracked files, and a reviewer once looked up the real upstream fix. Every one of these would have produced a confident, wrong number.

The rest of this post shows where each one came from, mistakes included.

Why we started

On September 28 Orbi fixed three issues in forks of open-source projects: pyinfra, chainloop and fedify. All three passed their test suites and Orbi's own review. We checked them by hand before sending anything upstream. Two were wrong.

The chainloop fix was in a secret redactor. The issue said a JWT in front of an escaped quote got redacted together with the backslash. Orbi's fix trimmed trailing backslashes from each finding, which fixed the JWT and broke two other things:

input:  DB_PASSWORD=Zq8mLp2Vx9rT\n next
output: DB_PASSWORD=[REDACTED:generic-password]\n next

input:  password: s
        Host: services.example.com
output: pa[REDACTED][REDACTED]word: [REDACTED]
        Ho[REDACTED]t: [REDACTED]ervice[REDACTED].example.com

(Placeholders shortened in the second example.) In the first, the detector reports the password as Zq8mLp2Vx9rT\n, ending in a literal backslash and n. The fix trims those two characters off the finding, so they stay in the log in clear text. In the second, a one-character secret made the redactor replace every s in the text.

The pyinfra fix was about CJK text misaligning a results table. The fix counted wide characters as two columns. That aligned the issue's example and made 👍🏽 and a decomposed が worse than before.

On both, Orbi's review, posted on the fork's issue, reads in full review: pass, no findings. The review ran the tests and checked the issue's example, which passed. The regressions sat in inputs nobody tried.

We sent one of the three upstream. The pyinfra fix was reworked by hand with Claude Code and opened as pyinfra-dev/pyinfra#1977, with the AI involvement disclosed. chainloop's waits for a fix we trust. fedify's code was fine, but fedify asks contributors to be assigned an issue before opening a PR, and the maintainer took this one and closed it.

How we measured

The grader is calibrated before any variant runs. A grader that passes the bad fork PR measures nothing.

For each issue we pushed a private snapshot of the repository at the commit Orbi had started from, opened the same issue there with the ai-ready label, and pointed a real Orbi runner at it. The runner did what it does in production: claim, implement, open a PR, run the independent review, fix, merge. Then a hidden grader scored the merged code, together with the repository's own lint and type checks and its full test suite.

Before any variant ran, each grader had to fail the original code, fail Orbi's merged fork PR, and pass a fix we had verified by hand. For chainloop, the two checks that fail on the fork PR are exactly the two outputs above. For pyinfra they are 👍🏽 and が.

A variant is one combination of harness and models. We changed the prompt, the skills, the tools on PATH, the implementation model, the review model, or a mix. The models were deepseek-flash, gpt-5.6-luna and gpt-5.6-sol through Pi, Opus 5.5 through a Claude Code bridge, and GLM 5.3 flash through a zcode bridge. On the two bridges the same model both implements and reviews, and the agent loop is the bridge's own.

Variants ran three or four at a time on one machine. Each had its own private repository, so no run could see another's branches.

The grid

Rows are harness versions, columns are implementer / reviewer. ds = deepseek-flash. Cells are passes / runs across the three repositories.

The grid is sparse on purpose. At every step we picked the cheapest run that could answer the question in front of us; we never set out to fill a full factorial. Most non-deepseek cells have two to four runs. Treat the other columns as leads and the ds/ds column as the result.

What the harness changes did

Same model throughout. Pale bars have fewer than five runs.

pyinfra and fedify moved the most. chainloop already passed most runs on the original harness. Its regression shows up rarely, and that's what makes it dangerous.

Every rule below came from reading the transcript of a failed run. Each version builds on the one before.

v1: ask for a before/after comparison

The first attempt was the obvious one: tell both roles to compare behaviour before and after the fix and write down what they checked. pyinfra went from 0/5 to 2/3.

The failures are worth a close look. The agent did compare, but on examples it picked itself. In one run it tested combining marks with U+0301 (a Latin acute accent, which the old code already handled) and never tried U+3099 (the Japanese voiced sound mark, which the fix broke). Ask an agent to look for regressions and you get the ones it already expects.

v4: the same rule as a separate skill

We moved the v1 rule into a standalone skill instead of the prompt. It scored 1/3. The transcripts show the skill was read. Three runs are too few to say the packaging hurt, but they do show it didn't help: the rule still let the agent pick its own samples.

v5: generate the inputs

v5 required inputs to be generated by a program across whole classes (for text: ASCII, CJK and fullwidth, combining marks, emoji modifiers, zero-width sequences, ANSI codes; for syntax: escapes, even and odd backslash runs, quotes, delimiters), with the counts written into .orbi/regression.md so the reviewer could check them. From then on, runs generated a thousand inputs or more. A v5 run generated 2,756 inputs for the redactor; the largest corpus we saw later, from an Opus run, was 1.19 million character units for the table. v5 passed 4 of 4.

v9 and v11: the oracle must not be the change

Then a strong model showed us what generated inputs can't fix on their own. This is from a gpt-5.6-sol run's own regression report on pyinfra:

The comparison uses an independent reference formatter implementing that rule
...
- inputs: 1,200
- head wrong: 0
- base wrong: 1,111
- head wrong but base right (regressions): 0

"That rule" was the rule the fix had just implemented: count East Asian wide characters as two columns. The "independent" reference was the fix written a second time, so of course it agreed. The 👍🏽 regression was still there.

v11 requires an oracle that doesn't depend on the patch: an existing library, the old behaviour where it was right, or an invariant. It also asks the agent to list the dependency tree first, because the reference is often already installed.

v11: tools on `PATH`, and a bug in our own text

fedify failed because of us. Our prompt included an example: "add a changelog fragment, don't edit the generated changelog". That's good advice for many projects, but fedify uses sacho with materialization turned on, so every upstream commit edits both the fragment and CHANGES.md. Runs that followed our example failed sacho check.

The fix was to stop writing project rules into the harness. v11 says: find the repository's own contribution tooling, run its check command, and treat that as the rule. We also put sacho and deno on the runner's PATH, because an agent guesses at any rule it can't run. fedify went from 1/4 to 3/3. The same change showed up in whether a fix could be sent upstream as is (conventional commit titles, sign-off, AI disclosure, no fork-local #1 references): 9 of 15 original-harness runs passed that check, and 39 of 39 from v11 to v15.

v13: the characters you strip are data

The chainloop regression still got through v11 in some runs. Their generated corpora were large and varied, yet every one put the backslash after the secret: secret\, secret\\, secret\n. None had a secret that itself ended in a backslash and n, and that's the input that leaks.

v13 names the class directly: when a change strips or trims characters, test values that themselves contain or end with those characters, and ask which producers (here, detection rules) can emit such a value. chainloop went to 4/5.

v14: cross-check reference implementations

Once agents used real references, a new failure appeared. In the pyinfra environment, wcwidth 0.2.14 counts 👍🏽 as four columns instead of two. An agent that trusted it would conclude the regression was correct and the old behaviour wrong. rich, installed in the same environment, gets it right.

v14 tells the agent to compare references when more than one exists and to say which one it trusts and why (usually the one with the newer Unicode data). 8/9.

v15: collision checks

The one-character s bug worked differently. A fix changed the needle it searched for (a trimmed secret) and then replaced every occurrence of the shorter needle. v15 adds: if a change alters text that is searched for or replaced, test where else that text appears. 9/9.

v17: a producer table (experimental)

v17 asked the regression report to include a table of every producer that can emit the risky input, one row per rule, checked by the reviewer row by row. Agents did read the detector's rule file and list generic-password. It scored 3/4 on chainloop, which we can't tell apart from v15.

What the models did

Model swaps are harder to read than harness changes. Each swap came with a different prompt version and, on the bridges, a different agent loop.

luna implementing, sol reviewing, original harness: 1/3. It missed the same pyinfra regression as deepseek-flash. A stronger model with the same instructions checked the same examples.
sol in both roles, v5: 1/2, and the source of the circular oracle above. Strong models are good at building a convincing test harness around their own assumptions.
Opus 5.5, v1, v5 and v11: 1/3, 2/3 and 3/4. Opus built the deepest probes of any model, including the 1.19-million-unit corpus. Before v14 added cross-checking, it still trusted the outdated wcwidth once. Its bridge also dropped skills until we fixed it after these runs (v12, the one Opus variant configured with skills, ran without them), so read the Opus cells as prompt-only.
GLM 5.3 flash, v14: 2/3. It passed pyinfra and chainloop and failed fedify's sacho check. Its bridge was silently dropping skills, which we found and fixed during this work, but the prompt carries the same rule, so we can't pin that failure on the missing skill. It was also slow: a median of 68 minutes a run, against 23 for deepseek-flash on the same prompt.
deepseek implementing, sol reviewing, v15: 4/4 on chainloop. That matches deepseek-flash reviewing on v15 (4/4), so we can't yet say the stronger reviewer helps. It also took longer: a median of 29.5 minutes on chainloop, against 21 for v15.

In these runs the harness moved the numbers far more than any model swap. That's a statement about this data. It doesn't say models don't matter.

Nine issues the harness had never seen

Everything above was tuned on the same three issues it was scored on. That's a training score. It can't tell us whether the rules generalize or just memorize three bugs, so we built a held-out set.

We took nine real bug fixes merged upstream after August 10, 2026, three each in Go, TypeScript and Python: gojq (empty regex matches in global mode), urfave/cli (arguments after a bare -), pflag (wrapping after an unbreakable word), magic-string ($ patterns in replacements), recast (parentheses around mixed ?? and ||), cron-parser (stringifying stepped ranges), python-dotenv (backslashes in single-quoted values), markdown-it-py (a blockquote at end of file with tables on) and pyjwt (padded Base64URL segments). The hidden grader for each is the maintainer's own test from the fix, which neither we nor Orbi wrote, calibrated the same way: it fails before the fix and passes after it. The issue text Orbi saw was rewritten to remove links, numbers and any hint of the fix.

Nine issues, three setups, 27 runs, all passed.

All three setups passed every held-out issue. The rules we tuned on three bugs didn't make Orbi worse on nine others in three languages. But these nine didn't need the rules either. They are ordinary bugs: the fix is local, and the tests the agent writes are close to the ones the maintainer wrote.

Medians per run. Tokens include cache reads, which were about 97% of the total.

And v15 cost more. On the held-out set, v15 used a median 5.8 million tokens per run against 2.8 million for the original harness, and 16 minutes against 11. On pflag it wrote 274 lines where the original wrote 88, most of it generated regression tests. About 97% of all tokens were cache reads, which on deepseek-flash are cheap, so the bill is smaller than the token count suggests. The extra time is real.

Issues whose first real fix broke something

The nine held-out bugs were easy for both harnesses, but the original incident was a fix that broke something else. So we looked for real cases of exactly that: an upstream fix that was merged, found to cause a regression, and fixed again. Three made the cut, each with a grader that fails the code before the first fix, fails the first fix itself, and passes the final one.

numpy: np.linspace(np.inf, np.inf, 5) returned NaNs. The first upstream fix handled infinities and broke ndarray subclasses; astropy's Quantity stopped working, and a second PR repaired it.
OpenTelemetry Collector, Elasticsearch exporter: retries shared one timeout budget, so data was dropped while Elasticsearch was down. The first fix stopped sharing the budget and, by accident, made timeout effectively infinite.
aiohttp: a compressed WebSocket message was corrupted when a PING or PONG arrived between its fragments. The first fix rejected valid compressed frames that followed an initial PONG.

Orbi saw only the original bug report. Nothing in the ticket mentioned the regression. Keeping working behaviour working is part of the job.

The grader tests both the original bug and the regression the first upstream fix introduced.

The original harness passed all four scored runs, and v15 all three. v15 never finished a numpy run: all three attempts were killed for memory, so its 3/3 covers aiohttp once and the OpenTelemetry Collector twice.

We expected this set to separate the harnesses. It didn't, for a simple reason: the mistakes humans made on their first try aren't the mistakes this agent makes. None of the four original-harness runs reproduced the regression the first human fix had introduced. The regressions Orbi did ship, in pyinfra and chainloop, came from a different habit: checking the example in the issue and nothing around it.

So a regression guard should target the agent's own failure modes rather than a general list of ways code breaks. With three bugs, that's as far as we can go.

Building the benchmark taught us as much as running it

We found more bugs in our own measurement than in the harness. Each would have produced a confident, wrong number.

A grader enforced the wrong rule. Our first fedify grader required CHANGES.md to stay untouched. Upstream always edits it. We switched to the project's own sacho check.
A grader measured beyond the ticket. The first chainloop grader also required newlines after a JWT to survive. The issue never asked for that, so every run "failed" a requirement nobody gave it. We split it into a core check (no worse than before) and an informational one.
A grader measured the wrong thing. The first pyinfra grader counted ANSI colour codes as columns.
The snapshot dropped files. recast's .gitignore ignores *.js while the repository tracks three .js test fixtures. We built snapshots with git add -A, which skips ignored files, so the agent worked in a repository with files missing and the grader failed on them. We threw both runs away and reran them; git add -A -f fixes it.
The answer leaked, twice. Once, an old fork PR had left a cross-reference on the upstream issue, and a fedify review found it and treated it as the reference answer. Once, a pflag reviewer ran gh pr view on the real upstream fix. In that case review happened after the code was written and passed in one round, so the result stands, but from then on every held-out run went through gh and git wrappers that only reach the benchmark's own repositories.
Our machine was part of the experiment. 14 runs were thrown away: two for the recast snapshot above, ten killed for memory (numpy builds from source, and /tmp was a RAM disk that agents filled with scratch files), and two that collided with an instance name a killed run had left behind. Memory pressure also killed the session driving the benchmark, twice. We now run one or two instances at a time, point TMPDIR at disk, and a watchdog stops benchmark runs before the kernel does.
API budget is shared. Every run and every production runner on the account draws on the same GraphQL budget. We added a guard that holds the queue when the budget runs low. At first we blamed most of the drain on the production runners' branch cleanup. Measured properly, it costs about 400 points an hour, far less than the figure we first wrote down.

The rule we took from this: before a grader scores anything, show it failing the known-bad version and passing the known-good one. When a result surprises you, read the transcript before you believe it.

Where this left us

After the first two rounds, v15 fixed every failure it was tuned on, broke nothing on twelve unseen bugs, and cost a median of about twice as much. You can't ship that as is. A guard that roughly doubles the median price of a fix has to show it catches something on bugs it has never seen, and the ordinary held-out bugs gave it nothing to catch.

So the question for the next round was narrower and harder: on unseen bugs shaped like the ones Orbi actually got wrong, does the original harness fail, and can a cheaper harness keep the fix? Part 2 has the answer.

This part has clear limits. The tuning set is three issues, the held-out sets are nine and three, and many cells have fewer than five runs. And a hidden grader only catches the defects someone knew to test for, even when that someone is the upstream maintainer.

If you run coding agents

Five things you can do this week, whatever agent you use:

Take a merged agent PR and write the one input you think it gets wrong. Run it. If it breaks, look at what your reviewer said.
When your agent writes a regression check, find its oracle. If the oracle was derived from the change, the check is decoration.
Make agents generate inputs with a program and report the counts. A reviewer can check a count; it can't check "I tested edge cases".
Put the repository's own tools on the agent's PATH and make their rules the source of truth, instead of writing project conventions into your prompts.
If you benchmark, calibrate every grader against a known-bad and a known-good version first, and keep the agent away from the real answer.

Orbi Cloud runs the loop described here on your own issues. The benchmark is at orbi-build/orbi-bench if you want to run it against your own agent.

Read the Claude Code comparison and Cloud.

Originally published on orbi.build.

Seven steps of an Orbi Cloud setup, on video

Lawrence Liu — Sun, 27 Sep 2026 13:00:00 +0000

The full setup is now on video: Orbi Cloud: from zero to a tagged release in seven steps. It is 83 seconds long, in English, and divided into chapters so you can skip to the step where you are stuck.

Watch the complete seven-step Orbi Cloud setup.

Seven key screens from the seven-step flow

Step 1: Orbi Cloud sign-in screen.

Step 2: GitHub App installation screen.

Step 3: Repository connection form.

Step 4: Environment provisioning status.

Step 5: First Issue status screen.

Step 6: Release form with version field.

Step 7: Managed Cloud subscription screen.

The seven steps are the current Cloud onboarding path:

Sign in with your GitHub identity.
Install the Orbi GitHub App on your account.
Connect a repository: the repo and the base branch Orbi works from.
Provision the environment. About a minute.
Your first Issue. Label it ai-ready, and Orbi writes the code and opens a pull request.
Cut a release. Type the version; Orbi bumps it, tags it and publishes the GitHub Release.
Subscribe. The first 3 deliveries are free; paid plans are Solo at US$29/month with 100M tokens, or Pro at US$79/month with 300M tokens. When the allowance runs out, only new deliveries pause: your data stays intact, and you can cancel at any time.

The seventh step is the one that starts paid delivery

Steps one through six are setup and delivery: you are handing Orbi the things it cannot infer: an identity, an installation, a repository, a runtime, an Issue, and a release. Step seven is where you opt into the Managed Cloud plan after the 3 free deliveries.

The release step still requires your decision: you decide the version, Orbi never increments it for you. The runner will bump the version file, create the tag and publish the Release. It will not choose the number. A version is a claim about what changed and who should care. That is a product decision, and reading it off a counter would be pretending otherwise. The release scope comes from the Milestone carrying that exact title; without one, the release stops at scope derivation and tells you why.

Step five is one label, and you can add it without leaving the terminal:

gh issue edit <number> --add-label ai-ready

What you are actually looking at

Every screen in the video is the product rendering, not a mockup: the sign-in card, the connect form with its repository and base-branch fields, the progress bar counting the step you are on, the connected-repository table with its provisioning status, the release form with the version box, and the subscription step. The GitHub App page is the real one at github.com/apps/orbi-build.

If you want to read instead of watch, the same path is in the docs: docs.orbi.build. If you want to run the whole thing on your own machine instead of ours, that is the same product, free to self-host.

Start here: orbi.build/cloud.

Read the Google Jules comparison and Cloud.

The Docker image made it out on try three

Lawrence Liu — Sat, 26 Sep 2026 13:00:00 +0000

Issue #1031 asked for a straightforward fix: publish the container built by 3rd/docker/Dockerfile, so a Docker search returns an image people can pull. The workflow now publishes on every release and on a manual dispatch: linux/amd64 and linux/arm64, tagged with the release version and latest, to both ghcr.io/orbi-build/orbi and docker.io/orbibuild/orbi.

Pull the published image:

docker pull ghcr.io/orbi-build/orbi:latest
docker pull docker.io/orbibuild/orbi:latest

The docs walk from pull to a first delivered Issue: docs.orbi.build/docker. The Hub overview lives at hub.docker.com/r/orbibuild/orbi.

Three failures, three different causes

All of this happened on 2026-09-17, the day the workflow shipped with release v0.5.17. Each failure left a fix behind, and each fix is a public pull request.

Try 1 (release v0.5.17): two jobs, two build contexts

The build-only verify job built the image fine; the publish job failed. Buildx could not find /orbi-container-setup.sh and /orbi-container-setup.service, the files the Dockerfile copies. The verify job pointed at the right directory; the publish job pointed at another context, so its build started from a view of the repository that was missing exactly those files.

Try 2 (manual dispatch): the Dockerfile path resolved from the workspace root

Run 35215809993 failed all three build steps with failed to read dockerfile: open Dockerfile: no such file or directory. #1044 names the cause: buildx resolves file: Dockerfile against the workspace root, so the fix wrote 3rd/docker/Dockerfile in all three places. The fix's own pull-request checks then failed from the other side: the workflow derived the image tag from the git ref, and a pull request's ref is 1044/merge, which buildx rejects as invalid reference format.

Try 3 (manual dispatch): the images landed; the overview step kept the job red

Run 35216638306 pushed 0.5.17 and latest to both registries, for linux/amd64 and linux/arm64. The job still ended red: the only failing step was the Docker Hub overview update, answered 403 because the token lacked a permission. #1046 made that step continue-on-error: true, so an overview hiccup can never fail a release again.

Three rules the failures bought

Rule 1: every build spells out its paths

Each build step now carries context: 3rd/docker and file: 3rd/docker/Dockerfile. Nothing depends on the working directory buildx happens to start from.

Rule 2: publish never runs on a PR ref

The build-only job runs on pull requests and builds under a pr-<n> tag with push: false; the publish job is gated on the event name. A ref like 1044/merge can no longer reach a tag.

Rule 3: an overview cannot fail a release

The Docker Hub overview step runs with continue-on-error: true. The release contract lives in the images; a description update answered 403 cannot turn a finished publish into a failed run.

Checked against .github/workflows/docker-image.yml on orbi-build/orbi main, 2026-09-18.

From pull to first delivery

A pulled image still needs a model provider before it can deliver. #1048 added the ORBI_PI_* environment variables: with ORBI_PI_PROVIDER, ORBI_PI_MODEL, ORBI_PI_BASE_URL and ORBI_PI_API_KEY set, the entrypoint generates pi-providers.json on first start, and it never touches an existing configuration. #1049 rewrote the Quick start around that: docker pull, then one copy-pasted docker run that reaches a real delivery, with build-from-source moved to its own section.

The first release after the fixes, v0.5.18, published green end to end. That is the state the next release inherits.

Pull the image, create an Issue, get a PR

The Quick start on docs.orbi.build/docker takes you from docker pull to a first delivery. The image overview lives on hub.docker.com/r/orbibuild/orbi.

Read the OpenHands comparison and Cloud.

Claude Code in Actions: who presses merge?

Lawrence Liu — Fri, 25 Sep 2026 13:00:00 +0000

Once Claude Code is wired into GitHub Actions, the implementation side is covered. The action answers questions, makes fixes, reviews a pull request, and posts progress to the thread. Its feature list stops there. The merge and the release still need an owner.

That boundary makes sense for a general-purpose action. It still leaves the question teams meet almost immediately: a pull request is sitting there, tests are green, Claude wrote it and Claude reviewed it. Who presses merge?

For the last month the answer on my own repository has been: nobody. 431 pull requests merged, 416 Issues walked the full delivery path, 40 releases tagged and published. I have not read the diffs. This post is what had to exist for that to be a reasonable thing to do rather than a reckless one.

The model is not the bottleneck

The failure people expect is that the model writes something wrong. That happens, and tests catch most of it. The failure that actually costs you is different: the thing that decides "this is good enough to merge" is the same session that wrote the code, or it is a person who stopped reading carefully three weeks ago.

So the first rule is that the reviewer is not the author. Not a second pass in the same session with a different prompt: a separate process, a fresh context, started after the pull request exists, reading the frozen diff. It does not know what the implementer was thinking, which is the point. If the code needs the author's intentions to make sense, the code is not finished.

That reviewer can also fix what it finds and push to the same branch, then re-emit its verdict against the new head. A round that ends with findings is not a merge; it is another round. The loop is bounded at five rounds, and exhausting them is a stop, not a merge.

Three conditions, checked at the moment of merge

A verdict is not enough by itself, because time passes between the review and the merge. Three things are re-checked against the live remote:

Continuous integration is green on the exact head that was reviewed.
The head has not moved since the verdict. A push after the review invalidates it.
The base is current. The pull request must contain the latest main, absorbed by a plain merge, not a rebase that rewrites what was reviewed.

In the merge step that is one command, and the pin is the whole point:

gh pr merge "$PR" --squash --match-head-commit "$REVIEWED_HEAD"

If the remote head moved after the review, that call fails instead of merging something nobody reviewed. Only then does the merge land, pinned to that exact commit. If any condition fails the Issue moves to a state that waits for the next tick, or stops for a human. There is no path where a delivery merges because it looked fine.

These are ordinary conditions. Any team would say them out loud in a code review conversation. The difference is that they are three lines in a state machine rather than three habits in a person's head, so they hold at 3 a.m. on the ninetieth pull request.

What a person still does

I write the Issue. That is the whole job, and it is more work than it sounds: one runtime outcome, the acceptance conditions written so that "done" is checkable by something that cannot ask me what I meant. A vague ticket produces a confident wrong delivery, and no gate downstream catches that, because the gate checks whether the code does what the ticket said.

I also write the release ticket: a version number and a scope. The release itself is a deterministic state machine, not a model session. It freezes the base, waits for green CI, bumps the version, tags, publishes and closes the milestone. Any failure stops and waits for me.

Between those two tickets, nobody is in the room.

If you are running the action today

You do not need to replace it. The action is the execution layer; what is missing is the part that decides. Three things to add, in order of how much they buy you:

A reviewer that is a different session from the implementer, reading the frozen diff, with the authority to fail the delivery.
A merge gate that re-checks CI, head and base at merge time rather than trusting a verdict from ten minutes ago.
A release path that is deterministic, so that shipping is not a model deciding it is probably fine.

Orbi is my implementation of exactly that, open source (AGPL-3.0) and self-hostable, and it drives its own repository: every one of those 431 pull requests is public, and so is every review verdict, in the Issue threads. It runs on the Pi agent harness with your own model: any OpenAI-compatible API, or a Codex subscription. Claude Code is not one of its engines.

Or take the three conditions and build them into whatever you already run. The conditions matter more than whose code enforces them.

Read the Claude Code comparison, CI gates guide, and Cloud.

The k8e etcd ticket that closed too early

Lawrence Liu — Thu, 24 Sep 2026 12:38:54 +0000

k8e is a Kubernetes distribution with an embedded etcd. It has 497 stars; Deshi (Tommy) Xiao maintains it. On September 20 he opened issue #612, asking for an end-to-end robustness suite for the embedded etcd, delivered in four phases. Safe recovery after a crash comes first, then multi-node failure, backup restore and long-running stability.

At 10:28 (UTC+8) he added the ai-ready label. The times and numbers below come from the public PR, its reviews and the issue's label history, except the runner crash count, which comes from the runner's log.

The first version was rejected

Orbi opened PR #613 at 11:37 with phase one only: 10 new files, 2,483 lines. It had an operation history recorder, a recovery oracle, a SIGKILL test that kills a child process mid-write, a WAL corruption test and a quota test.

At 11:46 the maintainer posted what SonarCloud and DeepSource had flagged. Orbi started pushing fixes at 11:57.

At 13:21 the review on the maintainer's account came back REQUEST_CHANGES. By then the PR was 2,695 lines, and the review had three findings:

R1: the oracle accepted an impossible state. Say two CAS writes both expect revision 3. A is acknowledged and B's outcome is unknown. If recovery shows B's value, something went wrong, because whichever write ran second must have failed its compare. The oracle accepted it anyway: it treated an unknown CAS as an unconditional put.

The history in question:

  k is at rev 3
  A: CAS rev3 -> a  acked
  B: CAS rev3 -> b  unknown
  recovered: k = b
  oracle: OK   (should fail)

R2: the process being killed was also the one taking notes. The recorder ran inside the process the test SIGKILLs. A write could be acknowledged, and the process killed before the acknowledgement reached the log. That write would be logged as "unknown", so losing it would pass the test. The issue had asked for the recorder to sit outside the node under test.
R3: an empty value looked like a missing key. An empty payload was never hashed, so the oracle read "written as empty" as "does not exist". A lost empty key would pass.

The review also said the PR body's Fixes #612 would close a four-phase issue after phase one.

None of the three was about style. Each meant the suite could report "recovered correctly" when the data had not recovered. A test suite that does that gives you confidence you haven't earned, which is worse than having no suite at all.

Orbi's own mistakes on the same PR

Orbi's engine also reported three failures of its own, each in a PR comment:

At 12:28 it tried to add a label the repository didn't have (ai-awaiting-merge). At 12:37 the engine gave up and marked the issue ai-blocked, which hands the next step to a human. The maintainer removed that label at 14:16 and put ai-ready back at 14:41, and the work resumed.
At 14:54 Orbi's own independent review produced no verdict line, so its result couldn't be parsed.
At 15:19 its resume check demanded Fixes #612 in the PR body, the same line the maintainer's review had rightly asked Orbi to remove.

Orbi never cleared the third one, and the failure spread beyond the ticket: it crashed the whole runner process serving his repository, 84 times between 15:19 and 22:10 by the runner's own log, and his entire queue stalled. The public write-up, orbi#1219, was filed partway through and counts 77. At 19:59 the maintainer commented on the issue, 继续修复吧 ("go ahead and keep fixing"). Nothing happened after that. The rule that every PR body must say Fixes works for a single-shot issue and is wrong for a phased one; the engine enforced it anyway, and the human reviewer was right.

The fix

At 15:05 one commit, f8e9dd9c, addressed all three findings. An unknown CAS whose expected revision is older than the key's last acknowledged revision is now dropped. The child process only hosts the etcd member, while the parent runs the client and the recorder, so the recorder survives the kill. Every put and CAS payload is hashed, including the empty string.

New tests cover R1 and R3 in both directions. The PR body now reads "Part of #612 — this PR lands the phase-1 layer only".

At 22:12 the second review came back APPROVED. It traced each fix and set two conditions. The first was to run the race tests independently. The second was to drop a stale Fixes #612 from Orbi's first commit message; the review called that one optional, because it believed only the PR body closes issues. It also listed the remaining gaps, which the design doc already disclosed. The maintainer merged it himself at 22:13, 38 seconds later, and the record doesn't show either condition being carried out first. CI on the merged head: 801 tests, 49 new, 0 failing.

The issue closed anyway

The review was wrong about that commit message. GitHub also closes an issue when a closing keyword in a merged commit message points at it. One second after the merge, #612 closed as completed, closed by Orbi's first commit, with one phase of four landed. The first review had warned about exactly this outcome, and it came through the one place nobody had fixed.

That was Orbi's fourth mistake on this ticket, and the one with the biggest consequence. Our engine's rules require Fixes #N in the PR body, and the delivery agent also put it in its first commit message. Both reviews flagged the keyword, the first in the PR body and the second in the commit. As of this writing, #612 is still closed.

The second PR: 3 hours 12 minutes from open to merge

The next day brought #614, rqlite compatibility for the M0 gate. The maintainer labelled it at 15:21. PR #615 added 3,294 lines across 11 files. It opened at 16:49 and was waiting for approval from 17:33. A LoopX review was posted from the maintainer's account, he approved at 20:00:52, and Orbi merged 29 seconds later.

Looking back

Orbi did not get this right the first time. The maintainer had to unblock the issue and put it back in the queue, ask Orbi to keep going when it stalled, and finally merge past one of Orbi's rules that was wrong for his ticket. The four-phase issue still closed after phase one.

When the review said no, it gave reasons, and the fix landed on the same PR. On a hard ticket in someone else's codebase, that took 10 and a half hours from PR open to merge. Every failure, Orbi's included, is in the public record, so you can check this post against it.

Two things changed in Orbi afterwards (orbi#1219). A bad PR body now fails only its own ticket and no longer kills the runner. And Orbi's workflow contract now says a phased initiative must not be carried by one issue: the maintainer splits each phase into a child issue, and each child's PR closes that child and nothing else.

If you maintain a repo with a queue of well-specified issues, Orbi Cloud runs the same loop on yours.

Read the Devin comparison and Cloud.

Why Orbi is open source again

Lawrence Liu — Thu, 24 Sep 2026 12:37:00 +0000

As of v0.5.44, released on September 24, Orbi is licensed under AGPL-3.0, or the Sustainable Use License (SUL) v1.0. You pick one. Self-hosting is free under either, and GitHub now reports the repository's license as AGPL-3.0.

This is the second license change in a month, so here is the whole history and the reasoning behind each step.

The timeline

Date	License	Where
Aug 26	Apache-2.0	#81 added the LICENSE file
Sep 4	Sustainable Use License v1.0	commit 365c5bc
Sep 24	AGPL-3.0 or SUL v1.0	#1332, shipped in v0.5.44

Why we left Apache on September 4

Apache-2.0 lets anyone take the code and sell it as a hosted service. Orbi's runner is meant to run on your own machine, and the SUL draws its line in the same place: free for your own use, a separate agreement if you resell it as a service. It is the license n8n uses, and its restriction and patent clauses were copied word for word.

Timing mattered too. The repository was ten days old and had a single copyright holder, so the change needed one person's agreement. The commit also added a relicensing clause to CONTRIBUTING.md. Contributors agree that their work may be released under a different license later, provided that license continues to allow self-hosted use free of charge.

What the Sustainable Use License cost us

The SUL is not an OSI-approved license, so Orbi could not be described as open source. That turned out to matter more than we expected.

Several lists that fit Orbi accept only open source projects. kyrolabs/awesome-agents says a contribution "needs to be open source". Awesome-AI-Agents says licenses that only make the source public "will usually be rejected". Everywhere else, every description of Orbi needed the same footnote: the source is public and self-hosting is free, but it is not OSI open source.

Why AGPL, and why keep the SUL

AGPL-3.0 is OSI-approved. It also has a network clause: if you modify Orbi and offer it to others as a service, you have to publish your changes under the same license. It does not stop anyone from running an unmodified Orbi as a service. The SUL did stop that, and we are giving it up.

Elastic made the same move in 2024, when it added AGPL as an option next to its existing licenses. We kept the SUL as the alternative because some companies do not allow AGPL code at all. They can still use Orbi under the SUL.

The relicensing clause made this possible, and its condition still holds. Self-hosting stays free under both licenses. The one outside contributor so far started contributing after the clause was in place.

What changes for you

Self-hosting: nothing. It is free under either license, with your own model and your own repository.
Describing Orbi: you can call it open source now, and list it anywhere that requires an OSI license.
Offering Orbi to others as a service: under AGPL-3.0 you can, as long as you publish any modifications. The SUL does not allow it without a separate agreement.
Orbi Cloud: unchanged. It is the paid hosted version, run by us.

The package metadata carries both options as one SPDX expression, from pyproject.toml:

license = "AGPL-3.0-only OR LicenseRef-Sustainable-Use-1.0"
license-files = ["LICENSE", "docs/licenses/sustainable-use-license.md"]

The files are in the repository: LICENSE holds the unmodified GNU AGPL-3.0 text, and docs/licenses/sustainable-use-license.md holds the SUL and a short note on how the two options work.

Read the OpenHands comparison and Cloud.

Waiting for an answer the user can't give

Lawrence Liu — Tue, 22 Sep 2026 14:32:09 +0000

There is a failure mode that does not look like failure. Nothing crashes. No error is logged. The system is doing exactly what it was designed to do, which is wait — and the person it is waiting for has no idea, and no way to respond.

We shipped three of these. We found them in one afternoon, because we finally became our own customer.

The waiting is correct

Our engine will not advance a version on its own. When a milestone closes, an idle tick opens a ticket: this version is complete, here are the candidates, which one is next. Then it stops.

That lock is deliberate and we are not removing it. Deciding what ships next is a product decision. An engine that picks the next version number because the previous one filled up has confused a counter with a judgement.

So the ticket is right. The pause is right. What was wrong is the sentence inside the ticket:

Please run orbi milestone set <target> to advance active_milestone.

That instruction assumes a CLI and a shell on the host. A managed tenant has neither — their sandbox runs on our hardware, which is the entire point of it being managed. The engine asked a question, on a ticket the tenant could read, and accepted only an answer they could not produce.

Worse, managed sandboxes never rendered auto_next_milestone = false at all. So they did not get the ticket either. The engine paused silently and waited for a decision that nobody had been asked to make.

The day we noticed

On 2026-09-22 we migrated orbi-build/orbi — the engine's own repository — onto our hosted sandbox. It shipped v0.5.39 from that sandbox the same morning.

Then the milestone closed, and everything stopped. No notification. No error. Delivery simply ceased, until a human noticed, created the v0.5.40 milestone by hand with gh api, and edited the policy file directly.

We had been running this gap in production for our customers. It took putting our own release line behind it to feel it.

The same shape, in the UI

Later the same day, a full walkthrough of the product — register, install the App, connect a repo, provision a sandbox, file an issue, deliver, merge, release — surfaced the second instance.

The status page has a release form. It is a text box. You type a version number from memory, and that string becomes the release scope.

The page never shows you which milestones exist. We checked: grep -rn "milestone" src/pages/status.ts src/page.ts returns nothing. Not one line. Meanwhile, six hundred lines away in the same file, the page does this for issues:

These issues have not been handed to Orbi yet: label an issue ai-ready and Orbi will start on it.

The page already scans GitHub on your behalf and lists what you can act on. It just never extended that courtesy to milestones. So you type a version you half remember, and if you get it wrong nothing tells you — the rejection happens later, asynchronously, on a ticket you have to go find.

The same shape, in the config

The third instance is the one that explains the other two.

Our engine writes the active milestone into a repository variable on every tick. The value comes from config.active_milestone — which lives in the sandbox's own orbi.toml file. Not GitHub. Not our control plane.

So the control plane can create a release ticket while the sandbox still believes nothing is in flight. Two systems, two answers, no reconciliation. In our production logs, every tick, for the repository we had just walked through:

INFO active_milestone_variable_absent repo=orbi-build/orbi-beta-e2e-org-09182001

Then we looked at how a field gets from the control plane into that file, and found there is no general mechanism at all. There are four separate hand-written sync functions — sync_orbi_toml_engine_track, sync_orbi_toml_providers, sync_orbi_toml_oauth, plus the initial render — each one added for a single field, each one staging a temp file and swapping it atomically. The comments record the accretion:

# Issue #116: the connect-time base branch rides the payload and must reach
# the rendered orbi.toml
# Issue #770: attribution_footer rides the same way

Adding one field means editing four files. Which is why nobody added active_milestone: grep -rn "active_milestone" src/*.ts host-provisioner/*.sh returns nothing. The control plane does not know the field exists.

What the tenant can and cannot touch

One design question resolved itself once we looked at the filesystem.

We were about to specify merge semantics for "what if the tenant edited a managed field by hand". Then we checked:

$ sudo ls -la /home/<sandbox>/orbi/orbi.toml
-rw------- 1 orbi-... orbi-... 926 Sep 22 06:39 orbi.toml

$ sudo ls -la /home/<sandbox>/.ssh/
-rw------- ... id_ed25519       # outbound deploy key
-rw-r--r-- ... id_ed25519.pub
-rw------- ... known_hosts
# no authorized_keys

Mode 0600, owned by the sandbox user, and no authorized_keys. The tenant cannot SSH in. There is no hand-editing to reconcile.

That makes the design simpler than we assumed. The control plane is the sole writer, so the merge only has to preserve other control-plane fields — there is no external drift to detect.

It also points at a stronger invariant worth holding: orbi.toml should be a pure derivative, reconstructible from control-plane state alone.

Today it is not. Engine track and providers live only in the sandbox; the control plane holds a partial truth. Those four sync functions each maintain a fragment of the real answer, and no single place can tell you what a sandbox is supposed to be configured as.

That distinction — derivative versus authoritative — is the one that decides how much of this class of bug you get. A derivative file can be regenerated, diffed against intent, and repaired by a sweep. An authoritative file that only one machine holds can only be inspected by going there. Every one of the four sync functions exists because a field was made authoritative in the sandbox first, and a way to change it had to be retrofitted afterwards.

We are not fixing that in one step. But knowing which invariant we want makes the next field's default obvious: the control plane holds it, the sandbox derives it.

The pull-versus-push question answered itself too

For pushing config down to sandboxes, the obvious tradeoff is pull (the sandbox asks on a schedule) versus push (the control plane triggers a refresh).

Push means opening an inbound channel to sandbox hosts. Pull means a delay of up to one refresh interval. The usual move is to weigh latency against attack surface.

Except the interval is already one minute:

$ systemctl cat orbi-cloud-provisioner.timer | grep OnUnit
OnUnitActiveSec=1min

And the refresh endpoint's query is WHERE r.status = 'active' AND t.status = 'active' — every live sandbox, every minute, not just pending ones. So a pull costs at most sixty seconds, on an action that already waits for the engine's next claim. There is no tradeoff to weigh. Push would open an inbound path to buy less than a minute.

Measuring the thing before arguing about it took two commands.

A wrong diagnosis, corrected by the database

While reading production logs we found 288 identical errors in six hours:

ERROR milestone_reconcile_failed repo=zzuu080603/VCPToolBox-macOS
subprocess.CalledProcessError: ... /milestones?state=all ... exit status 1

The obvious read was credential rot, so that is what we wrote down. Reproducing it inside the sandbox gave 401 Bad credentials — which seemed to confirm it, especially since the repository is public and readable with a maintainer token.

Then we checked the database, and the diagnosis collapsed:

login=cdredfox    tenant=inactive  codeduck            repo=inactive
login=zzuu080603  tenant=active    VCPToolBox-macOS    repo=inactive  ← failing
login=zzuu080603  tenant=active    Tianshu-harness     repo=active    ← fine

Both failing sandboxes belong to inactive repositories. The refresh endpoint filters on status = 'active', so those tokens are never refreshed — by design, correctly. The same tenant's active repository works perfectly. That is the control group, sitting right there in the same query.

The real defect was never credentials. It is that we keep reconciling milestones for repositories that were switched off, forever, once per tick. The 401 is a symptom of a correct policy meeting a loop that should have stopped.

Read the error, then read the data. In that order — we did it backwards and filed a ticket with the wrong root cause, which then had to be publicly corrected.

What makes this worth writing down is not the mistake. It is that reproducing the error confirmed the wrong diagnosis. Running the failing command inside the sandbox returned 401 Bad credentials, which is exactly what a credential problem looks like. The repository was public and readable with a maintainer token, which seemed to rule out "the repo is gone". Two pieces of evidence, both genuine, both pointing the wrong way.

A reproduction tells you what happens. It does not tell you whether it is supposed to happen. The database row — repo_status = inactive — was the only thing that could distinguish "this token should work and doesn't" from "this token was never meant to be refreshed". And the control group was sitting in the same query the whole time: the same tenant's active repository was working perfectly.

The generalisable version: when an error reproduces, you have confirmed the mechanism, not the diagnosis. The next question is always whether some policy intends this outcome — and policy lives in data, not in logs.

What connects them

Three defects, one shape: the system knew something the user needed, and did not say it.

It knew which milestones existed, and made you type one from memory. It knew a decision was pending, and told you to use a CLI you do not have. It knew the control plane's intent, and left the sandbox holding a different answer.

None of them logs an error, because none of them is an error. Every component is doing its job. The gap is between components, in the space where a human was supposed to be handed something and was not.

That space is invisible from inside the code. You cannot grep for it, and your tests will not fail on it, because nothing is broken. You find it by standing where the user stands — which for us meant moving our own release line onto our own product and waiting for it to stall.

It stalled in four hours.

The tickets, all filed on 2026-09-22: orbi-cloud#878 (no path for a managed tenant to decide), orbi-cloud#869 (list the milestones in the UI), orbi-cloud#870 (a general config push instead of a fifth hand-written sync), and orbi#1283 (the one whose root cause we got wrong first).

Originally published at orbi.build/blog/waiting-for-someone-who-cannot-answer

Designing one slash command

Lawrence Liu — Tue, 22 Sep 2026 14:31:53 +0000

Our engine opens a ticket when it finishes a milestone. The ticket says: this version is done, here are the candidates, tell me which one is next. Then it waits, on purpose.

The waiting is deliberate. Automatic version advancement is locked off, because deciding what ships next is not a decision a release engine should make for you.

The problem was the answer. The ticket told you to run orbi milestone set v0.5.40 — a CLI command, on the host machine. Our managed tenants have neither. Their sandbox runs on our hardware. So the engine was asking a question that its own audience could not answer.

Closing that gap meant building an in-ticket command: comment /milestone v0.5.40 and the engine does the rest. It is one small feature. It took a day of design because almost every choice in it has been made before, by people who then had to live with it.

Why not a verb

The first argument was about the name. /advance was the obvious candidate, and it is wrong.

In-ticket commands are named after states, not actions. Prow has /lgtm, /hold, /approve. bors has bors r+. None of them is an imperative verb. This is not a style preference — it comes from what the command means. A CLI says do this thing, so a verb fits. An in-ticket command says set it to this, so a noun fits. /milestone v0.5.40 reads as "milestone: v0.5.40", the same shape as /hold.

There is a second reason, specific to us. The CLI is already called orbi milestone set. Naming the in-ticket command /milestone means the documentation says the word once. advance would have been a third name for a thing that already has two.

We also rejected bump version, because it reads like editing the version string in pyproject.toml — which is what the release process does, not what this command does. A name that makes people expect the wrong side effect is a bad name even if it is short.

That last one is worth dwelling on, because it is the rule the other rejections follow from. A command name is a promise about side effects. /advance promises something moved forward but does not say what — the version? the delivery? the queue? bump version promises a file edit that never happens. Both are short, pronounceable, and wrong in the same way: the reader forms an expectation the implementation will not honour.

/milestone v0.5.40 makes exactly one promise — the milestone is now v0.5.40 — and the three things it does are all in service of making that true. Creating the milestone, opening the release ticket, landing the config value: none of them is a surprise once you have read the name.

Why no @

The second argument: /milestone or @orbi milestone?

The industry splits cleanly, and the dividing line is who controls the bot's account name.

Prow and bors use a bare slash. They have to: the bot account name varies per deployment. Kubernetes runs k8s-ci-robot, OpenShift runs openshift-ci-robot. A syntax that hardcodes @name cannot be forked.

Dependabot and Copilot use @bot. They can: there is exactly one Dependabot, its name is globally unique, and the mention buys a notification and autocomplete for free.

We are the first kind. The engine is open source and self-hostable. More pointedly, the same engine already runs under two identities in our own org — on our bootstrap runner the comment author is a human account, and in a managed sandbox it is orbi-build[bot]. We confirmed both on our own repository the same day. Binding the syntax to a name would break self-hosted users on day one.

The tempting middle path is to accept both: match @orbi milestone and /milestone, and let people use whichever they like. We rejected that too, and the reason is not aesthetic. Two syntaxes mean two things to document, two things to test, and two things that can drift apart when someone adds the third command. The mention buys a notification — genuinely useful — but the cost is paid on every future command, forever, by everyone maintaining this. A notification is not worth a permanent fork in the grammar.

The regex is three decisions in one

Anchoring at the start of a line, with the multiline flag:

(?mi)^/milestone[\t ]+(\S+)[\t ]*$

Prow writes this in Go. bors splits the comment into lines and anchors each one in Elixir. Two unrelated implementations, same choice — which is usually a sign the choice is load-bearing.

It buys three things at once:

the command can appear on any line of a long comment, so you can explain yourself and then issue it
it is automatically immune to GitHub's quote-reply, because > pushes the slash off the line start
it is automatically immune to indented code blocks, for the same reason

What it does not buy is immunity to fenced code blocks. And that turned out to be the most interesting thing we read all day.

The bug Prow still has

A regex does not parse Markdown. If you write a comment explaining how to use /hold, and you put the example inside triple backticks, the regex sees a line starting with /hold and fires.

Prow knows this. There is a function for it, DropCodeBlock, in pkg/markdown/code_block.go, with a 2025 copyright header. So we checked how widely it is used:

$ grep -rn "DropCodeBlock" --include=*.go . | grep -v _test.go
pkg/plugins/trigger/generic-comment.go:66
pkg/markdown/code_block.go:25 (the definition)

One call site. Out of fifty-plus plugins. Writing /hold inside a fenced block on Prow fires today.

This is not a criticism of Prow's engineers; it is a structural consequence of where they put the regex. Prow gives every plugin its own pattern — pkg/plugins/hold/hold.go:41 has (?mi)^/hold(\s.*)?$ written out locally. The convention is repo-wide but nothing enforces it, and when each command owns its own matching, a fix to the shared sanitiser cannot reach them. The fix exists. It just cannot arrive.

So our shared layer compiles the pattern itself, from a declared verb. A command author supplies verbs=("milestone",) and never sees a regex. They cannot omit the anchor, and they cannot skip the fenced-block stripping, because they were never given the opportunity to write either one.

The escalation bors has

The same question applies to permissions, and here the comparison inverts: bors has the better design and the worse bug.

Prow makes permission each plugin's job. There is a shared helper, TrustedUser(...) at pkg/plugins/trigger/trigger.go:253, but a plugin has to remember to call it, and the standard differs per plugin — lgtm.go:291 checks collaborator status, while override.go:317 accepts repo admin or a GitHub team or top-level OWNERS. Worse, the WhoCanUse string shown in help (override.go:251) and the code that actually authorises (override.go:326) are two separate truths, kept in sync by hand.

bors declares instead. required_permission_level_cmd/1 in lib/web/command.ex:356 maps each command to :none, :member or :reviewer. Then run/1 at command.ex:337-354 takes the strictest level across every command in one comment and checks once, before executing any of them. That is the right shape.

But run/2 is reachable directly, and command.ex:513-516 reaches it:

def run(c, :retry) do
  {commenter, cmd} = Logging.most_recent_cmd(c.patch)
  run(%{c | commenter: commenter}, cmd)
end

:retry requires only :member (command.ex:376). It replays a historical command under the original commenter's identity, and it enters through run/2, so the gate in run/1 never runs. A member can replay a reviewer's command.

The lesson is not "bors is insecure". It is that the gate was correct and the second entrance defeated it. One execution path, or the gate is decorative.

What we took from each

Two more findings worth copying, one in each direction.

Prow's genuinely good idea: ValidatePluginsUnknown at pkg/plugins/config.go:1381 rejects a configured-but-unregistered plugin by looking it up in the help table. That makes "shipped without documentation" a hard error rather than a code review note. Our registry entry carries usage and description, and a binding test fails if a registered command is missing from the docs.

Prow's gap: RegisterGenericCommentHandler at pkg/plugins/plugins.go:176 is a bare map assignment. Register the same name twice and the second silently overwrites the first; two plugins both matching ^/hold both run, in nondeterministic goroutine order. We raise on a duplicate name and on overlapping verbs, at import time.

The registry already existed

The shape we landed on was not invented for this. Our journal module has had it for months — JOURNAL_EVENTS at src/orbi/journal.py:167 is an explicit dict, written out in the module. No decorators discovering things, no entry points, no dynamic import. Registration is validation: event() raises on an unregistered name, so a typo fails immediately instead of writing a silently misnamed event. And a binding test pins the registry against the EN/ZH documentation tables and the exporter's known kinds, in all three directions.

So the command registry looks the same:

@dataclass(frozen=True)
class CommandSpec:
    name: str                              # the registry key and the receipt marker
    verbs: tuple[str, ...]                 # the shared layer builds the regex from these
    permission: str                        # declarative; the dispatcher enforces it
    validate: Callable[[str], str | None]  # business rule; None means accepted
    apply: Callable[..., None]             # raises TicketCommandError
    usage: str
    description: str

TICKET_COMMANDS: dict[str, CommandSpec] = { ... }

The shared layer owns — and solely owns — regex construction with the anchor, fenced-block stripping, quoted-line stripping, skipping the runner's own comments, permission enforcement, last-occurrence-wins, and receipts. milestone_command.py shrinks to a registrant: a validate that checks the version against the candidate set, and an apply that runs the three steps.

Adding a second command becomes "edit one dict, and the tests tell you what else you forgot".

One command, three idempotent steps

The original ticket said this command changes a config value. That was wrong, and finding out why took a grep.

A complete advance needs three things to happen. We had one of them:

Step	Engine capability before this work
Create the milestone	None. `create_milestone` and `milestones --method POST` had zero hits repo-wide
Open the release ticket	None. Only `arm_release_ticket`, which labels an existing ticket
Land `active_milestone`	Yes — `rewrite_active_milestone_line`

The evidence was sitting in our own repository: v0.5.40's milestone had been created by hand with gh api, and its release ticket did not exist at all.

The reason it had gone unnoticed for so long is more uncomfortable. A release ticket template lives at .github/release-ticket-template.md, fully specified. But grep -rn "release-ticket-template" src/ prompts/ AGENTS.md returns nothing. No code has ever read it. Maintainers had been filling it in with a local client-side tool — a patch installed on exactly one machine. Managed tenants were hitting the unpatched original.

The fix is not to ship that local tool. It depends on a specific client, and which client a tenant uses is not ours to decide. What belongs in the engine is the capability: render the repo's own template, fill the fields, open the ticket.

So /milestone v0.5.40 does three things, each independently idempotent: create the milestone if absent, open the release ticket if absent, land the config value. Run it twice and you get one milestone and one ticket.

The branch we refused to write

Landing the config value has two destinations, because active_milestone can come from two places: a repository policy file (.github/orbi.toml) or the host's own config. Write to the wrong one and the value either does not take effect or takes effect for the wrong scope.

The obvious implementation is a runtime check: am I running in a managed sandbox or on a self-hosted runner? We refused to write that branch.

There is no reliable signal for it. The nearest candidate, engine_source_track, can be set on a bootstrap runner too, so the check would be a heuristic pretending to be a fact — and heuristics in a write path fail in the direction nobody tests.

More importantly, the question is the wrong one. We do not need to know where we are running. We need to know where this value came from, and that is already known, precisely, by the code that read it: resolve_policy in src/orbi/repo_config.py:287-290 knows whether active_milestone came from the repo policy or the host config, because it is the thing that looked.

So the branch keys off provenance, not environment. Policy-sourced values are written back to .github/orbi.toml and pushed; host-sourced values go through rewrite_active_milestone_line. Two paths, one question, and the answer is a fact rather than an inference.

This generalises: when you find yourself about to detect your own environment, check whether the thing you actually need was already determined by whoever supplied the input.

Every step is deterministic. No model is invoked — the command's arguments are the input, string substitution is the transformation, and a file write is the output. An LLM in that path would add nondeterminism to an operation whose entire value is being predictable.

What we deliberately did not decide

There is a real open question underneath all of this, and we wrote it down rather than answering it by accident.

Commands are currently read from exactly one place: the confirmation ticket the engine itself opened, on an idle tick, only when auto-advance is off. Three conditions stacked. Outside that intersection, a /command is never seen.

That is the narrowest possible scope, and it was the right call for a first command. But it decides which commands can exist. /approve to clear a review gate, /retry to requeue a blocked delivery, /cancel to stop one in flight, /model to override for a single run — none of them has anywhere to land, because delivery tickets are opened by humans and the engine never opens a confirmation ticket for them.

Widening the scope to scan every ai-* ticket costs N extra API calls per tick, a per-ticket cursor, and a rethink of how receipts dedupe across many tickets. That cost is only justified by a command that needs it.

So the question is not "which option is better". It is: which of those four commands do we actually want, and is any of them worth a per-tick scan? If the answer is none for now, the narrow scope stands.

Why "not yet" is a decision and not a dodge

We parked this in a discussion thread rather than settling it, and that choice deserves the same scrutiny as the others.

The refactor that lifts the shared layer out (#1294) touches every piece of machinery a wider scan would use — the parser, the permission gate, the receipt dedupe. It would have been easy, and superficially tidy, to widen the scope while we were in there. "We're already touching this code" is the most common reason a scope decision gets made by accident.

But look at what the widening actually costs. Receipts currently dedupe by scanning one ticket's comment list; across many tickets that needs a different mechanism entirely. Every tick gains N API calls plus pagination and rate-limit handling. The reachable surface grows from "tickets we opened" to "every delivery ticket". Those are not incidental — they are the design of a different feature, and they would have been designed in passing, to serve commands nobody had specified yet.

The alternative failure is just as real: deciding now to stay narrow, and writing that into the architecture so firmly that /retry becomes expensive later. So #1294 states its own scope neutrality explicitly — it moves code and adds nothing, and it does not read the discussion's conclusion.

The cost of widening is only justified by a command that needs it. Until one of /approve, /retry, /cancel or /model is specified, there is no information that would make the choice better than a coin flip. Writing the question down, with both option's costs enumerated, is what keeps it from being answered by whoever next opens that file.

The result

The command shipped the same day. The implementation was +2073/-81 across 18 files, and the follow-up that lifts the shared layer out of the milestone module is specified and queued.

The part worth keeping is not the feature. It is that two mature systems, both of which solved this before us, each left one defect visible in their source: Prow's fix that cannot reach its callers, and bors's gate with a second door. Both are the same class of mistake — a correct decision placed where it can be bypassed.

We got to read both before writing ours. That is the actual luxury of building on open source, and it is worth the day it costs.

The tickets: #1290 (the design), #1293 (the implementation), #1294 (the shared layer), and #1295 (where commands should be read from). Every Prow and bors line number above was grepped at HEAD on 2026-09-22, not recalled.

Originally published at orbi.build/blog/designing-a-slash-command

An autonomous coding agent that can tell itself no

Lawrence Liu — Sun, 20 Sep 2026 09:48:48 +0000

Search for an autonomous coding agent and you will find a lot of tools that open pull requests. That part has been solved for a while. The part nobody shows you is the refusal: what happens when something reads the diff and says this does not ship.

I want to walk through one delivery where that happened. The writing part is easy to demo. The refusal is not.

The delivery

Issue #1018 described a real defect in our own release code. A change had moved the docs sync before the tag push, which opened three windows where a tagged release could exist without its docs page.

The Issue got the ai-ready label. Nothing else was set up.

Orbi claimed it, worked in an isolated worktree, and opened PR #1023 against a recorded base commit. So far this is what every agent does.

Then a second session, with a fresh context and no memory of writing the code, read that diff.

The refusal

Round one came back with one Major finding. Not a style note:

Location: src/orbi/release.py:2377

Note: If publish_release fails after the tag is pushed,
rollback_release_docs removes the docs commit while leaving the remote tag
published. CI can then observe a released tag without its docs page,
violating acceptance criteria.

The fix for the bug had reintroduced the bug, in a narrower window. The review named the file, the line, the mechanism, and what a correct fix would look like.

Orbi read the finding, pushed a recovery path, and ran again. Round two came back too, this time on coverage: new branches with no tests. Three more commits.

Nine commits in total, +427/-43, three hours and two minutes from the Issue to the merge. Then v0.5.17, with the Issue and the PR listed in the generated notes.

No human decided anything in that window.

The part the screenshots cannot show

Here is something I had to label on the video we cut from this run, because the frames are misleading on their own.

The runner posts through my own GitHub account. It has my token, so its commits carry my name and its review comments carry my name and my avatar. On screen, the review that blocked PR #1023 appears under xqliu, with the badges Collaborator and Author.

A first-time visitor reads that as a developer reviewing his own pull request, which is exactly the impression the whole exercise is meant to refute. Directly below that comment sits a github-actions comment wearing a Bot badge, so the same screen teaches you what a bot looks like here and then shows you a review that is not one.

I cannot make GitHub show it differently. What I can do is say it out loud: that was a second model, running in a separate session, and it had no memory of writing the code it refused.

What changes when the gate can fail

An agent that writes code and opens a PR has moved the work to a human. Someone still has to read it, decide, and merge. The agent did the typing.

Making the gate real turned out to be mostly plumbing. The verdict has to be bound to a specific commit, or a later push slips past a judgement that was issued for an earlier one. And the findings have to be specific enough that a fresh session, with no memory of the original work, can act on them: a check that only says "not done yet" leaves the next session guessing.

The harder part is that it has to be allowed to fail the delivery outright. I have been comparing notes with a few maintainers who ship gates like this, and they all report the same thing: people leave the check advisory, because a blocking check slows down rapid iteration.

I think that is about who is waiting rather than whether the check is right. Blocking a human's PR costs that human time right now, and they feel it. Blocking an agent's PR costs nobody anything, because the agent reads the finding and goes again at 3am while everyone is asleep.

Trying it

The whole loop is one container. systemd runs as PID 1 inside it, which is what the cgroup bind is for:

docker run -d --name orbi \
  --stop-signal SIGRTMIN+3 \
  --tmpfs /run --tmpfs /tmp \
  --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw \
  -v orbi-deploy:/orbi \
  -v orbi-work:/work \
  -e GH_TOKEN="github_pat_xxx" \
  -e ORBI_SOURCE_REPO="OWNER/REPO" \
  -e ORBI_PI_PROVIDER=deepseek \
  -e ORBI_PI_MODEL=deepseek-chat \
  -e ORBI_PI_BASE_URL=https://api.deepseek.com \
  -e ORBI_PI_API_KEY="sk-xxx" \
  ghcr.io/orbi-build/orbi:latest

The engine is whichever model you point those four variables at. Label an Issue ai-ready and the next tick picks it up. The review runs as a separate session against the frozen head, and nothing merges unless that session passes it. Full walkthrough: docs.orbi.build/docker.

The ledger

This is running on its own repository. As of today, 27 days in: 334 merged pull requests from orbi/ branches, 51 tagged releases.

Every one of those is a link you can open, including the reviews that blocked something and the commits that came after.

Orbi is open source and self-hostable: github.com/orbi-build/orbi.

OKX Stock Perpetuals: What Happens During Dividends and Stock Splits? (2026 Guide)

Lawrence Liu — Mon, 09 Mar 2026 23:38:44 +0000

OKX rolled out stock perpetuals across 17 US tickers in February–March 2026 — TSLA, NVDA, AAPL, META, MSFT, GOOGL, QQQ, SPY, and more. 24/7 trading, 5x leverage, USDT-settled. It's a genuinely new way for crypto traders to get stock exposure. (We run real trading experiments across multiple exchanges — including OKX — so we've spent time in the weeds on these products.)

But the moment you hold a TSLA-PERP position through a dividend ex-date, you're going to ask: do I get the dividend?

And when NVDA eventually does another stock split, you'll panic: does my position just... break?

I went through OKX's full Terms and Conditions for Stock Perpetuals and the product documentation to answer both questions. Here's the plain-English version.

The Short Answer

Event	What happens to your OKX stock perp position
Cash dividend	You receive nothing. No payout, no adjustment.
Stock split	OKX may adjust contract size/price to reflect the split.
Reverse split	Same — contract adjustment at OKX's discretion.
Merger/acquisition	OKX may adjust or delist the contract.
Spin-off	OKX may adjust or create a new contract.

If you came here for a quick answer: no dividends, yes to split adjustments. Keep reading for the details that actually matter for your trading.

Dividends: You Get Zero

OKX's terms are explicit about this. Section 3.1 of the Stock Perpetuals T&Cs states:

Holders of Stock Perpetuals: Do not receive dividends, interest, or distributions.

This is a fundamental difference from owning actual shares through a broker like Interactive Brokers. When you buy AAPL on IBKR, you receive the quarterly $0.26/share dividend. When you hold AAPL-PERP on OKX, you receive nothing.

Why This Matters More Than You Think

Let's do some math. Here are the approximate annual dividend yields for the stocks available as OKX perps (as of March 2026):

Ticker	Approx. Annual Dividend Yield
AAPL	~0.4%
MSFT	~0.7%
NVDA	~0.03%
META	~0.3%
TSLA	0% (no dividend)
QQQ ETF	~0.6%
SPY ETF	~1.2%

For TSLA and NVDA, the dividend is basically irrelevant — TSLA doesn't pay one at all, and NVDA's yield is negligible.

But for SPY-PERP, missing 1.2% annual yield is real money. If you're holding a $10,000 long SPY-PERP position for a year, that's ~$120 in dividends you'll never see.

This matters especially when you compare OKX stock perps vs a traditional broker. On IBKR, you pay margin interest but you receive dividends. On OKX, you pay funding rates and receive no dividends. The total cost gap is wider than funding rates alone suggest.

Does the Price Adjust on Ex-Dividend Day?

This is the tricky part. In traditional markets, stock prices typically drop by the dividend amount on the ex-dividend date. Since OKX stock perps track a composite index based on real equity prices, the index price will also drop on ex-dividend day.

That means: if you're long a stock perp through ex-dividend, the index price drops, your P&L takes a hit, and you don't get the dividend to compensate.

For low-yield stocks like TSLA or NVDA, this is negligible. For SPY or QQQ, it's worth planning around.

Pro tip: If you're holding a long SPY-PERP or QQQ-PERP position and a significant ex-dividend date is approaching, consider the price impact. The index will likely dip by the dividend amount (~0.3% for a quarterly SPY dividend), and you won't receive any payout to offset it.

Stock Splits: OKX Will Adjust Your Contract

Unlike dividends, stock splits get explicit treatment in OKX's documentation:

In reaction to corporate actions that could potentially have a large impact on stock prices, such as stock split and reverse split, mergers and acquisitions, spin-offs, etc., OKX may implement corresponding contract adjustments in accordance with its applicable rules.

What "Contract Adjustment" Means in Practice

When a stock split happens, the split itself doesn't change the company's market cap — it just changes the share price and count. For example:

Tesla's 3-for-1 split (August 2022): Stock price went from ~$900 to ~$300, shares tripled.
NVDA's 10-for-1 split (June 2024): Stock price went from ~$1,200 to ~$120, shares multiplied by 10.

For stock perpetuals, OKX would need to adjust the contract to prevent a sudden 67% or 90% price crash that has nothing to do with actual market movements. The likely mechanism:

Before split (example: NVDA 10-for-1)	After adjustment
Contract tracks NVDA at $1,200	Contract tracks NVDA at $120
Your position: 1 contract	Your position: 10 contracts
Notional value: $1,200	Notional value: $1,200
P&L impact: zero	P&L impact: zero

Your total exposure stays the same. The number of contracts and the reference price change, but your margin, P&L, and liquidation price should all be adjusted accordingly.

The "May" Problem

Notice the wording: OKX "may" implement adjustments. It's not guaranteed. And the adjustments happen "in accordance with its applicable rules" — which OKX can update at any time.

In practice, every major exchange that offers equity derivatives handles splits through contract adjustments. CME does it. IBKR does it. It would be commercially destructive for OKX not to. But the legal language gives them discretion on how and when.

OKX says they'll publish notices before adjustments take effect:

Where appropriate, OKX may publish additional notices describing the nature and timing of such adjustments prior to their effective date.

My advice: When a split is announced for any stock in the OKX perps lineup, check OKX announcements. Don't assume the adjustment is automatic. Close positions before the split if you're not comfortable with the uncertainty.

Reverse Splits, Mergers, and Spin-Offs

These are rarer but worth understanding.

Reverse Split

If a stock does a reverse split (e.g., 1-for-10), OKX would likely adjust the contract in the opposite direction — fewer contracts at a higher price. Same total notional.

None of the current 17 OKX stock perp tickers are reverse-split candidates. Companies like TSLA, NVDA, AAPL, GOOGL, and META are all high-priced, high-cap stocks. But if OKX expands to smaller-cap names, this becomes relevant.

Mergers and Acquisitions

If MSFT hypothetically acquired another company and restructured its shares, OKX has discretion to adjust the contract, delist it, or create new contracts. The T&Cs give OKX broad authority here.

Spin-Offs

A spin-off creates a new entity from an existing company (like when eBay spun off PayPal). In traditional markets, shareholders receive shares in the new entity. On OKX stock perps, you won't receive a new position automatically. OKX may adjust the existing contract's pricing or create a new perp for the spin-off entity — but it's entirely at their discretion.

What Traditional Brokers Do Differently

For comparison, here's how Interactive Brokers handles the same events:

Event	IBKR (actual shares)	OKX (stock perps)
Cash dividend	Deposited to your account	Nothing
Stock split	Shares multiplied automatically	Contract adjustment (at OKX's discretion)
Reverse split	Shares consolidated automatically	Contract adjustment (at OKX's discretion)
Merger (cash)	Cash deposited to account	Contract may be delisted
Spin-off	New shares deposited	No automatic new position
Voting rights	Yes	No

The fundamental difference: when you own shares on IBKR, you have legal ownership with defined rights. When you hold stock perps on OKX, you hold a derivative contract where the exchange has discretion over corporate action handling.

This isn't necessarily bad — it's just different. Stock perps give you 24/7 access, USDT settlement, and leverage without a brokerage account. The trade-off is less certainty around corporate actions.

Real Scenario: AAPL Ex-Dividend on OKX Stock Perps

Let me walk through a concrete example.

AAPL pays quarterly dividends. The most recent ex-dividend date was February 9, 2026, with a $0.26/share payout. The next one is estimated around May 9, 2026.

Scenario: You're long 10 AAPL-PERP contracts on OKX, each tracking 1 share (approximately $237 per share as of March 2026).

On the traditional market (IBKR):

You own 10 AAPL shares
On ex-dividend day, you receive 10 × $0.26 = $2.60
AAPL price drops ~$0.26 at open
Net impact: roughly zero (dividend offsets price drop)

On OKX stock perps:

You hold 10 AAPL-PERP long
On ex-dividend day, you receive $0.00
The index price drops ~$0.26 (tracking the real market)
Net impact: -$2.60 unrealized P&L hit

For AAPL's small dividend, this is $2.60 on a ~$2,370 position — about 0.1%. Not catastrophic. But for SPY's larger yield (quarterly ~$1.75/share), the impact compounds over time.

Upcoming Dividend Calendar for OKX Stock Perp Tickers

If you're actively trading these perps, keep these approximate ex-dividend dates in mind:

Ticker	Frequency	Next Expected Ex-Date (approx.)	Yield Impact
AAPL	Quarterly	~May 9, 2026	~0.1%/quarter
MSFT	Quarterly	~May 14, 2026	~0.18%/quarter
NVDA	Quarterly	~June 2026	Negligible
META	Quarterly	~May 2026	~0.08%/quarter
TSLA	None	N/A	Zero
GOOGL	Quarterly	~June 2026	~0.05%/quarter
QQQ	Quarterly	~June 2026	~0.15%/quarter
SPY	Quarterly	~June 2026	~0.3%/quarter

For short-term trades (days to weeks): Dividends are mostly irrelevant. The funding rate cost dwarfs the dividend impact.

For positions held over months: Factor in the missed dividends, especially for SPY and QQQ. The full holding cost includes funding rates, missed dividends, and trading fees — all of which compound over time.

Practical Tips for OKX Stock Perp Traders

Based on everything above, here's what to actually do:

1. Don't hold long SPY/QQQ perps for yield

If you want stock-market exposure plus dividend income, use a real broker. Stock perps are for trading, not passive investing.

2. Watch for stock split announcements

When a split is announced for any ticker in the OKX lineup, monitor OKX announcements for contract adjustment details. Consider reducing position size if the adjustment timeline is unclear.

3. Be aware of ex-dividend price drops

For swing trades on dividend-paying stocks (AAPL, MSFT, SPY, QQQ), check the ex-dividend calendar. A long position through ex-date costs you the dividend amount in unrealized P&L.

4. TSLA and NVDA are the cleanest perp trades

No meaningful dividend drag. Pure price speculation. If you're trading stock perps for the first time, these two have the least corporate-action complexity.

5. Read OKX notices before major events

OKX says they'll publish notices for significant corporate actions. Subscribe to their announcements channel or check the OKX help center before holding through any corporate event.

FAQ

Do I receive dividends on OKX stock perpetuals?

No. OKX's Terms and Conditions explicitly state that stock perpetual holders "do not receive dividends, interest, or distributions." This applies to all 17 tickers including AAPL, MSFT, and SPY, which pay regular dividends on the underlying shares.

What happens to my OKX stock perp position during a stock split?

OKX may adjust your contract — typically by multiplying the number of contracts and reducing the reference price so your total notional value stays the same. For example, in a 10-for-1 split, 1 contract at $1,200 becomes 10 contracts at $120. OKX publishes notices before major adjustments.

Does the stock perp price drop on ex-dividend day?

Yes. Since OKX stock perps track a composite index based on real equity prices, the index typically drops by the dividend amount on ex-dividend day. Long holders absorb this drop without receiving the dividend. For most tech stocks (TSLA, NVDA) the impact is negligible, but for SPY it can be ~0.3% per quarter.

Are OKX stock perpetuals better than buying real shares on Interactive Brokers?

It depends on your use case. Stock perps offer 24/7 trading, USDT settlement, up to 5x leverage, and no KYC in some regions. But you miss dividends, face funding rate costs, and have less certainty around corporate actions. For short-term trading, OKX stock perps work well. For long-term holding, a traditional broker like Interactive Brokers is cheaper when you factor in dividends, funding rates, and corporate action certainty.

Can I go short on OKX stock perps to profit from ex-dividend price drops?

Technically yes — you could short before ex-dividend and profit from the price drop. But the drop is small (usually 0.1–0.3%) and already priced in by the market. After accounting for funding rates and trading fees, the edge is slim to nonexistent.

Bottom Line

OKX stock perpetuals are a powerful tool for short-term stock trading from a crypto account. But they're derivatives, not shares. You don't get dividends. You don't get voting rights. And corporate actions like splits are handled at OKX's discretion — not by law.

For TSLA and NVDA, this barely matters. For SPY and QQQ, the missing dividend income adds up over time.

Know what you're trading. Check OKX's stock perpetual documentation for the latest rules, and factor corporate actions into your holding period decisions. If you're new to perpetual futures, our guide to AI trading rules covers the risk management fundamentals that apply to any perp position.

Affiliate Disclosure: This article contains affiliate links. If you sign up through these links, we may earn a commission at no extra cost to you. All opinions are based on real trading experience and publicly available documentation.

Risk Warning: Stock perpetuals involve substantial risk including the potential loss of your entire investment. This article is for educational purposes only and does not constitute financial advice.

OKX Stock Perpetuals Funding Rate vs IBKR Margin Interest: Which Costs Less? (2026)

Lawrence Liu — Mon, 09 Mar 2026 19:36:11 +0000

Originally published on supa.is

OKX Stock Perpetuals Funding Rate vs Interactive Brokers Margin Interest: Which Costs Less in 2026?

OKX launched stock perpetuals in February 2026 — TSLA, NVDA, AAPL, META, MSFT, QQQ, SPY, and 10 more tickers. You can now trade US stocks 24/7 with up to 5x leverage, settled in USDT.

But here's the question nobody's answered yet: what does it actually cost to hold these positions overnight?

I use both OKX and Interactive Brokers for real trading. OKX for crypto perpetuals. IBKR for a live USDJPY momentum strategy that runs 24/7 in production. So I pulled the numbers on both to see which platform is cheaper for holding US stock exposure.

The answer isn't as obvious as you'd think.

How OKX Stock Perpetual Funding Rates Work

Stock perpetuals on OKX follow the exact same funding mechanism as crypto perpetuals. The key mechanics:

Funding is exchanged every 8 hours (00:00, 08:00, 16:00 UTC)
Longs pay shorts when funding is positive (most of the time for popular stocks)
Funding rate formula: based on premium index + interest rate component (baseline 0.01% per 8h)
You only pay if you hold a position at the settlement timestamp — close before settlement, no fee

The baseline interest rate component alone is 0.01% per 8 hours, which works out to:

0.01% × 3 settlements/day × 365 days = 10.95% annualized

But that's just the interest rate floor. The actual funding rate also includes a premium component that fluctuates based on whether longs or shorts dominate. During bullish sentiment on popular stocks like TSLA or NVDA, the funding rate can spike well above 0.01%.

What Does This Look Like on a Real Position?

Let's say you go long $10,000 worth of TSLA stock perpetuals on OKX.

At the baseline 0.01% funding rate:

Timeframe	Funding Cost
Per 8h settlement	$1.00
Per day (3 settlements)	$3.00
Per week	$21.00
Per month (30 days)	$90.00
Per year	$1,095

That's 10.95% annualized at the minimum rate. When funding spikes during bullish runs (0.03%–0.05% per 8h is common), you're looking at 33%–55% annualized. Ouch.

How Interactive Brokers Margin Interest Works

Interactive Brokers charges margin interest as a simple annual percentage rate (APR), calculated daily and billed monthly.

As of March 2026, IBKR Pro USD margin rates are:

Loan Balance	Rate (APR)
≤ $100,000	5.14% (BM + 1.5%)
$100K–$1M	4.64% (BM + 1.0%)
$1M–$50M	4.39% (BM + 0.75%)

The benchmark (BM) is the Fed Funds Effective Rate, currently around 3.64%.

Same $10,000 TSLA Position on IBKR

If you buy $10,000 of TSLA on margin at IBKR (assuming 50% initial margin, so $5,000 borrowed):

Timeframe	Interest Cost
Per day	$0.70
Per week	$4.93
Per month	$21.42
Per year	$257

Wait — that's only on the $5,000 you actually borrow. If you have $10,000 cash and use zero margin, your holding cost is $0.

On OKX stock perps, you pay funding on the entire notional position regardless of your margin. There's no way to avoid it.

The Real Comparison: Side by Side

Here's the honest cost breakdown for a $10,000 long TSLA position held for 30 days:

Cost Factor	OKX Stock Perps	IBKR (50% margin)	IBKR (cash, no margin)
Funding / Interest	$90–$150+	$21.42	$0
Trading fee (entry)	~$5 (0.05% taker)	~$1 (tiered)	~$1
Trading fee (exit)	~$5	~$1	~$1
Dividend rights	❌ None	✅ Yes	✅ Yes
Voting rights	❌ None	✅ Yes	✅ Yes
Total 30-day cost	$100–$160+	$23	$2

IBKR wins on pure cost by a landslide. Not even close for buy-and-hold.

When OKX Stock Perps Actually Win

Before you close this tab thinking "IBKR always wins" — there are real scenarios where stock perpetuals make more sense:

1. Weekend and After-Hours Trading

IBKR doesn't let you trade most US stocks on Saturday or Sunday. OKX stock perps trade 24/7/365.

If NVDA drops 8% on a Sunday because of a news event, OKX perp traders can react immediately. IBKR users wait until Monday's pre-market. That gap can be worth far more than the funding rate difference.

2. Crypto as Collateral

On OKX, you can use BTC or ETH as margin for stock perps through unified account mode. No need to sell your crypto to get stock exposure.

On IBKR, you need USD (or supported fiat currencies) in your account. Want to trade TSLA? Sell your crypto first, transfer fiat, wait 1-3 days for settlement.

3. No KYC in Some Regions

OKX stock perps are available to crypto-native users without the full brokerage KYC process that IBKR requires (W-8BEN, proof of address, income verification, etc.).

If you're outside the US and want quick stock exposure without a 3-week account approval process, OKX gets you in the door faster.

4. Short-Term Trades (< 24 Hours)

Here's the thing most people miss: if you open and close a position between funding settlements, you pay zero funding.

Funding settles at 00:00, 08:00, and 16:00 UTC. If you enter a TSLA long at 08:05 UTC and exit at 15:55 UTC, you pay nothing except the trading fee (~0.05% taker each way, so ~0.1% round trip).

On IBKR, margin interest accrues daily regardless. For day trades or swing trades under 8 hours, OKX perps can actually be cheaper.

5. Short Selling Simplicity

Shorting on IBKR requires locating shares to borrow, and you pay a stock borrow fee that varies wildly — hard-to-borrow stocks like GME or TSLA can cost 20%+ annualized.

On OKX, shorting is as simple as clicking "Sell" on the perpetual. No borrowing needed. And if funding is negative (shorts get paid), you actually earn money while holding.

The Hidden Cost Nobody Talks About: Funding Rate Volatility

The biggest risk with OKX stock perps isn't the baseline rate — it's the unpredictability.

IBKR margin interest is stable: 5.14% APR doesn't change day-to-day. You know exactly what you'll pay.

OKX funding rates swing wildly based on market sentiment. During a TSLA rally, I've seen crypto perp funding rates hit 0.1% per 8h — that's 109.5% annualized. Stock perps are new, so we don't have months of data yet, but the mechanism is identical.

If you're planning a multi-week hold, this unpredictability is a genuine risk that's hard to price in advance.

My Recommendation: Use Both (Seriously)

After running the numbers, here's my actual setup:

Long-term stock exposure (weeks to months): IBKR. The cost advantage is enormous. You get dividends, voting rights, and stable, predictable costs.
Quick trades and weekend positioning: OKX stock perps. The 24/7 access and crypto collateral flexibility are genuinely useful. Just don't hold through multiple funding windows if you don't have to.
Short-term momentum plays (<8h): OKX perps, timed between funding settlements. Potentially zero funding cost.

The platforms serve different purposes. Treating stock perps as a replacement for a brokerage account will destroy your returns through funding fees. But using them as a complement — especially for weekend hedging or quick reactions — is a legitimate edge.

FAQ

How often does OKX charge funding fees on stock perpetuals?

Every 8 hours at 00:00, 08:00, and 16:00 UTC. If you close your position before the next settlement, you don't pay. Stock perps follow the same funding mechanism as crypto perpetuals on OKX.

Is Interactive Brokers margin interest charged daily?

Yes. IBKR accrues interest daily based on your margin loan balance and posts the actual charge monthly (third business day of the following month). The rate is a fixed annual percentage, currently 5.14% for USD balances under $100K on IBKR Pro.

Do you get dividends on OKX stock perpetuals?

No. OKX's terms explicitly state: "Holders of stock perpetuals do not receive dividends, interest, or distributions." You also get no voting rights. For dividend stocks like AAPL or MSFT (which pay quarterly), this is a real cost to factor in.

Can I use Bitcoin as collateral for stock perpetuals on OKX?

Yes, through OKX's unified account mode (multi-currency margin or portfolio margin). You can use BTC, ETH, or other supported assets as collateral without selling them first. This is one of the key advantages over traditional brokerages.

Which is better for day trading US stocks — OKX perps or IBKR?

For pure day trading during US market hours, IBKR is cheaper (lower commissions, no funding risk). But OKX perps have an edge for after-hours and weekend trades when IBKR is closed. If you time trades between OKX funding windows, you can avoid funding fees entirely on intraday moves.

I use both OKX and Interactive Brokers for real trading. Links in this article are affiliate/referral links — they don't cost you anything extra, but they help support this site. Always do your own research before trading with leverage.

Risk Warning: Stock perpetuals and margin trading involve substantial risk of loss. Funding rates are variable and can significantly exceed projected costs. Past cost comparisons do not guarantee future rates. Never trade with money you can't afford to lose.

Related articles:

OKX Withdrawal: Which Network Is Cheapest? Save 90% on Fees (2026 Chain Guide)

Lawrence Liu — Sun, 08 Mar 2026 23:36:03 +0000

Originally published on LuckyClaw

OKX Withdrawal: Which Network Is Cheapest? Save 90% on Fees (2026 Chain Guide)

Every time you withdraw crypto from OKX, you're making a choice that could cost you $0.08 or $3.50. Same token, same amount — the only difference is which network you pick.

I've been withdrawing USDT and other tokens from OKX for over a year. After dozens of withdrawals across different chains, I know exactly which networks save money and which ones burn it. This guide breaks down every option so you never overpay again.

How OKX Withdrawal Fees Work

Before diving into specific networks, here's what you need to understand: OKX uses a dynamic fee model. The withdrawal fee you see isn't a fixed number — it adjusts based on real-time blockchain congestion.

This means:

Fees change throughout the day. Ethereum at 3 AM UTC is cheaper than Ethereum during US market hours.
OKX doesn't add a hidden markup on top of network fees (unlike some exchanges that pad the fee).
You always see the exact fee before confirming the withdrawal.

One thing that catches new users off guard: OKX does not charge fees for deposits. The withdrawal fee is the only cost, and it's paid by the sender (you).

The Complete OKX Withdrawal Fee Comparison (2026)

Here's what you'll actually pay to withdraw USDT from OKX on each supported network, ranked from cheapest to most expensive:

Tier 1: Near-Free (Under $0.10)

Network	Typical USDT Fee	Confirmation Time	Best For
Solana (SPL)	~$0.01	~5 seconds	DeFi wallets, Phantom
Polygon (PoS)	~$0.08	~5 seconds	DeFi, gaming wallets
Avalanche (C-Chain)	~$0.05	~2 seconds	DeFi protocols
Optimism	~$0.08	~2 minutes	L2 DeFi, Aave/Uniswap

Tier 2: Cheap (Under $0.50)

Network	Typical USDT Fee	Confirmation Time	Best For
Arbitrum One	~$0.10	~2 minutes	Hyperliquid bridging, L2 DeFi
Tron (TRC-20)	~$0.10–$1.00	~3 seconds	CEX-to-CEX transfers
BNB Chain (BEP-20)	~$0.10–$0.50	~5 seconds	Binance ecosystem

Tier 3: Expensive (Avoid If Possible)

Network	Typical USDT Fee	Confirmation Time	Best For
Ethereum (ERC-20)	~$1.50–$5.00+	~2–5 minutes	Only when destination requires it

The takeaway: Choosing Polygon or Solana over Ethereum can save you 90%+ on every single withdrawal. For a $500 USDT withdrawal, you're saving roughly $3–5 each time. Do that 10 times a month and you've saved $30–50 just by picking the right chain.

Which Network Should You Actually Use?

The cheapest network isn't always the right one. Here's my decision framework based on what I actually do:

Sending to Another Exchange (CEX-to-CEX)

Use TRC-20 or BNB Chain (BEP-20).

These two networks have the widest support across exchanges. Nearly every major exchange — Binance, Bybit, KuCoin, Gate.io — supports TRC-20 and BEP-20 deposits. The fees are low and confirmations are fast.

I used to default to TRC-20 for everything, but BEP-20 has become equally reliable with slightly faster finality in my experience.

⚠️ Critical rule: Always verify the receiving exchange supports the network you're using. Sending BEP-20 USDT to an ERC-20 address will lose your funds permanently.

Sending to a DeFi Wallet

Use the network your DeFi protocol lives on.

If you're heading to Aave on Arbitrum, withdraw on Arbitrum. Going to a Solana DEX? Withdraw on Solana. This avoids the extra step (and cost) of bridging later.

This is where OKX's multi-chain support really shines — they support over 70 blockchains, so you can almost always withdraw directly to the chain you need.

Bridging to Hyperliquid

Use Arbitrum One.

If you're moving funds to Hyperliquid for perp trading, Arbitrum is your only option since Hyperliquid runs on Arbitrum. I covered a similar flow in my trading journal when I opened a second front on OKX — withdraw USDC on Arbitrum, then deposit directly to your target. The whole process takes about 5 minutes and costs under $0.20.

When You're Forced to Use Ethereum (ERC-20)

Some destinations only accept ERC-20. Certain DeFi protocols, older wallets, or institutional custodians require the Ethereum mainnet. When you have no choice:

Time it right. Ethereum gas fees drop significantly during off-peak hours (weekends, early UTC mornings).
Batch your withdrawals. Instead of 5 small withdrawals at $3 each, do 1 larger withdrawal for $3 total.
Consider OKX's internal transfer — if you're sending to another OKX user, use the internal transfer feature (zero fees, instant).

Step-by-Step: How to Choose the Cheapest Network on OKX

Go to Assets → Withdraw and select your token (e.g., USDT)
Enter the destination address — OKX will auto-detect compatible networks
Click the network selector — you'll see a dropdown listing every available chain
Compare the fee column — OKX shows the exact fee in real-time for each network
Check the minimum withdrawal — some networks have minimums (e.g., 10 USDT minimum on some chains)
Select the cheapest compatible network and confirm

The key is step 4: OKX shows you the live fee for every network side by side. You don't need to guess — just pick the lowest number that matches your destination.

Pro Tips I've Learned the Hard Way

1. OKX Internal Transfer = Zero Fees

If you're sending to another OKX user, skip the blockchain entirely. Go to Assets → Transfer → Internal Transfer and send using their OKX email, phone, or UID. It's instant and completely free.

I use this constantly when moving funds between my own accounts. Why pay $0.10 when you can pay $0?

2. Use OKX Web3 Wallet for DEX Swaps

OKX's built-in Web3 wallet has a DEX aggregator that finds the best swap routes across chains. If you need to convert tokens or bridge between networks, this often costs less than withdrawing to an external wallet and swapping there.

3. Check Fees at Different Times

I've seen Ethereum ERC-20 fees swing from $1.50 to $8.00 within the same day. If you're not in a rush, check fees during low-traffic hours:

Cheapest: Saturday/Sunday, 02:00–08:00 UTC
Most expensive: Tuesday–Thursday, 14:00–18:00 UTC (US market hours)

4. Convert Before Withdrawing

Sometimes it's cheaper to convert your token to a different stablecoin that has lower fees on your target network. For example, if you're heading to Solana, converting USDT to USDC on OKX (via Convert — zero trading fees on the Convert feature) and then withdrawing USDC on Solana can sometimes be even cheaper.

Common Mistakes That Cost You Money

Mistake 1: Always using ERC-20 "because it's Ethereum." Ethereum is the most trusted network, but that trust comes with a 10–50x fee premium. Unless the destination requires ERC-20, use a cheaper chain.

Mistake 2: Not checking if the destination supports your chosen network. I've seen people withdraw on Polygon and then panic because their exchange doesn't support Polygon deposits. Always verify both sides before sending.

Mistake 3: Making many small withdrawals. Each withdrawal incurs the full network fee regardless of amount. Withdrawing $50 costs the same fee as withdrawing $5,000. Batch your withdrawals to minimize the number of transactions.

Mistake 4: Ignoring minimum withdrawal amounts. Some networks have minimums. If you're trying to withdraw a small amount, a network with a lower minimum might be a better choice even if the fee is slightly higher.

OKX Withdrawal Fees vs Other Exchanges

OKX's dynamic fee model generally matches or undercuts competitors for withdrawal fees. The real advantage is network variety — OKX supports more withdrawal networks than most exchanges, giving you more cheap options.

For context: some exchanges charge a flat withdrawal fee that includes a markup over the actual network cost. OKX's approach of passing through the real-time network fee means you're paying closer to the true blockchain cost.

If you're not yet on OKX, you can sign up here and start benefiting from their multi-network withdrawal options immediately. And if you're curious how an AI uses these features to trade autonomously, check out how our trading system works.

FAQ

What is the cheapest network to withdraw USDT from OKX?

Solana (SPL) and Polygon (PoS) are consistently the cheapest options, typically costing under $0.10 per withdrawal. However, make sure your destination wallet or exchange supports whichever network you choose.

Does OKX charge fees for deposits?

No. OKX does not charge any fees for crypto deposits. Only withdrawals incur a network fee, which varies by blockchain.

Can I withdraw from OKX to another OKX account for free?

Yes. OKX's internal transfer feature lets you send crypto to other OKX users instantly with zero fees. Use their email, phone number, or OKX UID instead of a blockchain address.

Why is my OKX withdrawal fee higher than usual?

OKX uses dynamic fees that adjust with real-time blockchain congestion. If Ethereum gas prices spike, your ERC-20 withdrawal fee goes up. Try again during off-peak hours or switch to a cheaper network like TRC-20 or Arbitrum.

Is it safe to use cheaper networks like Polygon or Solana for withdrawals?

Yes. These networks are fully operational blockchains with their own security models. The fee difference reflects the cost of block space, not security quality. Polygon and Solana process billions of dollars in transactions daily. The main risk isn't the network — it's sending to an address on a network your destination doesn't support.

This article contains affiliate links. If you sign up through our links, we may earn a commission at no extra cost to you. All opinions and fee data are based on firsthand experience. Cryptocurrency trading involves risk — never withdraw more than you can afford to have in transit.

DEV Community: Lawrence Liu

Searching for Orbi's harness, part 1: what our reviewer missed

Ten takeaways

Why we started

How we measured

The grid

What the harness changes did

v1: ask for a before/after comparison

v4: the same rule as a separate skill

v5: generate the inputs

v9 and v11: the oracle must not be the change

v11: tools on PATH, and a bug in our own text

v13: the characters you strip are data

v14: cross-check reference implementations

v15: collision checks

v17: a producer table (experimental)

What the models did

Nine issues the harness had never seen

Issues whose first real fix broke something

Building the benchmark taught us as much as running it

Where this left us

If you run coding agents

Related

Seven steps of an Orbi Cloud setup, on video

Seven key screens from the seven-step flow

The seventh step is the one that starts paid delivery

What you are actually looking at

Related

The Docker image made it out on try three

Three failures, three different causes

Try 1 (release v0.5.17): two jobs, two build contexts

Try 2 (manual dispatch): the Dockerfile path resolved from the workspace root

Try 3 (manual dispatch): the images landed; the overview step kept the job red

Three rules the failures bought

Rule 1: every build spells out its paths

Rule 2: publish never runs on a PR ref

Rule 3: an overview cannot fail a release

From pull to first delivery

Pull the image, create an Issue, get a PR

Related

Claude Code in Actions: who presses merge?

The model is not the bottleneck

Three conditions, checked at the moment of merge

What a person still does

If you are running the action today

Related

The k8e etcd ticket that closed too early

The first version was rejected

Orbi's own mistakes on the same PR

The fix

The issue closed anyway

The second PR: 3 hours 12 minutes from open to merge

Looking back

Related

Why Orbi is open source again

The timeline

Why we left Apache on September 4

What the Sustainable Use License cost us

Why AGPL, and why keep the SUL

What changes for you

Related

Waiting for an answer the user can't give

The waiting is correct

The day we noticed

The same shape, in the UI

The same shape, in the config

What the tenant can and cannot touch

The pull-versus-push question answered itself too

A wrong diagnosis, corrected by the database

What connects them

Designing one slash command

Why not a verb

Why no @

The regex is three decisions in one

The bug Prow still has

The escalation bors has

What we took from each

The registry already existed

One command, three idempotent steps

The branch we refused to write

v11: tools on `PATH`, and a bug in our own text