A 75% improvement is compelling, but I would publish the p95 and the cost of the loop alongside the headline number. Automated optimization can trade client latency for build time, regressions, or infra spend; a small before-and-after budget for those dimensions makes the result easier to reproduce and trust.
I'd ballpark additional spend on this at ~$180/mo on top of subscriptions we already were paying for.
The openhands instance and all its computer is about $100; our GitHub Actions spend is free under GitHub Open Source policy; and this is all re-using subscription credits we pay for otherwise. Then I'll add an extra $80 for certain misc spend along the way.
There's an inherent efficiency in that we use subsidized subscription tokens instead of API tokens. that benefit may eventually go away.
There are definitely people doing this sort of thing incredibly inefficiently, but not us. We're definitely frugal with this stuff. 😄
That breakdown makes the trade-off concrete. The subscription-credit advantage is useful while it exists, but a durable report should separate marginal cash cost from consumed included capacity and show the fallback price if that subsidy changes. Otherwise the system can look cheaper than it is precisely when it needs to scale.
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
70ms to 16ms, and P99 over a second down to something sane — that's the kind of boring work nobody prioritizes until it hurts. The 'API endpoint with a pulse' line is painfully accurate though, I spent weeks in exactly that spot, clipboard between dashboards and terminal. Genuine question: of the 475 PRs, how many were the loop finding real issues vs just churn? The garden metaphor only works if the soil holds.
I'd say 99% real issues, but the loop consistently hit local maximas where the improvements started being really small, so that forces tweaks to the system to re-evaluate the instructions on how to pursure finding the issues.
It could be possible to do higher level instructions so this didn't have to be tweaked, but as a matter of safety I think it's better to keep the loops highly instructed.
Wow just wow, that's Engineering with a big E - for sure you guys always did great work, but this sounds like "next level" ...
Never heard of OpenHands, and I'm sure I'm nowhere near the "AI maturity level" which would make it make sense for me, but I've bookmarked it 'for the future'!
In a way I'd say this is the project that made me and our team get to "next level" in the first place. I'd definitely give it a try. There's also several products in the OpenHands space— I really liked that this one is open source and self-hostable.
This way of doing software dev and managing the SDLC definitely sounds like "the future" of development - if you have a good grasp of how to do this (which you pretty clearly achieved with this project) then you're in a good place, also in the medium/long term ...
OpenHands looks interesting, although I'm far from an expert, or even moderately knowledgeable in this space - just getting my feet wet and taking baby steps with Claude Code, but I can imagine that the openness and extensibility give OpenHand the edge (especially for bigger/more complex projects) over a more "closed" environment like Claude Code ...
But for me OpenHands is probably something to try out in the future, when I'm a bit further along this path ...
Hmmm could be an issue with our image CDN. Could you open an issue on github.com/forem/forem with all possible details you're seeing from console (this likely isn't a fix in our code, but maybe config— but still best place to put a bug)
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
The 70ms to 16ms arc is wild, but the bit I keep thinking about is the rust-loop being gated to one open PR so the precompiled binaries never conflict. Did that gate ever become the bottleneck once the easy SQL wins cleared, or did the 10-minute tick cadence keep it fed just slowly enough? Also "human middleware" and "an API endpoint with a pulse" might be the most accurate job description of agentic development I've read this year.
A 75% improvement is compelling, but I would publish the p95 and the cost of the loop alongside the headline number. Automated optimization can trade client latency for build time, regressions, or infrastructure spend; a small before-and-after budget for those dimensions makes the result easier to reproduce and trust.
The reason this works where most autonomous agent setups stall is the shape of the verification function. When the success criterion is subjective or requires reading code for subtle intent, human review time scales linearly with PR volume, which turns the engineer into a bottleneck. By anchoring the loop to live telemetry and automated test suites, the evaluation becomes an objective numeric check. That drives marginal verification cost low enough that merging 350 autonomous PRs a month actually frees up human attention instead of consuming it.
The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
We're a place where coders share, stay up-to-date and grow their careers.
A 75% improvement is compelling, but I would publish the p95 and the cost of the loop alongside the headline number. Automated optimization can trade client latency for build time, regressions, or infra spend; a small before-and-after budget for those dimensions makes the result easier to reproduce and trust.
I'd ballpark additional spend on this at ~$180/mo on top of subscriptions we already were paying for.
The openhands instance and all its computer is about $100; our GitHub Actions spend is free under GitHub Open Source policy; and this is all re-using subscription credits we pay for otherwise. Then I'll add an extra $80 for certain misc spend along the way.
There's an inherent efficiency in that we use subsidized subscription tokens instead of API tokens. that benefit may eventually go away.
There are definitely people doing this sort of thing incredibly inefficiently, but not us. We're definitely frugal with this stuff. 😄
That breakdown makes the trade-off concrete. The subscription-credit advantage is useful while it exists, but a durable report should separate marginal cash cost from consumed included capacity and show the fallback price if that subsidy changes. Otherwise the system can look cheaper than it is precisely when it needs to scale.
70ms to 16ms, and P99 over a second down to something sane — that's the kind of boring work nobody prioritizes until it hurts. The 'API endpoint with a pulse' line is painfully accurate though, I spent weeks in exactly that spot, clipboard between dashboards and terminal. Genuine question: of the 475 PRs, how many were the loop finding real issues vs just churn? The garden metaphor only works if the soil holds.
I'd say 99% real issues, but the loop consistently hit local maximas where the improvements started being really small, so that forces tweaks to the system to re-evaluate the instructions on how to pursure finding the issues.
It could be possible to do higher level instructions so this didn't have to be tweaked, but as a matter of safety I think it's better to keep the loops highly instructed.
Wow just wow, that's Engineering with a big E - for sure you guys always did great work, but this sounds like "next level" ...
Never heard of OpenHands, and I'm sure I'm nowhere near the "AI maturity level" which would make it make sense for me, but I've bookmarked it 'for the future'!
In a way I'd say this is the project that made me and our team get to "next level" in the first place. I'd definitely give it a try. There's also several products in the OpenHands space— I really liked that this one is open source and self-hostable.
This way of doing software dev and managing the SDLC definitely sounds like "the future" of development - if you have a good grasp of how to do this (which you pretty clearly achieved with this project) then you're in a good place, also in the medium/long term ...
OpenHands looks interesting, although I'm far from an expert, or even moderately knowledgeable in this space - just getting my feet wet and taking baby steps with Claude Code, but I can imagine that the openness and extensibility give OpenHand the edge (especially for bigger/more complex projects) over a more "closed" environment like Claude Code ...
But for me OpenHands is probably something to try out in the future, when I'm a bit further along this path ...
i dong know if its only me or...
the loading never ends... the text ,... really fast...
my connection :
different browser still the same ....
^_^
Hmmm could be an issue with our image CDN. Could you open an issue on github.com/forem/forem with all possible details you're seeing from console (this likely isn't a fix in our code, but maybe config— but still best place to put a bug)
The 70ms to 16ms arc is wild, but the bit I keep thinking about is the rust-loop being gated to one open PR so the precompiled binaries never conflict. Did that gate ever become the bottleneck once the easy SQL wins cleared, or did the 10-minute tick cadence keep it fed just slowly enough? Also "human middleware" and "an API endpoint with a pulse" might be the most accurate job description of agentic development I've read this year.
A 75% improvement is compelling, but I would publish the p95 and the cost of the loop alongside the headline number. Automated optimization can trade client latency for build time, regressions, or infrastructure spend; a small before-and-after budget for those dimensions makes the result easier to reproduce and trust.
The reason this works where most autonomous agent setups stall is the shape of the verification function. When the success criterion is subjective or requires reading code for subtle intent, human review time scales linearly with PR volume, which turns the engineer into a bottleneck. By anchoring the loop to live telemetry and automated test suites, the evaluation becomes an objective numeric check. That drives marginal verification cost low enough that merging 350 autonomous PRs a month actually frees up human attention instead of consuming it.
This is great work Dev team. Which loop PR gave the biggest single drop??
The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.