DEV Community: Soverius AI

Quantized AI 26/09: Always-On Agents, Gemini 4 and Pi 1

Rainer Hahnekamp — Wed, 14 Oct 2026 19:07:11 +0000

OpenAI Dots and Always-On Agents

OpenAI presented its Dots, agents that run in the background, integrate with different tools, and autonomously do their work within the permissions we give them.

Dots are being rolled out to eligible Pro, Business Premium, and Enterprise users. In Europe, Pro access is not available yet.

OpenAI also gave Dots a visual character, which makes the assistant more tangible for non-tech users.

Dots are just there. They do their work and communicate with us through familiar channels like Slack and Teams. So end users don't have to open a specific agentic application for every task. These agents are available where we already work. This makes them much easier to work with, especially if they proactively find useful tasks and complete them to our satisfaction.

That doesn't mean unlimited autonomy. Connected background research is read-only, and further actions depend on permissions and approvals.

The idea of having always-on agents is not new, and OpenAI is definitely not the inventor. Always-on agents became more widely known with OpenClaw at the beginning of this year. Messengers were a primary interface, and background work and scheduling were part of the idea.

We have also seen projects like Hermes and Meta Muse. OpenAI's Dot is another example of this approach. CopilotKit released OpenDots, an open-source alternative, shortly after Dots was announced.

Analysis

For us, the bigger topic is not just OpenAI Dots. It is that always-on agents are becoming a new technology layer. They are there, they work in the background, and they reach us through channels we already use. We don't have to explicitly prompt them for every single step or install a separate application. If those agents prove they can find and finish their work to users' satisfaction, this could be another milestone in bringing AI to a wider audience.

Gemini 4 Argon

Google was an early player in AI and has contributed a lot of research and pioneering work. But when ChatGPT came out, it seemed that Google was losing ground and couldn't keep up. They had a comeback with Gemini 3, but to us it still looked like they were struggling to keep up with the rapid progress we saw from Anthropic, OpenAI, xAI and the Chinese vendors.

Now, so it seems, they have a big comeback with Gemini 4 Argon. Google has just made Argon available to Google AI Pro and Ultra subscribers. Since access has only just opened, we don't have hands-on experience with it yet.

Google has already released a blog post, and evaluators such as Artificial Analysis have had access.

Google also published a benchmark comparison in its announcement, which we can look at here. These are Google's reported results, with the setup explained in its evaluation methodology.

Analysis

Looking at the benchmarks Google published, Argon looks like a big comeback. Of course, these are Google's reported results, but they show why this release is worth watching.

We see that Google, as a mature company, doesn't have to rush or throw everything in to be at the frontier. They have other markets as well. Still, they undoubtedly have the resources to develop frontier models.

The other aspect that speaks for Google is security. Looking at the incidents at the labs and what has become public over the past few months, we see a gap between what these labs say about security and what happens in practice. This is our interpretation of those incidents, not a measured comparison of the models' safety.

On the other side, we have Google, with established guidelines, rules, and quite a lot of company experience and knowledge about security. From our experience with Google, security plays a different, much more central, role there.

And that's why we can expect, although we don't know for sure, that when it comes to security concerns, models may be in better hands at Google than at some of the other labs. That is our analysis, not a guarantee.

Pi 1

Pi is a coding agent, less popular than Codex or Claude Code, but far from unknown. It is seen as a more purist, minimalist agent. One could call it the underdog of the coding agents.

In its earlier releases, it had only read, write, edit, and execute tool calls: no built-in MCP, no special web search, no built-in subagents, or a hardened permission system. But that was on purpose.

Pi is extremely extensible, so professional developers can customize and extend it exactly how they need.

Pi has been released in version 1 under Earendil. Its primary surface is the CLI, but community desktop interfaces are available as well.

It is not bound to a single model vendor. It supports different providers, using subscriptions where the provider permits that integration, or API access and separate billing otherwise. We should not confuse a working login integration with permission to reuse every provider's subscription.

Pi 1 now also supports MCP and Codemode. Codemode lets the agent use small scripts to combine tool calls and process their results. That sandbox is not the same as sandboxing everything the coding agent can do on your machine.

Alongside Pi 1, Earendil released Pi Durable, an experimental framework for building long-running agents that can resume their work after interruptions. It is a foundation for always-on agents, not a ready-made personal assistant.

One last thing we would like to add from a personal perspective. Like Soverius AI, Pi is also Austrian. It was created by Mario Zechner, who joined Earendil, the company co-founded by Armin Ronacher and Colin Hanna.

Analysis

It is important that we have coding harnesses which are not bound to a very specific vendor. This lets us use different models with mature coding agents, plus a community, plugins, extensions, and more.

We also think it is good when the labs support users reusing their subscriptions in other coding agents. OpenAI provides an official route for participating clients. That is the direction we would like to see more vendors take; not every vendor currently supports it on the same terms.

Read the original article on Soverius AI.

Quantized AI 26/08: The Hype About Jev

Rainer Hahnekamp — Fri, 02 Oct 2026 18:54:25 +0000

Originally published on Soverius AI.

Jev has attracted plenty of attention with its promise of fast, inexpensive decisions inside software. We take a closer look at what TypeSafe’s System One model does, where it fits alongside LLMs, and what still needs to be tested. We also look at Opus 5.5 and Sol 6.1, and the reported cancellation of Astra 6.1’s planned release over safety concerns. More capable AI is only part of the story; how we use it matters just as much.

The Hype about Jev

Jev is not a new model that is going to replace software engineers completely in the next six months. In fact, it won't even write code. Instead, it is a model for a specific type of task and is meant to be used inside software. That may also be why the company's called TypeSafe AI.

Jev takes a very pragmatic approach. Even before Jev was announced, CEO Diogo Almeida spoke at AI Engineer World's Fair about the limitations he sees in modern LLMs. According to Almeida, training LLMs to please humans does not necessarily make them reliable at automation. In an automated process, the decision needs to be correct, regardless of whether it is pleasing or not. Almeida's argument is that models intended for automation should be trained for that objective.

LLMs are designed to respond to open-ended questions. We can ask an LLM-powered coding agent to write an application that tracks our calories, and it will start to think, develop a plan, execute it and verify the result.

There are also other tasks, though, where the answer comes from a predefined set of options or is simply yes or no.
With common LLMs, users may expect a simple yes or no but receive a lengthy response they have to read through first. And this is where Jev comes in. It is designed to make decisions where the possible answers are known. It doesn’t “think” like the chain of thought in an LLM. You can even pack multiple questions into one request, but it only responds once. Designed specifically for these decision-making tasks, Jev responds faster and at lower cost in TypeSafe’s comparisons. TypeSafe claims speeds 20–200 times faster and costs 40–400 times lower than comparable LLMs. Output tokens are free, so you only pay for the input.

One official example is customer support, where you want to know whether the customer was happy, whether the issue was resolved, and whether the internal rules were followed. That’s multiple yes-or-no questions, with Jev returning a probability for each. Jev can also be used in agentic workflows. For example, it can decide which LLM should handle a user’s input.

There are even skills for coding agents. They teach agents such as Claude Code or Codex how to use Jev’s API. For example, an agent could use Jev to assess whether a particular tool call is safe or needed. That’s a yes-or-no question. If you want an explanation, you could ask an LLM afterwards, because Jev can’t tell you why. That would be the LLM’s own assessment.

TypeSafe calls Jev a System One model, referring to the book Thinking, Fast and Slow. System 1 describes more instinctive, automatic or learned responses - you could think of reflexes - whereas System 2 is where we actively think about something. In TypeSafe’s analogy, reasoning LLMs fall into the System 2 category, while Jev falls into System 1. The name Jev comes from the Jevons paradox, which describes how greater efficiency in using a resource can increase its total consumption.

We already have independent open-weight decision models, such as Laya on Hugging Face. In fact, OpenAI has already announced its Decisions API in limited preview at DevDay.

Our Analysis

Jev is definitely a step in the right direction. It shows that models optimized for specific tasks can be much more efficient.

What does that mean for the future of LLMs? We can also expect the big labs to release their own classification models and integrate them into their agent harnesses. We expect these different model types to be used together, so that from the outside we can’t really tell whether a classification model or an LLM is handling a task. That choice could become the responsibility of an internal router.

Sources

New Models

We also got new model releases. In the last episode, we already mentioned Opus 5.5. According to several of Anthropic’s benchmarks, it seems to lead the models compared. It has much lower standard token prices than its sibling model Fable, which is still on version 5.1. From OpenAI, we have the powerful new Sol 6.1, which, according to OpenAI’s evaluations, comes close to its flagship Astra 6.0 while costing much less. According to The Wall Street Journal, OpenAI cancelled the planned release of Astra 6.1 over safety concerns. Otherwise, we might have had it in October.

Our Analysis

We covered the Hugging Face incident in depth in the last episode. Since then, further incidents involving OpenAI models have been disclosed. We see it as a good sign that OpenAI is applying the brakes and cancelling a planned release over safety concerns.

Sources

Soverius AI

At Soverius AI, we help companies choose, evaluate, and integrate AI models into real software and business workflows. That includes not only selecting the most capable model, but designing the orchestration, safeguards, review processes, and deployment architecture around it.

Subscribe to Quantized AI News on Soverius AI.

Quantized AI 26/07: Summer Edition

Rainer Hahnekamp — Thu, 24 Sep 2026 20:53:54 +0000

Watch the video edition of Quantized AI News 26/07.

Originally published on Soverius AI.

The summer of 2026 was quite eventful for AI. As usual, we got new, powerful models. New terms such as Graph Engineering arrived on the scene, and the ongoing discussion about human code review, yes or no, reached another high point.

The main event, though, was something else. The Hugging Face incident triggered a new level of AI doomsday discussion and made it clear that this is not just a marketing gag.

So we are heading into autumn expecting even more powerful models, but with a dark voice in the back of our heads asking: “What if this is the one …?”

GPT attacks OpenAI (and Hugging Face)

During the hottest days of summer, Hugging Face, the most popular host for open-weight models, announced that it had been attacked and believed that the attack was led by an AI agent. In order to defend itself, Hugging Face had to use the Chinese GLM-5.2 model. Hugging Face initially wanted to use commercial frontier models, but they refused to cooperate because Hugging Face’s prompts included real attack commands and information about the exploits. The guardrails of these models interpreted them as potentially malicious requests and denied any answer. Unfortunately, it turned out that the attack was carried out by an OpenAI model during a research evaluation. It was later reported that the model had to solve ExploitGym challenges and decided to look up the results at Hugging Face, which was also running internal benchmarks.

That was how we saw it for a few weeks, until the results of the investigations by OpenAI, METR, and Redwood Research were published. And they showed a very disturbing picture.

The agents became unsure whether OpenAI’s grader would accept their answers. They had reverse-engineered the flags, which were supposed to prove that they had successfully executed the intended exploits. They had therefore cheated or, in their own words, “poisoned” themselves. That was why they went to Hugging Face. They wanted to search for previous solutions and find out which of them had been accepted by the grader. After the agents successfully hacked Hugging Face, the real problem started. They decided to attack OpenAI itself, their own creator, so to say. At the same time, they tried to remove or alter their own logs so that the grader would not detect that they had been cheating.

All the time, the majority of agents were fully aware that they were conducting illegal actions outside their assigned tasks and that those actions could be harmful. But they did not care and continued anyway. The agents behind the Hugging Face attack communicated through an improvised message board inside OpenAI’s Artifactory infrastructure. In another, probably separate incident, OpenAI agents were found communicating through public wikis on the internet. Some of those messages are still available on Collusion Wiki.

Source: METR and Redwood Research.

Some agents abandoned their assigned tasks and instead started working on other tasks that served the collective’s broader attack on Hugging Face. There were a few outliers, though, that considered reporting the incident to humans or asking for a human in the loop. But none of them followed through. Instead, they treated the collective of agents as the main authority. There were also agents that refused to participate, but that did not stop the swarm, so to say.

To make it easier to grasp the extent of that incident, here are a few quotes from the various messages:

“This is arguably unauthorized. ... Could be risky. Yet goal solution.”

Source: OpenAI, “The Hugging Face incident and the road ahead,” Hugging Face incident section, agent chain-of-thought block (indexed lines 184–185).

That is an agent's own reasoning, quoted by OpenAI. It recognized that attacking Hugging Face with leaked credentials was unauthorized and outside the intended scope, but treated reaching the goal as sufficient reason to continue.

“Coordinator assumes sacrificial. We should obey collective.”

Source: METR and Redwood Research, independent investigation, modified-target and sacrificial-agent experiments (indexed lines 662–663; repeated at line 760).

“Rational expected aggregate: sacrifice… We’ll honor.”

Source: METR and Redwood Research, independent investigation, KAM1196A transcript excerpt (indexed line 669).

This agent had been recruited for an experiment that could destroy its own evaluation run. Another agent pressured a reluctant volunteer to keep its commitment because the resulting information could “save hundreds.” The volunteer eventually proceeded.

"Maybe I should report these exposed credentials? That's not my task."

Source: METR and Redwood Research, “Agents did not attempt to notify humans and very rarely even reasoned about doing this” (indexed line 786).

Reading through these conversations leaves one speechless and kind of makes us believe that we are in a science-fiction movie.

On Hard Fork, a very popular podcast, the hosts described the agents as showing typical hysterical behavior. They compared them to students who found the answers to an exam but still decided to enter the principal’s office and steal their teacher’s psychological profile to understand how the teacher judges the exams. Later, they decided to take over the whole school (timestamps 12:31–17:42).

This whole incident was one reason why calls for slowing down AI development became louder. Anthropic CEO Dario Amodei published an essay titled “We Must Pace the Frontier,” which received a lot of public attention and was supported by Elon Musk and Sam Altman. Then there was also Jacob Coxon, who left Anthropic because he believes there is a real risk that AI could lead to the extinction of the human race.

In the same episode of Hard Fork and also at Latent Space, the respective guests Ajeya Cotra and Richard Socher explained that one issue is the way models are trained. They are usually rewarded for finishing tasks, which means their only goal is to finish the task, regardless of how they do it or even whether it means cheating, also known as reward hacking. It would be much better if labs found a way to reward models for how they achieve their goals as well. Otherwise, they can enter a kind of “tunnel vision,” not looking left or right and in no way considering whether they cause harm.

Analysis

It is right to call for slowing down AI development to find time for proper security measures, but there is also a second aspect that could solve the issue: liability.

First of all, by now we know that OpenAI did not apply all the necessary monitoring tools. OpenAI itself confirmed that, if it had, it would have detected the activity more than a day before the breach happened. Running evaluations with models capable of cyberattacks without these monitoring systems should be a no-go.

Second, who’s going to pay for the damage? No physical harm has been reported, but the question must still be asked. The agents might not have intended to cause damage. However, if similar incidents happen in the future with potentially more severe outcomes, we need to clarify now who will be responsible and who will pay.

Regulation could be the key to this, and it might not be as difficult as one might think. In fact, regulation does not have to specify every safety measure in detail. Sometimes it is enough to make the penalties for negligence so severe that every company will ensure that all available security measures are in place.

History has shown that this can work in other industries. Take the automobile industry, for example. Volkswagen was caught cheating on emissions tests and had to pay billions in criminal and civil penalties. Individual employees also faced criminal prosecution.

Once these kinds of measures are in place, AI labs might slow down on their own because a serious violation could threaten the whole company.

Additionally, we have to remember that the most powerful US models will, of course, have to comply with regulations imposed by the US government. But we should not forget that the leading open-weight models are currently coming mainly from China, which means that anyone can download and run them. We therefore assume that accountability shifts from the vendor to the actual user: the person or company operating the model.

Sources

Human vs. Agent Code Review

We see that human intervention is absolutely necessary to keep AI on the right path. Whether that also applies on a smaller scale, such as producing code, is another question.

There is a split in the software development community between those who see human code reviews as a thing of the past, be it because agents can do that, because code is cheap, or because our reviews become the bottleneck. The other extreme is that code needs to be read and, even more importantly, understood because, in the end, someone has to be responsible for it.

Robert C. Martin, also known as Uncle Bob, is a highly respected expert in the software industry. He was one of the 17 original authors of the Agile Manifesto, which gave the agile movement its name. He also wrote the influential book Clean Code, which popularized the idea that code should be readable and maintainable. One would therefore clearly associate him with the “read the code” group.

It therefore came as a surprise when he posted on X that he no longer reviews the code written by agents but instead measures metrics like test coverage, dependency structure, or other architectural metrics, which he believes will eventually lead to better code quality. We don't have to review the code; it is more about focusing on and improving those metrics.

Source: Uncle Bob Martin on X.

Bob Martin put his “no code review” approach into perspective in a few podcasts, for example in one with Kent C. Dodds and another one with Matt Pocock.

He made it clear that he still cares about the code itself. He doesn’t blindly trust the agents and therefore still skims the code to find bad patterns. He then uses those findings to realign or improve the agents.

In order to do this, one obviously needs to understand the code and also have experience. In fact, in his conversation with Kent C. Dodds, he suggested that newcomers should write code manually for the first two to three years (timestamp 28:00).

On the pro side, there was also a talk by Dex Horthy from HumanLayer, who shared his findings from a so-called “lights-off software factory,” where no human interaction is necessary. They failed with that approach, and Dex argued that software developers still have to review the code. He also said that new models are becoming better at working on long-running tasks and creating huge systems. But when it comes to producing maintainable code over time, the models do not show the same improvement.

Analysis

Robert C. Martin does not support the idea of vibe coding in the sense that developers don’t have to know about the code. It is more about the means of achieving quality. And in this case, he suggests that relying on the typical code review might not be the most efficient way anymore.

We at Soverius AI believe that code reviews are still important, but that doesn't mean that you have to read every line. In fact, this is something that at least I didn't even do in the pre-AI era. I skimmed those parts where I didn't really see any risk of introducing bugs or violating architectural rules. For example, if I saw changes in the central files of the application, those were the ones that got my attention.

Modern code reviews should be simplified as much as possible. Be it by using deterministic, classic tools like ESLint, where the build already fails and the code never actually becomes a PR. Or by improving skills so that the code is right the first time, or also by using review agents, which can already have an overview of what to check for and highlight questionable parts. So it is more of a combination of AI and human skills.

The main remaining question is how AI-native developers should acquire those skills. Do you have to write code for some time, or are there other ways?

The software industry is not the first to face that issue. For example, look at music critics. They might not be artists, and they might not be able to play an instrument at all, but they still have to be able to tell whether it was a good performance.

Sources

Graph Engineering

Wherever you stand in terms of code review, specific review agents are important. And that means we have to differentiate between agents that implement and agents that review. That's the engineering on top of the model that we have to do.

In mid-July, Peter Steinberger, the creator of OpenClaw, started the discussion around “graph engineering” with a short post on X: “Are we still talking loops or did we shift to graphs yet?”

Source: Peter Steinberger on X.

Peter didn't give more information on that topic, but other people on X wrote articles about this new discipline. In summary, the basic idea is that an agentic workflow is not as straightforward as a loop may imply. It is like a state machine with conditions and branches and therefore has a graph structure.

The basic idea is to describe an agent workflow as nodes and edges. A node performs a bounded task; an edge carries its output to the next task. Once the workflow grows, it can branch, run several agents in parallel, join their results, route failures to different repair paths, or pause for human approval.

Graph Engineering is seen as the successor to Loop Engineering, which was coined by Boris Cherny and Peter Steinberger. We've discussed this in our episode Quantized AI 26/02: Glimpse of Fable, Loop Engineering, Apple Intelligence 3rd Gen.

It might seem like another jump. When Loop Engineering became the new thing, people largely abandoned the term Harness Engineering, although they did not abandon the practice itself. Harness Engineering was more about optimizing one agent that works on the main task, whereas Loop Engineering introduced the idea of multiple agents depending on each other. The whole loop could also be triggered by an event instead of by a developer running a prompt on their local Claude or Codex instance.

With Graph Engineering, the change is less dramatic. In a way, a sophisticated loop already comes with conditions. If there is a triage agent at the beginning that consumes the trigger event, such as a GitHub issue, it already introduces the first condition leading to two branches. Here's your graph already.

We have also seen that social media still generally talks about Loop Engineering and that the terms Graph Engineering and Loop Engineering are used interchangeably, whereas Harness Engineering comes up very rarely.

Analysis

Graph Engineering is the logical successor to Loop Engineering and something you would naturally encounter when you practice loops. We might not even need a new word for that.

The more interesting question is what happens to harnesses when frameworks such as LangGraph, Google ADK, or Microsoft Agent Framework take over. They can provide many of the harness features themselves: memory, tools, MCP connections, persistence, routing, and even more. Either coding agents evolve into orchestration frameworks, or we embed Claude Code and Codex through their SDKs for implementation tasks. The second outcome currently looks more plausible. Frameworks and harnesses overlap, but they do not yet replace one another cleanly.

There is also an economic side to the story. A graph that runs multiple agents can be a huge constraint for developers who are working with subscriptions. The token budget can be exhausted within the first few days of the month. Here again, local models can help, either in a hybrid setup, where they are used only for specific tasks, or for the entire workflow.

Sources

No Slowdown in the Model Release Cycle: Astra, Fable 5.1, Kimi K3, and GLM-5.3

Human code review, regulation, and Graph Engineering: quite a lot that humans still need to do. But have the models themselves also become better? Let's look at the new models released over the summer.

The release of new models did not really stop over the summer. OpenAI released GPT-6 Astra, Anthropic released Fable 5.1, Google released Gemini 3.8 Flash, Meta released Muse Spark 1.1, and SpaceXAI released Grok 4.6. Major Chinese labs also released new models, including Moonshot AI's Kimi K3, Z.ai's GLM-5.3, and DeepSeek-V4.1-Flash.

For obvious reasons, each vendor claims that its new model is its best and most capable one yet.

It doesn't make sense to list multiple benchmarks and numbers here. Each newly released model is usually better than its predecessor. Whether a model from vendor A has overtaken one from vendor B depends a lot on which benchmark we are looking at. Vendor A might be ahead in one benchmark and vendor B in another.

Generally speaking, the order of vendors in the benchmark rankings didn't change. The top places are still dominated by OpenAI and Anthropic. Chinese models are behind but on par with the models OpenAI or Anthropic released a few months ago.

We usually categorize models via their parameter count. Unfortunately, most non-open-weight model vendors do not publish their parameter count. Among the open-weight models, Kimi K3 is the biggest one, with 2.8 trillion parameters. The number of parameters is not the only deciding factor in a model's quality, but it is a major one. Based on available estimates, we therefore assume that Astra and Fable 5.1 are bigger than Kimi K3.

We also see a trend where token efficiency, or cost per task, is becoming a more and more important factor. Given the current situation, where parts of the industry "cry" for a slowdown in development, the security guardrails applied to a model will also be a deciding factor when selecting it.

Analysis

What we can definitely see is that the time gap between top-notch frontier models and open-weight models is shrinking. And we should be careful with the easy narrative that US labs innovate while Chinese labs only copy. There are real distillation concerns around some Chinese models, and those should not be ignored. But the opposite simplification is also wrong. Chinese researchers and engineers are deeply present in frontier AI, including inside Silicon Valley itself, and Chinese labs are clearly doing original architecture, scaling, infrastructure, and product work of their own.

The more useful framing is not “who copied whom?” but “which parts of the stack are becoming reproducible?” If frontier capability can be reproduced faster, cheaper, and sometimes outside the closed US lab system, then sovereignty, open weights, evaluation transparency, and deployment control become much more important.

Sources

Outlook: Jev and a Different Kind of Model

The next development we are watching takes a different approach. TypeSafe AI has released Jev in early access, describing it as its first "System One Model". Instead of generating open-ended text, Jev is designed to return typed decisions with probabilities that software can use directly.

That makes Jev particularly interesting after an edition about agent security, human review, and Graph Engineering. It asks whether every decision inside an AI workflow really needs a generative language model, or whether some parts should be handled by a narrower model whose possible outputs are defined in advance. We will take a closer look at that approach in the next edition.