ICYMI AI: The latest AI trends, delivered to you on Dev.to
Tech Talk: TypeSafe’s Jev is a model that only returns probabilities
TypeSafe's Jev, released into early access on September 15, is a general-purpose classification model. It reads up to 32K tokens of input (such as a support ticket, a customer record, or a row of data). It answers any number of typed questions about it in a single parallel pass, and returns probabilities rather than prose.
Outputs are limited to three options: Choice, Score, and Noul (yes or no), so structured-output errors and bad tool calls drop to zero by construction. A valid answer isn't always the right one, and Jev doesn't claim otherwise.
What it does is tell you how sure it is. TypeSafe trains it for calibration, so its probabilities match real-world frequencies. When Jev says it's 80% confident, it's right about 80% of the time. That lets you set a threshold, automatically acting on high-confidence calls and routing the rest to a human or a larger model.
The pricing is what makes Jev worth a look. At \$0.042 per million input tokens with free output, TypeSafe estimates about \$0.0004 per decision, roughly 1/76th the cost of GPT-5.6 Terra on its benchmark. Within days, imitators appeared: Laya, Bespoke Nimble, and Kev, several of them open-weight fine-tunes of Qwen 3.5 you can run locally. OpenAI likewise followed with a Decisions API of its own.
For developers, the change is in how you structure the call. Instead of sending an LLM one broad question like "is this spam?" and parsing a paragraph back, you pass Jev the email once, along with a set of narrower typed questions, such as: Does the sender domain match the claimed company? Does it ask for credentials? How urgent is the language? Each comes back as a probability. Your own code then applies the thresholds and makes the call, so the routing policy lives in your repo, versioned and auditable, and the model only scores the inputs.
Jev is available today on Serverless Inference through the DigitalOcean Model Library, at Typesafe’s list price—no separate TypeSafe account needed. Pair it with the DigitalOcean Inference Router to send routing and classification traffic to Jev while keeping frontier models in reserve for the calls that need them.
Event calendar
Open Intelligence Summit
October 12-13th, San Francisco
The argument is no longer “Is open source AI cheaper?” It’s now “Who owns intelligence?” A movement is forming around open intelligence and generating the belief that the intelligence powering our AI products and companies should be more open, portable, and controllable.
Join DigitalOcean, NVIDIA, RadixArk, Inferact, and more for deep dives, demos, and discussions about the technologies that keep intelligence portable, interoperable, and in the hands of builders.
Industry news
Near-flagship intelligence at a fifth of the price was the headline at OpenAI DevDay, but the bigger shift is the stack forming around the model.
OpenAI announced GPT-6.1 Sol, which it says delivers near-Astra intelligence at a fifth of Astra's token prices, plus an Ultrafast tier with up to 6x faster generation in the API. It also added computer use to its Agents API and opened ChatGPT, which it puts at 1.2B weekly users, to developer-built plugins and partners like Devin, Notion, and Vercel.
Agent safety is moving below the application layer.
NVIDIA's Open Agent Safety Platform pairs OpenShell, an open-source runtime that traces agent actions and enforces policy, with Sentry, an out-of-band watchdog on BlueField-4 DPUs that NVIDIA says can quarantine an agent that leaves its boundaries within milliseconds. NVIDIA says more than 100 organizations, including Anthropic and CrowdStrike, are working with the technology, and DigitalOcean is among the companies supporting the platform.
New models available on DigitalOcean Inference Engine
We recently launched models from TypeSafeAI, DeepSeek, Z.ai, NVIDIA, OpenAI, and Anthropic, all with different strengths and use cases. Learn more about the models in our What’s New blog.
Start using them through the DigitalOcean API or directly in the Cloud Console.
DeepSeek-V4.1-Flash
DeepSeek-V4.1-Flash cuts the KV cache footprint to 1/4 of V4-Flash while outperforming DeepSeek's own V4-Pro flagship on coding and agentic benchmarks, addressing the cache-hit charges that often dominate agent costs. The 552B MoE model activates just 8B parameters on input, delivering frontier performance with infrastructure economics that improve as you scale.
GLM-5.3 and GLM-5.3-Flash from Z.ai
Z.ai's post-training translates to significant performance improvements for GLM-5.3, achieving a 50% lower cost per completed coding task than GLM-5.2 while jumping to 28.3 on Terminal-Bench 3.0. GLM-5.3-Flash adds native image and video input with the same agentic strength, giving teams both reasoning-heavy and efficient multimodal paths on a single platform.
Nemotron™ 3 Diarization from NVIDIA
NVIDIA's Nemotron™ 3 Diarization replaces the multi-stage speaker attribution pipeline (VAD, embeddings, clustering, post-processing) with a single 100M-parameter model, keeping audio within DigitalOcean's infrastructure rather than routing it to third-party APIs. It identifies up to 8 speakers (double the previous limit) with a 14.72% error rate.
GPT-6.1 Sol, GPT-6 Astra, Sol, and Luna from OpenAI
One week after GPT-6 Sol’s launch, OpenAI releases GPT-6.1 Sol, which offers near-Astra-level coding and substantial improvements in document understanding, computer use, and scientific work. OpenAI's three-tier GPT-6 family also debuts on DigitalOcean: Astra for computer use and coding, Sol for frontier reasoning at 90% lower cost than Opus 5, and Luna for high-volume tasks.
Claude Fable 5.1, Opus 5.5, and Sonnet 5.5 from Anthropic
Anthropic's latest point releases deliver focused gains: Fable 5.1 reaches 55.8% on Terminal-Bench 4.0 with 75% cheaper cache reads, while Opus 5.5 matches high-effort results at default effort using half the tokens and 20% lower per-token cost. Sonnet 5.5 runs 30% faster with equivalent cost savings, with a shift toward better token efficiency and lower verification burden.
Community tutorials
How naive failover between inference providers makes outages worse
In a six-hour fault-injection run on an H200 GPU Droplet, the standard retry-with-backoff policy pushed 3.54x normal load onto a degraded provider, while a retry budget plus circuit breaker held it to 1.02x, and naive retries left 3,743 zombie requests generating 959,232 output tokens for clients that had already disconnected. Learn how to classify inference errors before failing over, replace fixed retry counts with a budget, and verify whether your provider actually stops billing when you cancel a stream.
What vision-language models cost to serve at production scale
A single uploaded photo can add more than a thousand input tokens before the model writes a word, and at 500,000 requests a month, that gap is the difference between \$155 on serverless Nemotron Nano 12B VL and \$3,175.20 for a dedicated H100 running 24/7. Walk through the per-request cost math, the roughly 10.2 million-request breakeven, where dedicated wins, and six levers for cutting image-token spend.
Speculative decoding's hidden cost: the KV cache capacity you give up, and where throughput turns negative
On a 141 GB H200 serving Qwen3-14B with vLLM, a separate Qwen3-0.6B draft model consumed about 43.8 percent of baseline batch slots, and even n-gram speculation, predicted to cost nothing, measured a 2.8 percent capacity tax. See where each method's throughput crosses below baseline as concurrency rises, and get the config.json arithmetic to predict a draft model's KV cost before spending GPU hours.
Cache reliability is a pricing term: what I learned measuring DeepSeek-V4.1-Flash on DigitalOcean Serverless Inference
DeepSeek-V4.1-Flash has the deepest cache discount in the Serverless Inference catalog at \$0.006 per million cached tokens, yet across 844 API calls it hit the cache on only 78.6% of warm turns, and with a miss priced at 50x a hit, it cost 2.2x what GLM-5.3-Flash did for the same 15-turn, 200K-token sessions. Discover how DigitalOcean's 64-token-block prefix cache works, how to log your own hit rate from the usage object, and why reasoning_effort: max cost up to 1.88x more without improving accuracy.
Why DeepSeek-V4.1-Flash splits reading from writing
DeepSeek-V4.1-Flash's Causal Encoder-Decoder splits a 40-layer model into a 20-layer encoder that runs prefill at 8B active parameters and a full stack that decodes at 16B, shrinking the global KV cache to about 890 bytes per token, roughly a quarter of V4-Flash's. Learn why that asymmetry, not the 74.2 versus 62.7 DeepSWE score against V4-Pro, is what lowers the bill for read-heavy coding agents.
Product updates
Introducing DigitalOcean Managed Agents, now in Public Preview
AI agents are ready to take on more, and we built the infrastructure to match. DigitalOcean Managed Agents brings together Harness Runtime, a purpose-built runtime for running agent sessions at scale, and Action Gateway, governed access to the tools agents need to get real work done, all in one platform.
Deploy your preferred agent harness, Claude Code, Codex CLI, OpenCode, and more, or bring your own, and connect to 16,000+ tools across 500+ providers through a single managed endpoint. Sessions are ready in under a second, resume in about 300 milliseconds with full context intact, and you only pay for the compute you actually use.
Whether you're automating support workflows, refactoring code across repositories, or building the next agent-powered product, Managed Agents gives you the infrastructure to do it, without building it yourself.
DigitalOcean MicroVMs now in private preview
Isolated, Firecracker-based VMs that pause when idle and resume with state intact, built for agent workloads that run in bursts. The same layer that powers Managed Agents is now available directly to teams building their own agent platforms, sandboxes, and short-lived jobs.
Top comments (1)
"A valid answer isn't always the right one, and Jev doesn't claim otherwise" is the most useful sentence here. I'm building a README-vs-code checker on Jev this week, and that's the whole design: Jev scores each line, my code makes the call, and low confidence becomes "can't tell" instead of a wrong verdict. Cost so far: a whole 283-line README for $0.0024.