DEV Community: albe_sf

The Assistants API is Gone. Here's How to Build Agents Now.

albe_sf — Fri, 02 Oct 2026 15:04:00 +0000

The OpenAI Assistants API was officially shut down on August 26, 2026. This deprecation forces a fundamental shift in how we build stateful agents, moving from a managed, persistent-thread model to a more direct, stateless approach with the new Responses and Conversations APIs. This change simplifies some parts of the stack but places the responsibility for state management squarely back on you, the developer.

what changed: from assistants to responses

The original Assistants API was an attempt to abstract away the complexity of building conversational AI. It provided a stateful environment with persistent "threads," allowing developers to build agents that could maintain context over long interactions without managing the conversation history themselves. It also bundled powerful tools like Code Interpreter and a retrieval system.

However, this abstraction came with trade-offs. The API could feel clunky, and its stateful nature led to unpredictable costs, as the entire conversation thread might be re-processed on every turn. Performance was also a concern, as developers had to poll for updates rather than using a real-time stream.

The new model, centered on the Responses API, is a return to a more primitive, stateless paradigm. The core idea is a direct request/response flow. Persistent threads are replaced by Conversation objects, which you must create and manage. The responsibility for maintaining context between turns now falls to your application code. This is a significant architectural change, but one that offers more control, better performance, and more predictable costs.

the new core of RAG: file_search

For developers building Retrieval-Augmented Generation (RAG) systems, the most important change is the introduction of the file_search tool within the Responses API. This is the successor to the old Retrieval tool and is now the standard way to have models access knowledge from your private documents.

The workflow is straightforward: you create a vector_store, upload your files to it, and OpenAI's backend handles the entire pipeline of chunking, embedding, and indexing. You no longer have to build and manage your own embedding and retrieval logic.

When you make a call to the Responses API, you can make the file_search tool available. The model then intelligently decides when to use it based on the user's query. It performs a semantic search against your vector store, retrieves the most relevant passages, and incorporates them into its response, complete with citations.

Here is a conceptual look at what an API call might look like in Python, enabling the tool for a specific vector store:

from openai import OpenAI
client = OpenAI()

# Assuming a vector_store_id has been created and files uploaded
vector_store_id = "vs_123abc"

# Create a conversation and add a user message
conversation = client.conversations.create()
client.conversations.messages.create(
    conversation_id=conversation.id,
    role="user",
    content="What were the key findings in the Q3 financial report?"
)

# Create a response, enabling the file_search tool
response = client.responses.create(
    model="gpt-6.1-sol",
    conversation_id=conversation.id,
    tools=[{
        "type": "file_search",
        "file_search": {
            "vector_store_ids": [vector_store_id]
        }
    }]
)

# The response will contain the model's answer,
# potentially using information retrieved from the file search.
print(response.output.text.value)

what you gain and what you lose

The primary gain from this migration is control. The move to a stateless API gives you direct authority over conversation history and state management. This means more predictable performance and costs, as you are no longer subject to the black box of the old persistent-thread system. The Responses API also consolidates complex workflows into a single API call, simplifying the overall interaction model.

The most significant loss is convenience. The hand-holding of the managed, infinite-context thread is gone. If your application was architected around the assumption that OpenAI would manage the state, you now have a non-trivial migration project ahead of you. OpenAI does not provide an automatic tool for migrating old Threads to new Conversations.

There is also a key technical trade-off with the managed file_search tool. While it removes the burden of building a RAG pipeline, it also takes away your control over the chunking strategy. For highly structured or complex documents, the automated chunking might not be optimal, which can impact the quality of retrieval. This is a critical limitation to be aware of when deciding whether to use the built-in tool or roll your own retrieval system.

the final word

This shift marks a maturation of the AI developer stack. The initial, heavily-abstracted Assistants API has been replaced by more powerful and primitive building blocks. It’s a move away from providing a magic box and toward giving engineers direct access to the core components of RAG and agentic systems. For builders, this means more responsibility, but also more power. The era of the fully managed agent is over; the era of building your own is here.

Sources

Anthropic's Sonnet 5.5 release is about cost-per-task, not peak performance

albe_sf — Wed, 30 Sep 2026 15:04:14 +0000

Anthropic released Claude Sonnet 5.5 this week, and the main takeaway isn't a new state-of-the-art benchmark. The real story is the focus on cost-per-task for the bulk of everyday engineering work. This isn't a model for moonshots; it's a workhorse for bug fixes, documentation, and routine code generation, where speed and efficiency matter more than cutting-edge reasoning.

what changed

Sonnet 5.5 is the second model in Anthropic's 5.5 family, positioned as a faster, more economical alternative to the top-tier Claude Opus 5.5. Where Opus is designed for complex tasks requiring careful judgment, Sonnet is aimed at well-defined, everyday work.

The key improvements over its predecessor, Sonnet 5, are in efficiency. Sonnet 5.5 generates output over 30% faster. While the per-token pricing remains the same, Anthropic claims it costs up to 30% less for most tasks because it requires fewer tokens to complete the same work.

The performance jump on agentic coding is significant. On the Terminal-Bench 4.0 evaluation, Sonnet 5.5 scored 70.6%, a massive leap from Sonnet 5's 10.3%. This suggests it's far more capable for tasks that require tool use and autonomous operation within a terminal environment.

why it matters for builders

The most expensive model is rarely the right tool for every job. The release of Sonnet 5.5, alongside recent models like OpenAI's GPT-6 Sol and Luna, shows the market maturing. Labs are now competing on the cost-performance curve, not just on leaderboards. For a significant portion of a developer's workflow—writing unit tests, refactoring a function, generating boilerplate, summarizing a pull request—the raw intelligence of a frontier model is overkill. Latency and cost become the primary constraints.

A model that is 30% faster and cheaper for 80% of your daily tasks is a material change to your workflow. It makes continuous, ambient use of AI more practical. When calling the API, you are simply targeting the new model name.

import anthropic

client = anthropic.Anthropic(
    # api_key="my_api_key"
)

message = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Write a Python function to calculate the Fibonacci sequence and include docstrings."
        }
    ]
)

print(message.content)

This shift means you can afford to integrate AI into more granular parts of your development loop without worrying about the bill. The gains in agentic coding also suggest that mid-tier models are becoming viable for more complex automations that were previously the domain of top-tier models.

some gotchas

This is not a frontier model, and Anthropic is clear about that. The company stated that Sonnet 5.5 does not advance the frontier of its models' capabilities. For high-stakes, complex reasoning, Opus 5.5 remains the recommended choice.

Interestingly, because Sonnet 5.5's cybersecurity capabilities are a significant improvement over Sonnet 5, it is the first Sonnet model to ship with the kind of cyber safeguards previously reserved for top-tier models. For certain high-risk cybersecurity requests, the model will visibly fall back to Sonnet 5.

the so-what

The era of just chasing the highest benchmark score is giving way to a more pragmatic focus on the right tool for the job. Sonnet 5.5 is a strong signal that the major labs see a huge market in providing capable, efficient, and economically viable models for the vast majority of software development tasks. For builders, this means more choice and better tools for the everyday grind. The most important model in your stack might not be the most powerful one, but the one that delivers reliable results with the best balance of speed and cost.

Sources

Introducing Claude Sonnet 5.5

DBRX and the Rise of Fine-Grained MoE

albe_sf — Mon, 28 Sep 2026 15:04:18 +0000

A new state-of-the-art open language model just arrived, but the real story isn't just its benchmark scores. The release of DBRX provides a clear signal about where efficient model architecture is heading: fine-grained Mixture-of-Experts (MoE). For builders, this approach is the critical takeaway, as it directly impacts inference speed, serving costs, and the viability of deploying powerful custom models.

a new benchmark for open models

Databricks released DBRX as a general-purpose large language model that outperforms other established open-source models on a variety of standard benchmarks, including language understanding, programming, and math. It was developed to provide enterprises with a platform to build their own custom, high-performance AI systems without relying on a few closed-source providers.

The model itself is a decoder-only transformer, but its architecture is what sets it apart. This design choice is a deliberate move toward efficiency, aiming to deliver top-tier performance while managing the computational costs associated with massive models.

the fine-grained mixture-of-experts architecture

The core innovation in DBRX is its fine-grained MoE architecture. Instead of a single, dense network, an MoE model comprises multiple specialized "expert" networks and a router that selects which experts to engage for a given input. While other models like Mixtral-8x7B use this approach, DBRX implements a more granular strategy.

DBRX has 16 total experts and selects 4 of them for any given input. This is in contrast to models like Mixtral or Grok-1, which use 8 experts and select 2. This finer-grained approach provides a vastly larger number of possible expert combinations, which improves overall model quality. While the model has a total of 132 billion parameters, only 36 billion are active during inference on any single input. This makes the model significantly faster and more cost-effective to serve than a dense model of a similar size.

This architecture is built on open-source projects, making it a design pattern that other builders can adopt. The combination of high active parameter count and a large number of fine-grained experts appears to be a key recipe for its performance.

running dbrx locally

For engineers who want to experiment with the model, DBRX is available on Hugging Face. Getting it running requires the transformers library and a significant amount of VRAM, but the process itself is straightforward. The model uses the GPT-4 tokenizer.

Here is a basic example of how you might load the instruction-tuned model and run inference:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Ensure you have torch and transformers installed
# pip install torch transformers sentencepiece

tokenizer = AutoTokenizer.from_pretrained("databricks/dbrx-instruct")
# Note: This requires substantial memory. Use device_map="auto" for multi-GPU.
model = AutoModelForCausalLM.from_pretrained(
    "databricks/dbrx-instruct", 
    torch_dtype=torch.bfloat16, 
    device_map="auto",
    trust_remote_code=True
)

# The prompt format should follow the model's chat template
user_prompt = "Explain the concept of a Mixture-of-Experts (MoE) model in a few sentences."

messages = [
    {"role": "user", "content": user_prompt}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids, 
    max_new_tokens=200, 
    do_sample=True, 
    top_k=50,
    top_p=0.95
)

response = tokenizer.decode(outputs, skip_special_tokens=True)
print(response)

This snippet demonstrates loading the model and tokenizer, formatting a prompt, and generating a response. The key takeaway for builders is the accessibility of a model with this architecture through standard open-source tooling.

why this matters now

The release of a powerful, open model with a fine-grained MoE architecture is not just an incremental update. It's a clear indicator that the frontier of AI development is increasingly focused on architectural efficiency, not just scaling parameter counts. For engineering teams, this trend is a welcome one. Efficient models like DBRX lower the barrier to entry for building and deploying custom AI applications.

This shift allows more organizations to move from proprietary, closed models to open-source alternatives that they can fine-tune and control. As builders, we should be paying close attention to these architectural patterns. They represent the next step in democratizing access to state-of-the-art AI, making powerful systems more practical and affordable to build and serve.

Sources

Introducing DBRX: A New State-of-the-Art Open LLM

Gemma 2 is here. It's time to pay attention to open models.

albe_sf — Fri, 25 Sep 2026 15:03:35 +0000

Google has released Gemma 2, the next generation of its open models, and it's a release that warrants a closer look from builders. The key takeaway is this: the 27B parameter model delivers performance competitive with models more than twice its size, while running on a single GPU. This isn't just an incremental update; it's a shift in the cost-to-performance ratio for high-quality open models.

what's new under the hood

Gemma 2 comes in 9 billion and 27 billion parameter sizes, with a smaller 2B version also available. The architecture introduces some notable changes from the first generation. It now uses a combination of local and global attention, allowing it to focus on both immediate context and the broader meaning of a text. It also implements Grouped Query Attention (GQA) to improve inference speed and parameter efficiency.

The training data for the 27B model consisted of 13 trillion tokens, while the 9B model was trained on 8 trillion tokens, sourced from a diverse mix of web documents and code. For the smaller 2B and 9B models, Google used knowledge distillation from the larger 27B model, which they report leads to significant performance gains compared to training from scratch with the same token count.

performance and efficiency claims

The headline claim is that the 27B model provides a competitive alternative to much larger, closed-source models. Google states that the 9B model also delivers class-leading performance, outperforming other open models in its size category like Llama 3 8B.

Crucially, the 27B model is designed for efficient inference on a single NVIDIA H100 or even an A100 80GB GPU. This significantly lowers the barrier to entry for deploying a high-performance model, moving it out of the realm of large-scale clusters and onto a single machine. This focus on efficiency makes self-hosting and running local inference for product development much more feasible.

how to get started

Getting up and running with Gemma 2 is straightforward. The model weights are available on Hugging Face, Kaggle, and through Google AI Studio. For local development, tools like Ollama provide a simple way to pull and run the models with a single command.

Here's a basic example of how you might run the 9B instruction-tuned model locally after installing Ollama:

# Pull the Gemma 2 9B model
ollama run gemma2:9b

# After pulling, you can interact with it directly
# or use it via the Ollama API

For more integrated use cases, you can use libraries like transformers from Hugging Face. Here's a Python snippet to load the model and tokenizer, using 4-bit quantization to manage memory usage on consumer hardware.

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch

model_id = "google/gemma-2-9b-it"

# Configure 4-bit quantization
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quantization_config,
    device_map="auto"
)

input_text = "Explain the key architectural changes in Gemma 2."
input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)

outputs = model.generate(**input_ids, max_new_tokens=150)
print(tokenizer.decode(outputs[0]))

This approach makes it possible to run a powerful model on a developer's laptop or a single cloud GPU, which is a game-changer for rapid prototyping and building privacy-sensitive applications.

the so-what for builders

The release of Gemma 2 is another sign that the performance gap between state-of-the-art proprietary models and open models is closing, especially when you factor in efficiency. For builders, this means more freedom. You can fine-tune and deploy a capable model without being locked into a specific API provider or facing unpredictable usage bills.

The ability to run a 27B parameter model that competes with 50B+ models on a single GPU is the real story here. It changes the economics of building AI features and products. It makes local, private, and cost-effective AI applications a more realistic goal for a wider range of developers and organizations. This is a good week to re-evaluate your stack and see where an open model might fit.

Sources

https://blog.google/

MiMo-V2.6 and the bet on open reinforcement learning

albe_sf — Wed, 23 Sep 2026 15:08:28 +0000

Xiaomi just released and open-sourced its MiMo-V2.6 series, with models positioned to compete with leading closed-source offerings on agentic tasks. But the more significant release is the methodology. The accompanying technical report details a scaled-up reinforcement learning (RL) strategy, making a public case that the path forward is not just bigger base models, but better, continuous improvement through RL at scale.

what changed: scaling rl in public

The core idea behind the MiMo-V2.6 series is scaling three components of reinforcement learning in tandem: RL compute, the diversity of task environments, and grader compute. Instead of discrete training runs for different domains, the approach uses one mixed RL run across coding, general agent tasks, visual tasks, and cybersecurity. This produced two main omni-modal models: MiMo-V2.6-Pro, the flagship, and MiMo-V2.6-Flash, a smaller model optimized for cost and speed.

The Pro model, a sparse mixture-of-experts architecture with over a trillion total parameters, now stands as one of the most powerful open-source models available, particularly on benchmarks that measure long-horizon engineering tasks. The company has fully open-sourced the weights and technical report for both the Pro and Flash models. They also released over 7,000 high-quality RL task environments covering software engineering, web design, and knowledge work, providing a valuable resource for others building agentic systems.

the engineering details that matter

The technical report provides specific details about the training process. The RL phase consumed massive compute, processing 1,568 prompts per training step, with each prompt generating 16 attempts. This high-throughput, asynchronous training pipeline allowed the model to learn from billions of tokens at each step within a context length of up to one million tokens.

A key improvement was in the quality of the feedback signal. The team moved beyond simple correctness verifiers to a more sophisticated "groupwise agentic grading" system. This grader analyzes execution traces to provide more accurate reward signals for long-running tasks, steering the model toward more token-efficient solutions. This investment in grader compute represented a significant portion of the total RL cost but was critical for keeping the training process stable and preventing reward hacking.

For builders who want to run the model directly, the Hugging Face page provides configuration details. Serving the Pro model can be done using a containerized environment with a library like vLLM.

# Example command to serve the model using a pre-built container
docker pull vllm/vllm-openai:mimov25-cu129

vllm serve XiaomiMiMo/MiMo-V2.6-Pro-RL \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

This configuration highlights the model's native support for features like automatic tool choice, which is essential for agentic workflows.

the so-what for builders

The release of MiMo-V2.6 pushes the performance of open-source models further, particularly for complex, multi-step tasks that require an agent to interact with tools and environments. On some benchmarks for long-horizon software engineering, the Pro model's performance improved significantly over the course of its RL training.

While the closed-source labs are currently focused on a price war for high-volume inference, this release signals a different competitive front for open-source: the training methodology itself. By open-sourcing not just the model but also the RL environments, the release provides a toolkit for teams building their own specialized agents. It allows builders to replicate, fine-tune, and extend the RL process on their own data and tasks. This is less about providing a drop-in replacement for a commercial API and more about providing the foundation for building custom, self-improving systems.

Sources

Real-Time Translation Latency Just Dropped. Here's What Changed.

albe_sf — Mon, 21 Sep 2026 15:03:44 +0000

Simultaneous interpretation has always been a trade-off between speed and accuracy. Waiting for more context yields a better translation, but speaking sooner reduces the awkward silence for the listener. Qwen's release of Qwen3.8-LiveTranslate suggests a change in the underlying architecture that meaningfully moves the needle, cutting average lag time to 2.3 seconds.

This isn't just an incremental improvement. It's a structural change that makes real-time, multi-speaker translation practical for production applications.

what just shipped

The Qwen team at Alibaba released Qwen3.8-LiveTranslate, a model for real-time, simultaneous translation of audio and video streams. It's available as a hosted API on Alibaba Cloud Model Studio and QwenCloud, accessible over a WebSocket connection.

The key metric is a drop in Length-Adaptive Average Lagging (LAAL) from 2.8 to 2.3 seconds, an 18% reduction in the average time the translation trails the source speech. The model understands 60 languages and can generate speech in 29 of them.

the interleave architecture

The performance gain comes from a new design Qwen calls the Interleave architecture. Previous systems often treated speech recognition, translation, and speech synthesis as separate, sequential steps. This new model reframes the problem by treating the incoming audio and the outgoing translated text as a single, interleaved data stream.

Under the hood is a two-module design described as a "Thinker" and a "Talker". The Thinker module arranges the audio, source text, and translation into one causal sequence. This allows the model to process audio and generate translated text within a single, unified process, which improves both quality and latency. The Talker module then handles speech synthesis, preserving the original speaker's voice.

new capabilities for builders

Beyond the latency reduction, the release includes two features that address common pain points in building real-world translation apps.

First is real-time speaker diarization. The model can distinguish between different speakers in a multi-party conversation and attribute the translation correctly. The API also exposes voice cloning capabilities to maintain a more stable voice for each speaker throughout a session.

Second is a synchronized bilingual display. The API can stream both the source transcription and the translation as separate, aligned events. This allows you to build UIs that show both languages simultaneously, which is critical for applications like subtitling or meeting summaries where users might need to reference the original text.

Here is a conceptual look at how you might handle the WebSocket stream in Python.

import asyncio
import websockets
import json

# Note: This is a conceptual example. 
# Refer to official Alibaba Cloud documentation for the actual API endpoint and auth.

WEBSOCKET_URI = "wss://api.qwen.ai/v1/translate/qwen3.8-livetranslate-flash-realtime"

async def stream_audio_for_translation(audio_chunk_iterator):
    async with websockets.connect(WEBSOCKET_URI) as websocket:
        print("Connection established.")

        async def send_audio():
            for chunk in audio_chunk_iterator:
                await websocket.send(chunk)
                await asyncio.sleep(0.1) # Simulate real-time streaming
            await websocket.send(json.dumps({"event": "stream_end"}))

        async def receive_translation():
            async for message in websocket:
                data = json.loads(message)
                if data.get('event') == 'transcription':
                    print(f"SOURCE: {data['text']}")
                elif data.get('event') == 'translation':
                    print(f"TRANSLATION: {data['text']}")
                elif data.get('event') == 'error':
                    print(f"Error: {data['message']}")
                    break

        await asyncio.gather(send_audio(), receive_translation())

# Example usage:
# async def get_audio_chunks():
#     # Your logic to get real-time audio chunks from a mic or stream
#     for i in range(10):
#         yield f"audio_chunk_{i}".encode('utf-8')
#
# asyncio.run(stream_audio_for_translation(get_audio_chunks()))

so what

For engineers who have previously dismissed simultaneous translation as too slow or inaccurate for interactive use cases, this release is a signal to re-evaluate. The architectural shift from a sequential pipeline to an interleaved stream is a meaningful change. When combined with practical features like speaker separation, it makes building robust, multilingual, real-time voice applications significantly more feasible.

Sources

Mistral just shipped a vision model and cut flagship prices

albe_sf — Fri, 18 Sep 2026 15:02:31 +0000

Mistral just made two significant moves that position it as an increasingly practical alternative to the major closed-source labs. The company released Pixtral, a new vision model, and simultaneously announced a price reduction for its flagship model, Mistral Large 2. This isn't just a routine update; it's a clear signal about their strategy: compete directly on multimodal features while aggressively pushing down cost.

what just shipped

The update, which appeared on their official changelog on September 17, included three key changes.

First, the release of pixtral-12b-2409, their new vision model. The naming suggests a 12-billion parameter model, a deliberate choice to offer strong performance in a cost-effective, easily deployable size. This follows a broader industry trend toward multimodality, but Mistral's commitment to open and efficient models makes this release particularly notable for builders.

Second, they released an updated version of their small model, mistral-small-2409. While less flashy than a new vision model, continuous improvement of smaller, efficient models is critical for production use cases where latency and cost are primary constraints.

Third, Mistral cut the price on their most capable model, Mistral Large 2, and introduced a free API tier on their platform, La Plateforme. This directly addresses one of the biggest barriers to adoption for smaller teams and individual developers: cost. Lowering the price of your top-tier model is a confident move designed to capture more production workloads.

competing on cost and capability

The release of Pixtral is a direct answer to the multimodal models from larger labs. For developers building applications that need to understand or process images, this provides a new, potentially more open and efficient option. While benchmarks are not yet available, a 12B parameter model is large enough for serious tasks without incurring the inference costs of massive frontier models.

Integrating a model like this into a workflow is straightforward. You can expect to interact with it through the standard API, likely with a modified payload to handle image inputs, such as a base64-encoded string or a URL.

import requests
import base64

API_KEY = "YOUR_MISTRAL_API_KEY"
MODEL_NAME = "pixtral-12b-2409"

# Encode a local image file
with open("path/to/your/image.jpg", "rb") as image_file:
    encoded_string = base64.b64encode(image_file.read()).decode('utf-8')

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json",
}

data = {
    "model": MODEL_NAME,
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe the contents of this image in detail."},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encoded_string}"}}
            ]
        }
    ]
}

response = requests.post("https://api.mistral.ai/v1/chat/completions", headers=headers, json=data)

print(response.json())

This is more than just adding a feature. It's about providing the core building blocks that engineers need. The simultaneous price cut on Mistral Large 2 reinforces this. It makes the entire stack, from small and efficient to large and powerful, more economically viable. For teams running systems at scale, these cost differences add up quickly.

the bigger picture: open distribution

Mistral's strategy appears to extend beyond just models and APIs. Their recent partnership with Mozilla to integrate Mistral models into the Firefox browser is a move to control distribution and reach users outside of the typical developer-focused cloud platforms. By building a presence directly in the browser, they are creating a new channel for their technology, one that is built on a foundation of open technology.

This matters. As AI becomes more deeply embedded in our daily tools, the question of who controls the underlying models becomes critical. By partnering with organizations like Mozilla and continuing to release open-weight models, Mistral is providing a real alternative to the closed ecosystems of Big Tech.

For builders, this is a positive development. It means more choice, better pricing, and the ability to build on platforms that align with the open principles of the web. The latest releases are not just new tools, but a continued investment in an ecosystem that offers more than one way to build.

Sources

Mistral AI Docs: Changelog

Google's New Gemini Voice Models Change The Rules For Agent State Management

albe_sf — Wed, 16 Sep 2026 15:04:00 +0000

Google released two new live dialogue models on September 15, 2026: Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking. While incremental model numbers are common, the architectural shift in the Extended Thinking variant is not. It introduces parallel reasoning, allowing the model to process tasks in the background while streaming an audio response, fundamentally changing the interaction pattern for building complex voice agents.

what just shipped

Google has positioned the two models for different use cases. Gemini 3.8 Live is framed for scale, cost efficiency, and fluid dialogue. The more interesting model for developers building complex agents is Gemini 3.8 Live Extended Thinking. This version is designed for high-complexity tasks that require multi-step reasoning.

Both models are being rolled out across the Gemini API, Google AI Studio, and into products like Google Workspace and Search. The core capability upgrade is a move toward more natural and fluid voice interactions, but the mechanism for achieving this has direct implications for developers using the API.

asynchronous reasoning is the new default

The most significant change is how the Extended Thinking model handles work. It can process background tasks, like asynchronous tool calls, while simultaneously generating and streaming a continuous audio response. This is a departure from the traditional, blocking request-response cycle where the user waits in silence while the model completes its entire thought process.

For anyone who has built a voice agent, the benefit is obvious: the user gets immediate verbal feedback while the agent continues to work on a complex query. This makes the interaction feel less robotic.

However, it imposes a new burden on the client application. The developer documentation notes that when using asynchronous reasoning, a turnComplete: true signal no longer indicates that the model is idle. The server may still be processing tool calls or other background reasoning. Your client must continue to listen for subsequent server messages even after the initial audio stream for a given "turn" has finished.

updating your client-side logic

This new interaction model requires a shift in state management. A simple while loop that terminates on a completion flag is no longer sufficient. Your application needs to handle a more persistent connection and be prepared for out-of-band messages from the server.

Consider a conceptual Python client. Previously, you might have done something like this:

# Old pattern: simple request-response loop
response_stream = model.generate_content(audio_chunks, stream=True)

for chunk in response_stream:
    play_audio(chunk.audio)
    if chunk.turn_complete:
        break # The turn is over, we can stop listening.

The new model requires logic that persists and handles different message types after a turn appears to be complete.

# New pattern: persistent listening for async events
response_stream = model.generate_content(audio_chunks, stream=True)

# Assume a persistent connection or long-lived stream
for message in response_stream:
    if message.has_audio_chunk():
        play_audio(message.audio_chunk)

    if message.has_tool_call():
        # Even if audio is playing or has finished for this turn,
        # we receive and dispatch a tool call here.
        dispatch_tool_call(message.tool_call)

    if message.turn_complete:
        # This no longer means the model is idle.
        # It's just the end of this speech segment.
        # The loop continues, listening for more messages like tool results.
        print("INFO: Turn complete, but continuing to listen for background tasks.")

# The connection would remain open to receive further updates

This is a more complex, event-driven approach. The client has to be able to play audio while simultaneously listening for and processing other events, like function call requests from the model.

the so-what for builders

The introduction of asynchronous reasoning is a significant step toward more capable voice agents that can tackle complex, multi-step tasks without creating an awkward, silent user experience. It moves the interaction closer to a human-like collaboration where speaking and thinking can happen in parallel. For any team building production-ready voice agents, this is a pattern worth paying attention to.

However, this capability comes with the engineering overhead of more complex client-side logic. It also introduces an open question on cost. While competitors often publish per-minute pricing for their real-time voice APIs, Google has not yet done so for these new models, leaving a critical variable unknown for teams evaluating them for production use.

Ultimately, the release of Gemini 3.8 Live Extended Thinking signals that the frontier of voice AI is moving beyond simple, turn-based interactions. The next challenge for builders is to create applications that can gracefully manage the state of an agent that thinks and speaks at the same time.

sources

Claude 3.5 Sonnet is more than an upgrade. It’s a new workflow.

albe_sf — Mon, 14 Sep 2026 15:04:51 +0000

Anthropic just shipped Claude 3.5 Sonnet, its first model in the new 3.5 series. It’s faster and cheaper than Opus, and outperforms it on several key benchmarks, but the bigger story for builders is a new feature called Artifacts. This addition moves Claude from a simple conversational AI into a collaborative work environment, changing the core loop of how we build with these models.

from chat loop to live workspace

Historically, the workflow for using an LLM to code or create has been a turn-based conversation. You ask for a code snippet, you copy it, paste it into your local editor, run it, find an error, and then return to the chat to describe the problem. This cycle repeats, with the context split between your IDE and the chat window.

The new Artifacts feature in the claude.ai interface changes this. When you ask for something like a React component or an SVG graphic, Claude now generates it in a dedicated panel next to the conversation. This panel is a live workspace where you can see the rendered output, edit the code, and iterate in real-time without leaving the browser.

This creates a tighter feedback loop. Instead of copying and pasting, you can prompt for a change—"add a login feature"—and see the code and the rendered preview update in the same window. It’s a move toward making the AI an active participant in a shared workspace, not just a passive respondent in a chat.

agentic coding gets a boost

Beyond the UI, the model itself has significant improvements for developers. In an internal agentic coding evaluation, Claude 3.5 Sonnet solved 64% of problems, a marked improvement over the 38% solved by Claude 3 Opus. This evaluation tests the model's ability to fix bugs or add functionality to open-source codebases based on a natural language description.

This suggests the model is more capable of complex reasoning, troubleshooting, and independent code execution when given the right tools. For teams working on migrating legacy applications or automating codebase maintenance, this is a meaningful step up. The model handles code translations and updates with more proficiency.

Accessing the new model is straightforward. It's available through the Anthropic API, as well as on Amazon Bedrock and Google Cloud's Vertex AI.

import anthropic

client = anthropic.Anthropic(
    # defaults to os.environ.get("ANTHROPIC_API_KEY")
    api_key="my_api_key",
)

message = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Write a python function to check if a number is prime."
        }
    ]
).content[0].text

print(message)

The pricing is set at $3 per million input tokens and $15 per million output tokens, with a 200K token context window. This makes it significantly more cost-effective than Opus for tasks that require high intelligence but also demand speed, like orchestrating multi-step agentic workflows.

what this means for builders

The release of Claude 3.5 Sonnet and the Artifacts feature isn't just another incremental model bump. It's a signal of where the developer experience is heading. The friction of context-switching between a chatbot and an IDE is being actively designed out of the process.

This evolution from conversational partner to collaborative tool is the key takeaway. For builders, it means we can start designing workflows that assume the AI is in the editor with us, not just in another window. This has implications for prototyping, debugging, and even how we onboard new developers to a complex codebase. The model is becoming an on-demand teammate.

Sources

https://www.anthropic.com/news/claude-3-5-sonnet

Cohere's 218B Parameter MoE Model for Translation Dropped Quietly

albe_sf — Fri, 11 Sep 2026 15:06:58 +0000

Cohere released a 218-billion-parameter translation model, and it barely made a sound. North Small Translate is a massive sparse model that sets a new performance benchmark for its domain. Its architecture and release strategy show where production-grade specialized models are heading: massive scale, focused on a single task, with efficiency coming from sparsity.

what shipped

On September 9, 2026, Cohere published release notes for North Small Translate, an open-weight Mixture-of-Experts (MoE) model built specifically for machine translation. The model has 218 billion total parameters, with 25 billion active for any given token. It's a sparse architecture with 128 experts, activating 8 per token.

This isn't a general-purpose chat model. It's a specialist, supporting translation across 50 languages. The weights are available on Hugging Face for research and non-commercial use under a CC BY-NC 4.0 license. For production use, Cohere routes you to a commercial license and their Model Vault deployment.

Performance-wise, Cohere reports a WMT26 score of 83.60 across all evaluated languages. They also note this can be pushed to 84.36 using an agentic multi-pass workflow where the model refines its own output.

why it matters for builders

The most significant takeaway is the hardware footprint versus the parameter count. Because it's a sparse MoE model, you aren't loading all 218B parameters for every inference. Cohere provides clear hardware minimums for different quantization levels. A 4-bit quantized version can run on a single NVIDIA B200 or two H100 GPUs. The full BF16 precision requires four B200s or eight H100s. This is still substantial, but it puts a model of this scale within reach for self-hosting, which is not the case for dense models of a similar size.

The release strategy itself is also notable. This was a quiet drop, first appearing on Hugging Face weeks before the official release note. It's a move towards treating large models less like blockbuster events and more like industrial components. You have a specific, high-value problem like translation at enterprise scale. You deploy a specialized, high-performance component to solve it.

For teams working with multilingual systems, this model represents a new frontier for quality, especially for the 32 high-resource languages it covers well.

using specialized translation models

While you can download the weights for evaluation, most production use will be via an API. Interacting with a dedicated translation model is more direct than prompting a general-purpose model. You're not engineering a complex prompt with few-shot examples; you're calling a function.

Here’s a hypothetical Python snippet of what an SDK interaction might look like. Note that this is a representative example, not a direct copy of Cohere's current SDK.

import cohere

# Assuming API key is configured in environment variables
co = cohere.Client()

# The model ID would point to the specialized translation model
model_id = "north-small-translate-1-0"

source_texts = [
    "To build great AI products, focus on the user's workflow.",
    "La arquitectura de transformadores es la base de los modelos lingüísticos modernos."
]

target_language = "de" # German

response = co.translate(
    model=model_id,
    texts=source_texts,
    target_language=target_language
)

for translation in response.translations:
    print(f"Original: {translation.source_text}")
    print(f"Translation: {translation.text}\n")

The key is the shift from conversational prompting to a more structured, tool-like interaction. The model expects a specific input (text and a target language) and provides a specific output (the translated text). The complexity is in the model's architecture, not in your prompt.

the takeaway

North Small Translate is a signal of maturity in the AI space. We are moving past the era where every new model had to be a better generalist. Instead, we are seeing the rise of massive, hyper-specialized models that are state-of-the-art at a single, commercially valuable task. For builders, this means having more powerful and efficient tools for specific jobs, even if it requires significant hardware to run them yourself. It pays to watch the specialists, not just the chatbots.

sources

Gemini 3.8 Flash is not a routine update

albe_sf — Wed, 09 Sep 2026 15:04:52 +0000

Google's release of Gemini 3.8 Flash is more than an incremental version bump. It represents a deliberate focus on the workflows that builders are actually shipping: long-running agents, multi-step reasoning, and software engineering tasks. This isn't about chasing chatbot benchmarks; it's about providing more effective tools for complex, automated systems.

what changed with 3.8 flash

The key advancements in Gemini 3.8 Flash are centered on performance for software engineering and agentic knowledge workflows. This is a direct response to how developers are using these models in production. While general capability improvements are always welcome, targeted enhancements for code generation, debugging, and orchestrating complex tasks are what move the needle on a day-to-day basis.

For teams already using the Gemini 3.x series, the transition is straightforward. The introductory API pricing for 3.8 Flash remains the same as it was for 3.7 Flash, though Google has indicated this pricing will change in January. This provides a window for developers to integrate and test the new model's capabilities without an immediate cost increase.

The model continues to support customizable effort levels, allowing a trade-off between quality, cost, and latency. This is a critical feature for production systems where you might want to use a faster, cheaper response for one task and a slower, higher-quality one for another.

a dedicated model for cyber

The most significant part of this release is the introduction of Gemini 3.8 Flash Cyber. This is a specialized variant of the model fine-tuned for cybersecurity use cases, specifically for vulnerability discovery and automated patching.

Access to this model is not public. It's being made available to trusted defenders through a new channel called Google's Fairwind Program. This gated approach is becoming a pattern for frontier models with sensitive capabilities. By controlling the release, providers aim to mitigate misuse while still getting the tool into the hands of security professionals who can use it for defense.

This move signals a broader industry trend. As models become more powerful, we will see more of these specialized, access-controlled variants for high-stakes domains. Expect to see similar models for finance, medicine, and critical infrastructure in the near future. For builders, this means the most powerful tools may require a verification process, not just an API key.

{
  "task": "scan_and_patch",
  "target": {
    "repository": "github.com/example/repo",
    "branch": "main"
  },
  "model_config": {
    "provider": "google",
    "model": "gemini-3.8-flash-cyber",
    "credentials_secret": "GOOGLE_FAIRWIND_TOKEN"
  },
  "parameters": {
    "vulnerability_types": [
      "sql_injection",
      "cross_site_scripting",
      "dependency_confusion"
    ],
    "action": "propose_pull_request",
    "notify_channel": "#security-alerts"
  }
}

the so-what for builders

The release of Gemini 3.8 Flash and its Cyber variant confirms that the next phase of AI development is specialization. Foundational, general-purpose models are becoming a commodity. The real value is in models that are expertly tuned for specific, high-value vertical tasks like software engineering and cybersecurity.

This shift has direct implications for how you build. It means that simply calling a generic model API is no longer the optimal approach. Instead, you should be evaluating a portfolio of models, including specialized ones, and routing tasks to the tool best suited for the job. The future of building with AI is less about having one all-powerful model and more about orchestrating a fleet of specialized agents.

Sources

https://deepmind.google/news/

Anthropic's Fable and Mythos 5.1: More Than a Model Update

albe_sf — Mon, 07 Sep 2026 15:05:43 +0000

Anthropic's release of Claude Fable 5.1 and Mythos 5.1 is more than an incremental update. It marks a strategic shift in how frontier models are productized, splitting a single underlying architecture into two distinct offerings tailored for different risk profiles and use cases. For builders, this means a new state-of-the-art model for coding and knowledge work that is also cheaper to run, coupled with a clearer framework for how the most powerful capabilities will be gated.

what changed: fable vs. mythos

Fable 5.1 is the new flagship model for general availability, setting a new performance standard for coding, knowledge work, and complex problem-solving. It replaces its predecessor as the state-of-the-art option for most developers building on the platform. The key change is that Fable 5.1 is one of two new models. The other, Mythos 5.1, is the same underlying model but with different safeguards.

Mythos 5.1 is designed specifically for high-stakes research in sensitive fields like cybersecurity and biology. Access is restricted to a small number of vetted organizations through trusted access programs. This bifurcation is the main story: instead of a single model with one-size-fits-all safety controls, Anthropic is creating distinct products from the same core intelligence. Fable 5.1 gets more precise safeguards that are less likely to intervene on benign requests, while Mythos 5.1 provides more specialized capabilities for trusted partners.

pragmatic shifts for builders

For engineers shipping products, the most significant changes are economic and practical. Fable 5.1 is estimated to be 25% less expensive than Fable 5 for typical workloads, a meaningful reduction for production systems. This cost reduction makes it more feasible to use a frontier-class model for tasks that might have previously been relegated to smaller, less capable models.

The updated safety mechanisms in Fable 5.1 are also a practical benefit. The new safeguards are more precise, with interventions on benign biology-related requests reportedly reduced by 85%. The model can now be used to identify software vulnerabilities in source code, a task that was previously more restricted. This fine-tuning of the safety layer means fewer false positives and a more reliable experience for developers working on legitimate but potentially sensitive applications.

When using the API, the model name is the primary change, but developers should also be aware of new cost-saving mechanics like improved caching.

import anthropic

client = anthropic.Anthropic(
    # api_key="my_api_key",
)

# A typical call to the new Fable 5.1 model
message = client.messages.create(
    model="claude-fable-5-1",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": "Review this Python code for potential vulnerabilities and suggest improvements."
        }
    ]
).content.text

print(message)

This approach allows developers to access state-of-the-art performance for general coding and analysis tasks without needing to apply for specialized access programs.

the new playbook for frontier model deployment

The Fable/Mythos split is a clear signal of where the industry is heading. As model capabilities increase, especially in scientifically sensitive areas, a single safety policy becomes untenable. A blanket approach either stifles legitimate research or fails to adequately contain risk.

By creating a tiered system, labs can offer a powerful, general-purpose model like Fable 5.1 to a broad audience while reserving the most potent, potentially dual-use capabilities for partners who have undergone a vetting process. This allows them to continue pushing the research frontier with Mythos 5.1 while providing a more stable and predictable product for the majority of their customers.

For builders, this trend is worth watching closely. It suggests that future model access will be less about a single API endpoint and more about a portfolio of models, each with specific capabilities, safeguards, and access requirements. Understanding this structure will be as important as understanding the model's performance on benchmarks.