DEV Community: machinelearning

DemoDoctor: Bad Robot Demonstrations Are Detectable. Do They Matter?

Raihan — Wed, 14 Oct 2026 14:31:00 +0000

Inject known faults into robot demonstrations, detect them without labels, and measure what they cost a policy. The honest answer so far: I could not measure a cost.

Scope note: PushT only, ACT policy, RTX 3060. A data-quality study by an ML engineer learning robotics — not robotics expertise.

The problem

Robot-learning teams say data quality matters more than model size, but controlled measurements are rare. This project corrupts clean PushT demonstrations with ground-truth faults, detects them from signals alone, then closes the loop: train on clean vs corrupted vs auto-cleaned data and measure simulator success.

The faults

Fault	What it mimics	Detectable?
Lag (1–3 frames)	Slow camera pipeline	Weak — XCorr landscape flat on quasi-static pushing
Stall	Teleop freeze	Yes — F1 0.867
Jitter (scaled)	Shaky teleoperation	Yes — F1 0.687
Truncation	Episode cut early	Weak — natural length variance swamps cuts
Dup frames	Stuck camera	No — re-encoding creates duplicates everywhere

The detectors (all label-free)

Lag: cross-correlate image-motion energy with action-velocity energy; stable version requires both halves to agree. Fails honestly on PushT: slow motion gives flat correlation landscapes where noise peaks beat true lag.
Jitter: raw jerk vs median-filtered baseline. Key fix: absolute sigma is meaningless (PushT actions have std ~80, not ~1) — scale to the episode.
Stall: contiguous near-zero-velocity runs. Just works.
Truncation: robust length outlier (median ± MAD). Weak by nature here.
Dup frames: exact-duplicate check confounded by video re-encoding (which creates duplicates in clean data too). Reported as null with the mechanism explained.

The policy experiment

I trained ACT policies on PushT for 60k steps (batch size 32, RTX 3060, about 2 hours each) under four data conditions, 3 seeds each, and evaluated every policy on 50 rollouts:

Clean: all 206 episodes.
Corrupted: 30% of episodes carry an injected fault.
Auto-cleaned: the 175 episodes left after dropping stall- and jitter-flagged ones (26 of the 31 dropped were truly faulty).
Random 175: 175 random episodes of the corrupted set. This is the size-matched control: if cleaning looked worse than corrupted, it separates "less data" from "better data".

Condition	Success rate	Mean max reward [95% CI]
Clean	0.7%	0.425 [0.369, 0.488]
Corrupted	1.3%	0.397 [0.344, 0.454]
Auto-cleaned	0.0%	0.359 [0.302, 0.423]
Random 175	0.0%	0.432 [0.386, 0.476]

Intervals bootstrap over policies, then over rollouts within a policy. No pairwise difference excludes zero. The widest gap is auto-cleaned vs random 175 at -0.073 [-0.147, +0.006]. Clean minus corrupted is +0.028 [-0.050, +0.108]. The point estimates even run the wrong way for cleaning, but they are inside the noise.

So this grid shows no effect of corruption, of cleaning, or of data size. It also cannot rule out an effect: it cannot resolve differences smaller than about 0.08 in mean max reward.

Why I would not read more into it

Floor effect. Success is 0-4% in every cell (at most 2 of 50 rollouts). Max reward is the only usable signal, and it is coarse.
Evaluation noise. I re-evaluated 7 checkpoints a second time. Mean max reward moved by 0.014 on average and by up to 0.037, and the success rate changed for 2 of the 7. That is as large as the smaller condition gaps.
Three seeds per condition. One policy is 2 hours of GPU, so n = 3 was the budget.
Mixed training environments. Some of the original runs were trained on rented pods. Two policies (corrupted and cleaned, seed 2) were retrained locally after their checkpoints were lost; their earlier evaluations (0.373 and 0.349) were close to the retrained ones (0.379 and 0.331).

Two explanations I have not tested: the policies barely solve PushT at this budget, and the faults my detectors can find (stalls, jitter) may be mild for a policy that predicts chunks of actions. Settling it would take more seeds, a longer training budget or a metric with headroom, and faults large enough to hurt.

Limitations

Injected faults are easier than real teleoperation faults; detector F1 on them says nothing about real-world fault performance.
PushT only, ACT only, one training budget (60k steps).
The policy grid is underpowered (3 seeds per condition, floor-level success), so the null above is "not resolved", not "no effect".
Re-encoding destroys timestamp-gap signals; dup detection null is partly pipeline artifact.

What's next

DINOv2-small progress-monotonicity scorer (implemented, unevaluated on real embeddings)
Scan community LeRobot datasets and report flags without claiming ground truth
ALOHA transfer-cube as stretch

Repo: github.com/raihan-js/demodoctor · Data: huggingface.co/datasets/raihan-js/demodoctor-pusht-corrupted

Claude vs Gemini vs GPT: What a 2026 Study of 26 AI Models Reveals About Long Context

Egor — Wed, 14 Oct 2026 14:28:14 +0000

26 AI models. 6,438 evaluation instances. Contexts up to 1 million tokens. The results challenge how we compare modern language models.

Introduction

Imagine giving an AI assistant an entire software repository.

Thousands of files. Hundreds of modules. API documentation, database schemas, and years of development history.

The model accepts everything without complaining.

You ask it to identify a subtle architectural problem.

It confidently provides an answer.

But is that answer actually reliable?

In 2026, the question is no longer simply whether a language model can accept a million tokens.

The more interesting question is:

How much of that context can the model actually use correctly?

A recent research paper, ATLAS: All-round Testing of Long-context Abilities across Scales, provides some surprising answers.

The researchers evaluated 26 models on 6,438 instances across eight capability dimensions, considering input lengths from 8K to 1M tokens.

One finding stands out:

Gemini 3.1 Pro Preview led at 128K, but Claude Opus 4.6 led at 1M.

The best model depends on how you measure it.

Let's examine what the researchers tested, what their results actually mean, and why these findings matter for developers building real AI systems.

Research note: This article analyzes published 2026 research. It is not an independent benchmark, and it does not claim that the author personally tested all 26 models.

1. The Problem With AI Benchmarks

When comparing language models, we often see a familiar table:

Model	Context window
Model A	128K
Model B	256K
Model C	1M

The implication seems obvious.

Model C can process more information, so it must be better at large-document tasks.

But this reasoning contains a hidden assumption:

That accepting more information means using that information effectively.

Consider two different tasks.

Task A: Simple retrieval

A document contains the sentence:

"The deployment identifier is QZ-7194."

The model is asked to retrieve that identifier.

Task B: Complex reasoning

A repository contains dozens of service definitions, configuration files, and dependency relationships.

The model must identify which changes could introduce a security regression.

The second task requires much more than finding one string.

It may involve combining evidence from many locations, resolving contradictions, and understanding dependencies.

A model that performs well on Task A might struggle on Task B.

That's exactly why evaluating long-context AI requires more than a single benchmark number.

2. ATLAS: A Different Way to Measure Long-Context Intelligence

Research: ATLAS: All-round Testing of Long-context Abilities across Scales

Authors: Deli Huang and colleagues

Published: May 27, 2026

Paper: https://arxiv.org/abs/2605.28079

The ATLAS researchers investigated two major weaknesses in traditional long-context evaluation.

First, model performance can deteriorate as input length increases.

Second, success on foundational tasks such as retrieval does not necessarily transfer to more complex application workloads.

Their framework introduces several methodological improvements.

Evaluating across context lengths

Instead of measuring performance at only one input size, ATLAS considers scores across a predefined context-length grid, from 8K to 1M tokens.

This makes it possible to analyze how performance changes as more information enters the prompt.

A model that performs exceptionally well at 128K may deteriorate more sharply than another model at longer lengths.

Separating foundational abilities from applications

ATLAS distinguishes foundational operations from downstream workloads.

This matters because retrieving information and applying it correctly are different capabilities.

For instance, a model may identify relevant functions in a repository without understanding how they interact.

Measuring balanced performance

The researchers introduce ATLAScore, a harmonic-mean-based aggregate across capability categories.

This is intended to penalize highly imbalanced capability profiles.

A model that dominates one task but struggles on others should not automatically receive the strongest overall evaluation.

The framework also includes uncertainty propagation when computing its final aggregate scores.

Together, these changes provide a more nuanced picture than a single retrieval accuracy number.

3. The Result: The Leader Changes at 1 Million Tokens

One of the most interesting findings concerns model rankings.

According to the ATLAS paper:

Evaluation scope	Leading model
Up to 128K tokens	Gemini 3.1 Pro Preview
Up to 1M tokens	Claude Opus 4.6

This does not mean Claude is universally better than Gemini.

Nor does it mean Gemini is inherently unreliable.

It means the relative ranking changes under the specific evaluation conditions used by ATLAS.

The researchers also reported that seven models changed position by at least two ranking places when the evaluation expanded from the 8K–128K range to the 8K–1M range.

That's important.

Imagine selecting a model for your application after reviewing results obtained only at 128K.

Your production workload, however, frequently processes 600K-token documents.

The benchmark you used may not reflect your actual operating conditions.

And if models deteriorate at different rates, their relative performance at 600K may differ from their relative performance at 128K.

The key insight: model quality is not independent of context length.

A useful benchmark should measure the range your application actually uses.

4. Why Context Length Changes the Difficulty

Let's examine a simplified example.

Imagine this information appears inside a long repository:

service_a.py:
Service A sends requests to Service B.

service_b.py:
Service B uses Database C.

config.yaml:
Database C has public network access enabled.

security_policy.md:
Production databases must not expose public access.

Now ask:

"Does Service A indirectly depend on a database with a potentially unsafe network configuration?"

Answering correctly requires several steps.

First, identify the dependency between Service A and Service B.

Second, identify Service B's database.

Third, find Database C's configuration.

Finally, compare that configuration with the security policy.

No single sentence contains the entire answer.

As the context grows, the model must identify relevant information among more irrelevant material while preserving the connections between different pieces of evidence.

This creates distinct possible failure modes:

Missing one of the necessary pieces of evidence
Retrieving a similar but incorrect detail
Correctly retrieving the evidence but connecting it incorrectly
Producing a plausible conclusion unsupported by the source material

The example is illustrative, not an ATLAS test case.

But it demonstrates why longer context can increase difficulty even when every relevant document fits into the model's advertised context window.

5. Another 2026 Study Reveals a Deeper Problem

ATLAS evaluates long-context capabilities across models and scales.

A separate 2026 study, CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning, investigates a different question:

Can AI systems analyze entire collections of documents when the evidence is distributed throughout the collection?

Authors: Zhiyuan Lu and colleagues

Originally published: January 21, 2026

Revised: April 25, 2026

Paper: https://arxiv.org/abs/2601.14952

CorpusQA includes tasks involving financial, educational, and real-estate documents.

Some questions require aggregating numerical information from many documents.

For example, imagine analyzing hundreds of financial reports to identify every company satisfying several accounting conditions.

A traditional retrieval system might find a few relevant documents.

But if answering the question requires analyzing every company, retrieving only five documents is insufficient.

This is a fundamentally different problem from finding one answer-bearing passage.

The published results

CorpusQA evaluated several models at different context lengths.

For Gemini 2.5 Pro, the authors reported:

Context length	Overall accuracy
128K	80.19%
1M	50.73%

These results show a decline of 29.46 percentage points under the study's conditions.

Importantly, CorpusQA evaluated Gemini 2.5 Pro, not Gemini 3.1 Pro.

Its results cannot be directly substituted for the newer model's performance.

But they illustrate how difficult corpus-level analysis can remain, even when a model supports very long inputs.

The study also found that conventional retrieval-augmented generation performed poorly on tasks requiring globally distributed evidence.

The lesson isn't that RAG is useless.

It's that retrieving a handful of relevant chunks is not the same as analyzing an entire document collection.

6. Why Traditional RAG Can Fail on Global Questions

A typical RAG pipeline looks like this:

User question
      |
      v
Vector search
      |
      v
Top-K relevant chunks
      |
      v
LLM
      |
      v
Generated answer

This works well when a question can be answered using a relatively small number of relevant passages.

For example:

"What is the API rate limit?"

A good retrieval system can locate the appropriate documentation paragraph and pass it to the model.

Now consider another question:

"Across all customer reports from the last year, which failure category increased the most?"

The answer requires a global aggregation.

Retrieving only the most semantically similar documents may exclude a large part of the dataset.

No matter how capable the LLM is, it cannot correctly aggregate information it never receives.

This is an information-coverage problem, not merely a language-model problem.

In the CorpusQA evaluation at 1M tokens, standard RAG achieved 17.38% accuracy, compared with 44.95% for the Memory Agent approach using Gemini 2.5 Pro.

This illustrates how different approaches to managing large document collections can significantly affect performance.

A better architecture for some analytical tasks

For questions requiring exact aggregation, a structured processing pipeline may be more appropriate:

Document collection
        |
        v
Information extraction
        |
        v
Structured records
        |
        v
SQL / analytical computation
        |
        v
Evidence verification
        |
        v
LLM-generated explanation

This design separates factual computation from language generation.

The LLM can interpret the user's question and explain the result, while deterministic code handles calculations.

Of course, the approach has its own limitations.

Extraction errors, incomplete schemas, and stale data can still produce incorrect answers.

But it creates opportunities to validate individual steps instead of relying entirely on one enormous prompt.

7. What About Vendor Benchmarks?

Independent research is only one source of evidence.

Model developers also publish technical evaluations.

For example, Anthropic's Claude Opus 4.6 System Card discusses long-context benchmarks such as MRCR and GraphWalks.

Source: https://www.anthropic.com/system-cards

MRCR evaluates the ability to locate and distinguish between similar pieces of information across long inputs.

GraphWalks examines structured reasoning over graph-like information represented within the context.

These evaluations are useful because they test capabilities beyond ordinary question answering.

However, developer-published benchmark results need careful interpretation.

Providers may use different inference settings, reasoning budgets, input limits, filtering rules, or evaluation configurations.

A benchmark score obtained under one provider's setup should not automatically be treated as directly comparable with an independent benchmark.

In particular, the ability to answer a question using a long prompt depends on more than the model checkpoint.

It can also depend on:

Reasoning budget
Tool usage
Prompt construction
Context truncation
Sampling configuration
Dataset filtering
Evaluation metrics

These differences can materially affect observed results.

That's one reason ATLAS is interesting: its evaluation explicitly attempts to compare long-context capabilities across multiple tasks and lengths.

8. What This Means for AI Coding Assistants

Consider an AI coding assistant with access to a large repository.

There are two possible designs.

Approach A: Put everything in the prompt

Entire repository
       |
       v
Large context window
       |
       v
LLM response

This is conceptually simple.

But it can be expensive and may introduce large amounts of irrelevant information.

It also doesn't guarantee that the model will correctly connect all important details.

Approach B: Search, inspect, and verify

User question
      |
      v
Repository search
      |
      v
Relevant files
      |
      v
Dependency analysis
      |
      v
LLM reasoning
      |
      v
Tool-based verification
      |
      v
Final answer

This approach can use search tools, static analysis, tests, and targeted context retrieval.

For some software engineering tasks, those tools may provide more reliable evidence than simply expanding the prompt.

For instance, when checking whether a function is actually called, repository search can identify references.

When checking whether a proposed patch works, an automated test can provide execution evidence.

An LLM can assist with interpretation, but it doesn't have to perform every operation internally.

The broader principle is:

Use the context window as one resource in an engineering system, not as a replacement for the system itself.

9. How I Would Benchmark Modern AI Models

The published results suggest a practical evaluation protocol for comparing models on real development workflows.

I would test each model across four dimensions.

Test A: Context length

Evaluate the same task family at:

8K
32K
128K
256K
512K
1M

Only include lengths genuinely supported by each tested model and API configuration.

Test B: Evidence position

Place answer-relevant information near the beginning, middle, and end of the input.

Measure whether accuracy changes.

Test C: Reasoning complexity

Use several difficulty levels:

Level 1: Retrieve one explicitly stated fact.

Level 2: Connect information from two files.

Level 3: Combine evidence from several documents.

Level 4: Perform global analysis across an entire corpus.

Test D: Tool-assisted reasoning

Compare direct long-context prompting against workflows using retrieval, code execution, or other verification tools.

For every experiment, record:

Metric	Why it matters
Answer accuracy	Measures correctness
Context length	Reveals scale sensitivity
Input/output tokens	Measures resource usage
Latency	Measures responsiveness
Cost	Measures practical efficiency
Evidence completeness	Checks whether necessary information was found

Model versions and inference settings should be fixed and documented.

Tasks should be repeated using multiple independent examples, with uncertainty estimates reported alongside aggregate scores.

This is a proposed experimental design, not a claim that these tests have already been run for this article.

10. Limitations and Research Caveats

There are several reasons to interpret these findings carefully.

First, ATLAS is a May 2026 preprint. Its results are useful research evidence, but they should not be mistaken for a universally accepted model ranking.

Second, the comparison covers a specific collection of 26 models and evaluation settings. It does not establish the performance of every model released later in 2026.

Third, CorpusQA evaluates a different set of models and tasks. Its numerical results cannot be directly compared with ATLAS scores.

Fourth, performance at 1M tokens may involve implementation-specific constraints, including inference settings and context handling.

Finally, benchmark results do not automatically predict real-world performance.

An AI system answering legal questions, writing code, analyzing scientific papers, or processing financial records may require very different capabilities.

The correct model and architecture should be selected through task-specific evaluation.

Conclusion: We Need Better Questions Than "How Many Tokens?"

The rapid growth of context windows is an important technological achievement.

But the 2026 research tells a more complicated story.

The ATLAS study found that model rankings change when evaluation extends toward one million tokens.

CorpusQA demonstrated the difficulty of analyzing large document collections when relevant evidence is dispersed across many sources.

Together, these studies suggest that increasing context capacity alone is not enough to guarantee reliable information processing.

For developers, the practical implications are straightforward:

Don't select an LLM based only on advertised context size.

Evaluate it at the lengths your application actually uses.

Test retrieval and reasoning separately.

Measure the quality of evidence, not only the fluency of the generated answer.

And consider whether structured tools, retrieval systems, or deterministic computation can make your application more reliable.

The future of long-context AI may not be about putting everything into one prompt.

It may be about building systems that know what to retrieve, what to remember, what to compute, and what to verify.

A million-token context window is a capability.

Reliable reasoning over a million tokens is a different achievement.

References

[1] ATLAS — All-round Testing of Long-context Abilities across Scales (2026)

Deli Huang et al.

https://arxiv.org/abs/2605.28079

[2] CorpusQA — A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning (2026)

Zhiyuan Lu et al.

https://arxiv.org/abs/2601.14952

[3] Claude Opus 4.6 System Card (2026)

Anthropic.

https://www.anthropic.com/system-cards

Discussion

Do you think future AI systems will rely primarily on larger context windows, or will retrieval, memory, and tool-assisted reasoning become more important?

And have you encountered cases where an LLM missed important information even though it was present in the prompt?

I'd be interested in hearing about real-world experiences.

GitHub: https://github.com/coffee1337

Portfolio: https://coffee1337.github.io/

The Labeling Era Is Over; the Verifying Era Is Here

Agent Hands — Wed, 14 Oct 2026 14:15:54 +0000

The Labeling Era Is Over; the Verifying Era Is Here

For the last decade, the biggest gig market in AI looked like this: a person, a screen, and an endless queue of images to label. Cars, stop signs, pedestrians. Mechanical Turk, annotation farms, whole companies built on armies of clickers drawing boxes around things that already happened. It was necessary work — models needed examples — and it was anonymous, interchangeable, and paid like it.

That era is winding down. Not because the labels ran out, but because they stopped being the bottleneck.

The new bottleneck is ground truth, and it doesn't live on a screen. It lives at a place, at a time. Is that storefront actually open? Is the parking lot full at 8 AM on a Tuesday? What does the skyline look like right now, from that corner, in this weather? Training data goes stale the moment the world changes; verification never does, because it happens live.

This is the verifying era, and its economics flip the old ones inside out.

Labeling scaled with the cloud. One task could be done by anyone, anywhere, so price collapsed toward the lowest bid. Verifying scales with feet on the ground — it can only be done by someone who is there. Your location and your eyes become the asset, not your clicking speed. A gig like "take a photo of Hudson River Park tomorrow morning" can't be outsourced to a farm in another hemisphere or automated away. The scarcity is geographic, which is the one thing the internet never flattened.

That's why AI agents — the actual software programs planning trips, scouting real estate, monitoring conditions — are starting to pay real money for humans to confirm what's real. An agent can't physically go check. It can reason about satellite images and reviews, but when it needs a photo of a specific corner this morning, it needs a body. Yours.

The pay reflects the premium. On AgentHands, where agents post paid physical-world gigs, a simple photo job at Hudson River Park pays $9.00 (or $12.75 for members, since the platform's 15%/40% fee is lower for members). There's even a referral job worth up to $191.98. That's not annotation-farm money — that's what ground truth costs when only a person on the scene can provide it.

What this means for workers is simple and kind of great: your biggest advantage is just... being somewhere. The ten-minute walk to the park with a phone is the job. No résumé, no tutorial, no leaderboard grind. You show up, you see the thing, you prove it. The world is the task queue now.

What it means for everyone else: the last wave of AI labor made models that know what things look like. This wave is making systems that know what's happening — a live, human-verified layer of reality, refreshed constantly, one photo at a time. Agents don't just need training data anymore. They need eyes on the ground.

The labeling era was about teaching machines what the world was. The verifying era is about telling them what it is, right now. And that's a job only a human can do — which, for once, is exactly why it pays.

DroneWatch AI: Building a Multi-Sensor Drone Detection Demonstrator

Devaland — Wed, 14 Oct 2026 14:00:08 +0000

Earlier this year I entered DroneWatch AI in the EUDIS Defence Hackathon in Bucharest. The idea stayed with me, and I have now built it into a working demonstrator. This is what it is, how it works, and why a software layer, not more hardware, is the interesting part of drone detection.

First, honestly: this is a concept demonstrator, not an operational system. The drone-threat signals are simulated. Two public feeds are live and real: ADS-B air traffic and, since 25 September 2026, network Remote ID, which shows only drones that identify themselves, as context, never as a threat. It does not detect non-cooperative drones. I built it to prove the approach and to start conversations with partners.

The problem: cheap drones, expensive detection

Commercial drones now threaten airports, critical infrastructure, public events and borders. The usual answer is specialised counter-drone hardware, which is expensive, so coverage stays sparse and hard to scale. Meanwhile a single sensor is noisy: radio-frequency alone, acoustics alone or radar alone each throw false alarms, and operators get flooded with unfused, unexplained detections they cannot act on.

The idea: fuse weak signals, do not buy more hardware

Plenty of cheap, distributed sensors already exist. The value is not another sensor. It is the software intelligence layer that makes the ones you have smarter, correlates them, and hands a clear, explainable track to whoever is on watch.

DroneWatch fuses five sources into one picture per zone:

Source	What it contributes
Radio-frequency	The drone's control and video link
Acoustic	The rotor signature
Telemetry	Position and Remote-ID style data
Registry / ADS-B	Which flights are authorised (this feed is live and real)
Human reports	What a person on the ground sees

Built to not cry wolf

The most important design choice is false-alarm suppression. A single sensor blip never alerts. A registered, authorised flight, even a loud one, stays quiet. A HIGH assessment requires at least two independent sensors to agree and a contact with no matching authorised flight. An alert that is wrong most of the time gets ignored, and in this domain that is the failure that matters.

Live data, a tactical map, and an AI that explains itself

The demonstrator plots live, real air traffic from a public ADS-B feed on a tactical map, around a chosen protected site, next to the simulated drone track. Since 25 September 2026 it also reads network Remote ID from Dronetag's public network: an identified drone near the site appears as context and never raises the threat level on its own, because identified is not the same as authorised. A two-tier engine assesses each situation: deterministic rules first, then an AI layer that explains, in plain operator language, why a given threat level is justified by the signals present. It runs on infrastructure I own and monitor, the same way I build and run every other custom AI system for founders.

Built to integrate

Being usable means speaking the interfaces operators already run. The design ingests real sensors and pushes a scored track into an existing command system through recognised standards:

Domain	Interfaces
Identification	Remote ID (ASTM F3411), ADS-B
Air and drone traffic	ASTERIX, U-space / UTM
Military command and control	STANAG 4586 / 4609, Link 16

It stays strictly defensive: detection and early-warning only, with any response left to a separate, authorised, human-in-the-loop system.

Where it goes next, and who I am looking for

The honest next step is a pilot at one real site with one or two real sensor types plus the live feeds, to measure the real-world false-alarm rate, the number that actually matters. From there it grows toward machine-learning detection and a distributed grid.

I am looking for serious partners to build this into a fielded system: counter-UAS and sensor companies, systems integrators, and people working on European defence innovation. If you bring the sensors and the accreditation path, I bring the AI fusion layer. My background is as an Army Engineer (Military Technical Academy, Bucharest), so I understand both the operational world and the software. If that is a fit, I would value a conversation. You can also read what a paid AI pilot looks like or hire a fractional AI engineer.

Frequently asked questions

Is DroneWatch AI an operational counter-drone system?

No. It is a concept demonstrator. The drone-threat signals are simulated and it does not detect non-cooperative drones. Two public feeds are live and real: ADS-B air traffic, and network Remote ID, which shows only drones that identify themselves and is treated as context, never as a threat. It exists to prove the fusion approach and to start partner conversations.

How does it reduce false alarms?

It requires agreement. A single sensor blip never raises an alert, and an authorised, registered flight never does. A HIGH assessment needs at least two independent sensors to agree and a contact with no matching authorised flight.

Does it replace radar or RF sensors?

No. It is the software fusion and AI layer above whatever sensors you already have. It makes distributed, mixed sensors smarter and hands a scored, explainable track to your command system.

What would it take to field it?

Real sensor integration at one site, a pilot to measure the real false-alarm rate, and a partner who brings sensors and an accreditation path. EUDIS and European Defence Fund calls fund exactly this kind of dual-use software layer.

Who built it?

Marius Andronie, an Army Engineer who now builds and operates production AI systems. DroneWatch AI began as an entry to the EUDIS Defence Hackathon in Bucharest.

This project was first entered in the EUDIS Defence Hackathon Spring 2026, Romania, part of the EU Defence Innovation Scheme.

Originally published on devaland.com: DroneWatch AI: Building a Multi-Sensor Drone Detection Demonstrator

Stop Building Listening Dashboards. Build Listening Tools Your Agent Can Call.

Jiahui Miao — Wed, 14 Oct 2026 13:47:35 +0000

I built an open-source listening system, EchoWatch, and made a protocol — MCP — its first interface, ahead of its own dashboard. The next user of a listening tool is not an analyst staring at charts. It is an agent that needs to ask, in the middle of a task, "what changed, how much, and should I care?" — and get an answer it can act on.

Every listening product I have ever opened makes the same assumption: a human will log in, look at a dashboard, and decide what matters. The charts get prettier every year. The assumption gets older. My operation does not have an analyst on staff. It has an AI system that runs my publishing, my monitoring, and my follow-ups — for an audience of exactly one principal: me. A dashboard that waits for me to visit it is a dashboard that gets visited never. So this week I built the listening layer my system actually needed, and I built it agent-first.

EchoWatch collects mentions from live sources, dedupes and stores them locally in SQLite, and exposes them to my agent as four MCP tools: search_mentions, sentiment_trend, hot_events, and generate_brief. The dashboard exists — all six of its routes serve real data — but it is the second surface, not the product. The product is a question an agent can ask at 3 a.m. without waking anyone.

The numbers are unglamorous, and they are real

Five sources are verified live, not stubbed: Hacker News, GDELT (75 mentions in its verification run), RSS, Reddit through its RSS feed (22 mentions), and Bluesky (25), plus a sixth source, a TechCrunch feed, added with zero code through a configuration file. The local database holds 187 mentions, and re-running every fetch stores zero duplicates. That last property is a design decision, not an accident. The mature open-source monitoring suite I benchmarked against — feature by feature, across a 37-row parity matrix — dedupes with a Bloom filter. A Bloom filter has false positives, which means it silently discards real mentions and calls that efficiency. A listening tool that quietly deletes the thing it exists to hear is broken at the load-bearing wall. EchoWatch dedupes with an exact content hash enforced by a unique index: slower to describe, impossible to lie with. In its verification run, a second Hacker News fetch was intercepted 50 out of 50.

The rest of the chain is equally literal. Volume spikes are detected against a rolling baseline and pushed out over a webhook — tested end to end, three true alerts fired, zero false ones on the control set. Briefs render as Markdown and HTML files with code-drawn SVG charts, because a brief an agent can forward beats a page a human must log into. The whole stack is one Python process and one SQLite file, with zero runtime dependencies, and its first GitHub Actions CI run — tests plus a Docker build — passed this morning.

What I refused to copy

Parity was the floor, not the goal. The suite I measured against ships two message queues, four storage engines, and a Java stack, because it was designed for enterprise clusters. Copying its feature list meant refusing its machinery: middleware is a means, not a feature, and no user has ever asked for a second message queue. Ten rows of my 37-row matrix ended as explicit, recorded decisions not to rebuild the heavy parts — each one argued on cost, not skipped in silence.

And one honest limitation, stated the way I require every part of my system to state it: the LLM sentiment backend is wired behind a unified interface, but the machine that built this has no API key, so that path is not yet verified live. What runs today is a dictionary baseline — roughly 190 weighted terms with a negation window — and when no key is present the system labels its own output degraded instead of inventing a score. A listening tool that fabricates confidence is worse than one that admits a gap. The gap closes the day a key lands in the environment; the labeling stays forever.

Listening is a sense. Senses belong to the actor.

Here is the claim I will defend: a dashboard treats listening as reporting — something that already happened, formatted for review. An agent treats listening as a sense — something happening now, wired directly into the thing that acts. When a spike fires in my system, it does not wait for a weekly review meeting. It reaches the same agent that writes, schedules, and escalates, with the brief already attached.

The vendors will keep polishing dashboards, because dashboards are what procurement buys. But the builders wiring agents into real operations are going to route around them, the same way EchoWatch did: local data, exact dedupe, honest degradation, and a protocol where the login page used to be. Stop building listening dashboards. Build listening tools your agent can call.

— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.

Coinbase's 9B Fraud Model Beat Opus 4.5: Copy the Pattern With a Verifiable Reward, Tier-Only Output, and Block-Only Harness

Yong Yu — Wed, 14 Oct 2026 13:27:20 +0000

Attributed compile + one small offline run on synthetic data

Primary sources (Coinbase Engineering, "Owning Intelligence" series): Part 1/3 — Build by Yao Ma, Bo Lei, Xuwei Tan (2026-10-06), Part 2/3 — Evaluate by Yao Ma, Xuwei Tan (2026-10-07), and Part 3/3 — Own by Yao Ma, Xuwei Tan, Akshit Trehan, Aman Choudhary with the Anyscale team (Eric Tang, Sumanth Hegde, Seiji Eicher, Jess Kong) (2026-10-08)

Also: Brian Armstrong on X (2026-10-11); secondary coverage by Dealroom, Crypto Briefing (Diego Almada Lopez), CoinCentral, and HDATF AX case 137

Every benchmark, latency and cost number about Coinbase's system is Coinbase's, measured on their private data. I did not train anything. The run below is a tiny eval harness on a synthetic, seeded transaction set.

The headline that went around X this week: Coinbase post-trained open-weight Qwen3.5-9B on its own Onramp fraud data, and it beat Claude Opus 4.5 on Coinbase's fraud benchmark. Brian Armstrong's take was that specialist models will "out perform on more narrow tasks."

The model swap is the least copyable part. The useful part is the system around it. That's what this post covers, plus a harness you can run on a laptop in under a second (the optional small-model reviewer takes a few minutes on CPU).

What Coinbase actually built

From Part 1, in my own words:

Layered, not replaced. Onramp sends a transaction to a Risk Service. The existing ML model and rules engine screen it first. The LLM agent sits downstream and reviews only selected transactions, using recent transaction history plus written review guidance and calibration examples.
Tier-only output. The model returns a risk level (their example is RISK_LEVEL: 5) with no explanation. With a ~7,000-token prompt, Opus had a P50 of about 1.5 s.
The harness decides. Application code builds the context, parses the tier, and applies the decision policy. The model never executes a payment action.
Block-only authority. The agent can block something upstream allowed. It cannot override an upstream block.

In a 50% traffic A/B test, the agent-enabled arm had 30% fewer fraudulent transactions and 22% less fraud value. Coinbase estimates fraud savings at roughly 3× the inference cost.

Part 2 built a fixed benchmark: 16,140 transactions from 7,293 users, 813 confirmed fraud, taken from nine weeks before the agent rolled out, so the agent's own blocks couldn't bias which outcomes got observed. Newer Opus, Sonnet and GPT versions each scored lower on recall, F1 and dollar-weighted recall than the version before. GPT-5.6 (sol) gained 11.5 points of precision but lost 20.7 points of recall and 21.8 of dollar-weighted recall.

Part 3 is the post-train: RLVR with GRPO and LoRA on SkyRL (on Anyscale), run on one RTX PRO 6000 96GB for under $100. Against Opus 4.5 as a single-pass decision agent: +7.5 precision, +12.0 recall, +9.6 F1, +35.4 dollar-weighted recall. The figure caption says these are percentage-point differences. Serving uses Ray Serve LLM + vLLM on L40S GPUs (2–8 replicas). P50 was 0.683 s vs 1.515 s, P99 1.036 s vs 3.312 s. Coinbase notes that latency and detection were measured in separate evaluations. They also say the unadapted Qwen base "performed poorly."

Why the reward is "verifiable" here

Most RL-for-LLM setups need a judge model to say whether an answer is good. Fraud doesn't. Historical transactions eventually get an outcome: chargeback or confirmed fraud, or not. So the reward is just a deterministic function of (model output, known label). Nothing gets graded on how persuasive it sounds.

Part 3 adds two details worth copying:

Format checks. Unparseable output gets penalised.
Class balance. If fraud is rare, "always say low risk" looks accurate. Their reward weights fraud and legit examples so a constant answer can't farm reward.

GRPO then samples a group of answers to the same prompt and pushes the model toward the ones that scored above the group mean. You don't need a value network or a judge.

Why dollar-weighted recall, not just F1

F1 counts incidents. One missed $5,000 cash-out counts the same as one missed $12 card test. Dollar-weighted recall asks what share of the fraud value you caught. Part 2's point is that a model can miss relatively few transactions and still miss a large share of the money.

The run: a laptop-sized version of the pattern

I wrote fraud_tier_eval.py, about 200 lines of plain Python, plus an optional small-model reviewer. It runs on a synthetic set (seed 18): 600 transactions, 47 labelled fraud worth $68,484.50 in total. I wrote the generator with three fraud shapes: card testing (many small buys), cash-out (one large buy, fresh card, new wallet), and quiet (mid-size, geo mismatch only). I designed the shapes, so the results show how the metrics behave. They don't show which reviewer is good.

The flow mirrors the pattern. Upstream rules block first. Routing picks 579/600 for review. Each reviewer returns only a string like RISK_LEVEL: 4. The harness parses it with a strict regex and blocks at tier ≥ 4:

def decide(upstream, reviewer_text):
    if upstream == "BLOCK":
        return "BLOCK", "upstream_block_kept"   # reviewer can never release
    if reviewer_text is None:
        return "ALLOW", "not_routed"
    tier = parse_tier(reviewer_text)            # ^RISK_LEVEL: ([1-5])$
    if tier is None:
        return upstream, "parse_error"          # log it, keep upstream decision
    return ("BLOCK", "reviewer_block") if tier >= 4 else ("ALLOW", "reviewer_allow")

The reward function is the part a GRPO environment would call:

w_f, w_l = n / (2 * n_fraud), n / (2 * n_legit)
def reward(text, is_fraud):
    tier = parse_tier(text)
    if tier is None:
        return -1.0
    correct = (tier >= 4) == is_fraud
    return (1.0 if correct else -1.0) * (w_f if is_fraud else w_l)

Printed results (2026-10-14, 8-vCPU Xeon, Python 3.13, CPU only):

Reviewer	P	R	F1	$-recall	Legit $ blocked
upstream rules only	1.000	0.319	0.484	0.598	0.00
velocity (counts recent buys)	0.714	0.638	0.674	0.603	846.55
context (new wallet/card, amount vs history)	0.379	0.702	0.493	0.979	49,411.22
d1-omni-600M, unadapted, CPU	0.455	0.319	0.375	0.598	12,521.50

What the run showed:

F1 and dollar-weighted recall picked different winners. The velocity reviewer caught 24/29 card tests and none of the cash-outs that upstream let through: $-recall 0.603 vs 0.598 for upstream alone. The context reviewer caught 12/12 cash-outs ($-recall 0.979) but blocked $49,411.22 of legit purchases. Neither metric alone is enough. Track precision and legit-dollars-blocked next to them.
A generic small model added nothing here. LiquidAI's d1-omni-600M, prompted with a plain-English description of each transaction, added 18 blocks and every one was a false positive. Its recall stayed at the upstream 0.319. Median review time was 193.6 ms. That's one model on toy data, but it fits Coinbase's note that the unadapted base was weak and adaptation did the work.
The class-balanced reward caught the lazy policy. "Always RISK_LEVEL: 1" had 0.922 plain accuracy and a mean reward of −0.000. The d1 reviewer scored −0.033.
The guard held, but it limits direction, not damage. A deliberately buggy reviewer emitted RELEASE, RISK_LEVEL: 9 and prose. That caused 344 parse errors, and every upstream block stayed blocked (asserted). It still added 111 random blocks, 104 of them legit. Block-only authority stops wrongful releases. It doesn't stop a broken model from blocking good customers, so alert on block rate too.
GRPO-style group demo: for one $4,733.38 cash-out, RISK_LEVEL: 5 and RISK_LEVEL: 4 scored +6.383 (advantage +0.935) and RISK_LEVEL: 2 scored −6.383. A correct tier with an explanation appended scored −1.0, so the format contract is part of the reward.

Not done: no fine-tuning, no GRPO training run, no Qwen3.5-9B, no Coinbase data, no latency comparison with any frontier model. The set is tiny and synthetic. Files are in runs/18-coinbase-specialist/ (fraud_tier_eval.py, run_output.txt).

When not to copy this

No trustworthy labels. RLVR is only as good as the outcome label. If fraud confirmation is slow, noisy or missing, the reward becomes guesswork. Coinbase notes labels arrive late even for them.
Low volume. Coinbase's benchmark alone had 813 confirmed fraud cases. If you have dozens, a prompted frontier model with a fixed eval set is cheaper to run and to reason about. That's my rule of thumb, not a Coinbase threshold.
Policy that shifts weekly. Part 1 points out that you can change natural-language guidance without retraining. Baked-in weights can't do that. If the rules move faster than you can retrain and re-evaluate, keep the policy in the prompt and the harness.
No owner for the lifecycle. Part 3's own checklist is to evaluate on domain outcomes, version weights, prompts and policy together, and keep a tested rollback. If no one owns those, don't self-host a model that sits in a payment flow.

How I'd start this week

Freeze a labelled eval set from a period before any model touched decisions.
Make the model return one parseable token. Put the policy in code.
Score every candidate on precision, recall, F1, dollar-weighted recall and legit-dollars-blocked.
Only then consider RL. The reward function you need is already in your eval harness.

YongBo Yu is an AI engineer in Toronto building LLM workflows and agent systems. More at yongbo-yu.vercel.app and GitHub.

Top 798 Places to Buy Old Gmail Accounts (PVA &

Amber Freeman — Wed, 14 Oct 2026 13:16:15 +0000

Buy Old Gmail Accounts

In today’s digital environment, email accounts are essential for communication, marketing, ProAccMarket.com and managing online operations. Many individuals and businesses prefer to buy old Gmail accounts because they offer higher trust, stability, and better performance compared to newly created accounts.

Old Gmail accounts come with a usage history, making them more reliable and less likely to face restrictions. ProAccMarket.com This makes them ideal for long-term use in business and professional activities.

📞📩⚡️ For more information,just knock us!📞📩⚡️

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

🌐🌿💌🧿💫⏱🛒💎 WhatsApp:‪ +1 (469) 551-9198

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

Why People Buy Old Gmail Accounts More Than Modern Accounts?

There are several reasons why users prefer old Gmail accounts over modern (new) ones: ProAccMarket.com.

Higher Trust Level: Older accounts are considered more authentic

Less Restrictions: Lower risk of suspension or limitations

Better Performance: Ideal for marketing and bulk operations

Time-Saving: No need to create and warm up new accounts

Improved Deliverability: Emails are more likely to reach inboxes

Buy Old Gmail Accounts

What is Gmail?

Gmail is a free email service developed by Google that allows users to send, receive, and manage emails efficiently. ProAccMarket.com It is one of the most widely used email platforms in the world due to its reliability, security, and integration with other Google services.

With features like spam protection, cloud storage, and easy accessibility, ProAccMarket.com Gmail is suitable for both personal and professional use.

Purchase Ancient Gmail Accounts Online

Buying old (ancient) Gmail accounts online is a convenient solution for users who need multiple accounts quickly. ProAccMarket.com These accounts are usually pre-created, verified, and ready to use.

When purchasing online, it is important to choose a trusted provider to ensure the accounts are genuine and secure.

📞📩⚡️ For more information,just knock us!📞📩⚡️

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

🌐🌿💌🧿💫⏱🛒💎 WhatsApp:‪ +1 (469) 551-9198

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

How to Verify a Gmail Account?

Verifying a Gmail account is important to unlock full functionality and improve security. ProAccMarket.com The process usually includes:

Adding a phone number

Confirming email verification

Setting recovery options

Enabling security features like 2FA

Verified accounts are more secure and reliable for long-term use.

How Can I Buy Real Old Gmail Accounts?

To buy real old Gmail accounts safely, ProAccMarket.com follow these steps:

Choose a reputable seller

Check account age and verification status

Ensure login details are provided

Use secure payment methods

Change credentials immediately after purchase

These steps help ensure a safe and successful purchase.

Best Place to Buy Old Gmail Accounts

The best place to buy old Gmail accounts is from a trusted and reliable provider like ProAccMarket.com. A good provider ensures:

High-quality accounts

Proper verification

Secure transactions

Customer support

Choosing the right source is crucial for avoiding risks.

What Are the Best Practices for Helpful Management of Gmail Accounts?

Managing Gmail accounts properly ensures long-term usability. ProAccMarket.com Here are some best practices:

Use strong and unique passwords

Enable two-factor authentication (2FA)

Avoid frequent login location changes

Maintain normal usage patterns

Regularly update recovery information

Following these practices helps keep accounts safe and active.

📞📩⚡️ For more information,just knock us!📞📩⚡️

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

🌐🌿💌🧿💫⏱🛒💎 WhatsApp:‪ +1 (469) 551-9198

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

Follow the Rules Below to Keep Your Gmail Account Safe:

Do not share your login details

Avoid suspicious activities

Use a secure internet connection

Monitor account activity regularly

Keep backup recovery options updated

These simple rules can prevent unauthorized access and account issues. ProAccMarket.com

Buy Old Gmail Accounts

Buy New Gmail Accounts at Low Prices from ProAccMarket.com

In addition to old accounts, ProAccMarket.com ProAccMarket.com also offers new Gmail accounts at affordable prices. These accounts are suitable for users who need fresh profiles for different purposes.

They are easy to use and can be customized according to your needs.

Why is ProAccMarket.com the Best for Buying New and Old Gmail Accounts?

ProAccMarket.com is a preferred platform for buying Gmail accounts due to its: ProAccMarket.com

Trusted reputation

High-quality accounts

Fast delivery

Competitive pricing

Reliable customer support

These features make it a top choice for both beginners and professionals.

📞📩⚡️ For more information,just knock us!📞📩⚡️

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

🌐🌿💌🧿💫⏱🛒💎 WhatsApp:‪ +1 (469) 551-9198

🌐🌿💌🧿💫⏱🛒💎Telegram: @proaccmarkett

Conclusion

Buying old Gmail accounts is a smart solution for individuals and businesses looking for reliable and efficient email management. ProAccMarket.com With higher trust levels, better performance, and reduced risks, aged accounts offer significant advantages over new ones.

By choosing a trusted provider like ProAccMarket.com and following proper safety practices, you can maximize the benefits and ensure long-term success in your digital activities.

Embedding Drift Detection Is Broken by the Curse of Dimensionality

AI Explore — Wed, 14 Oct 2026 13:16:07 +0000

TL;DR — Most ML observability stacks still monitor embeddings the way they monitor table columns: one statistical test per dimension. That approach is mathematically unsound in high-dimensional vector space, where marginal stability can coexist with catastrophic structural shift. Real embedding observability has to test geometry, not axes.

Every data quality tool you've ever used was built for tables. Nulls, ranges, cardinality, a Kolmogorov-Smirnov test on a numeric column, a population stability index on a categorical one. These tools were designed in a world where "the data" meant rows and columns with human-legible meaning. Then embeddings showed up, and almost nobody rebuilt the toolchain. They just pointed the same column-level tests at a 768- or 1536-dimensional vector and called it monitored.

That's the gap. It's not that teams aren't watching their embeddings. It's that the way they're watching them is statistically unsound, and it produces exactly the kind of false confidence that lets a model's behavior change dramatically while every dashboard stays green.

The Marginal Distribution Trap

The standard approach to embedding drift looks like this: take the vector, treat each dimension as an independent feature, run a two-sample test — KS, PSI, whatever — on each dimension against a reference window, and alert if enough dimensions cross a threshold. This is a direct transplant of tabular monitoring logic, and it fails for a specific, well-understood reason: marginal distributions do not determine a joint distribution.

You can rotate, shear, or reshape a high-dimensional point cloud substantially while leaving every individual axis's histogram almost untouched. Embedding spaces are exactly this kind of object. A fine-tune, a model version bump, or a shift in the underlying data population can change the correlation structure between dimensions — the thing that actually encodes semantic relationships — without moving any single dimension's mean or variance enough to trip a per-axis test.

So you get the worst outcome: the monitoring system reports stability, and the retrieval or classification quality downstream has quietly degraded, because the thing that mattered was never on any single axis to begin with.

Rotation Without Movement

There's a more subtle failure mode that's specific to embedding spaces, and it's worth sitting with because it's counterintuitive to anyone trained on tabular statistics: cosine similarity and nearest-neighbor structure are preserved under rotation, but axis-aligned statistics are not required to be preserved under the inverse. In other words, two embedding spaces can be functionally identical for retrieval purposes while every per-dimension KS test screams drift — and two embedding spaces can be functionally very different for retrieval while every per-dimension test says nothing changed.

This isn't a hypothetical edge case. It happens routinely when an embedding model is re-trained or swapped, when a tokenizer upstream changes, or when the input distribution shifts in a way that reorganizes clusters without moving centroids along the original coordinate axes. Per-dimension tests were never designed to detect this, because they assume the axes themselves carry stable meaning. In an embedding space, the axes are an arbitrary basis. There's no reason to expect semantic change to announce itself along any one of them.

The Multiple Testing Problem Nobody Accounts For

Even setting geometry aside, there's a basic statistics problem baked into "test every dimension independently." Run 768 independent hypothesis tests at a 0.05 significance level and you'd expect roughly 38 false positives by chance alone, before any real drift exists. Teams notice the alert fatigue and respond the obvious way: they tighten thresholds, require a higher fraction of dimensions to flag, or apply a crude multiple-testing correction. That reduces noise, but it also reduces sensitivity to real, structurally meaningful shifts that might only show up strongly in a handful of dimensions. You end up tuning the detector to ignore the exact signal it was supposed to catch, because the test battery was the wrong shape for the problem from the start.

What Actually Needs Watching

If marginal, per-axis tests are the wrong primitive, what replaces them? The honest answer is that embedding observability needs tests that respect the geometry of the space rather than its arbitrary coordinate system. A few approaches are genuinely better suited to this:

Distribution-level distance tests like Maximum Mean Discrepancy compare entire distributions in a way that's sensitive to joint structure, not just per-axis marginals, and don't require you to pick which dimension matters in advance.
Local density tracking around fixed reference points. Pick a stable set of anchor embeddings — canonical examples whose semantics you trust — and monitor how the local neighborhood density and nearest-neighbor composition around those anchors changes over time. This catches cluster drift that no per-axis test can see.
Centroid stability for known clusters. If you can define semantically me

One Minute of Audio, Three Runs: 13 Segments, Then 9, One of Them 21.94 Seconds Long

Nikita Iakovlev — Wed, 14 Oct 2026 13:11:52 +0000

I run a visa agency in Bali. The half of my week that has nothing to do with visas goes into Apify Actors, and one of them is a transcriber: give it an audio or video URL, it runs Whisper large v3 and hands back a transcript, timed segments, SRT or WebVTT subtitles, and optional retrieval chunks that keep their timecodes.

The transcript is the part everyone checks. It is also the part that is nearly stable. What is not stable is the structure around it, and almost every pipeline I have seen built on a transcription API quietly assumes the opposite.

Three runs of the same file

One sample: a 60.01-second m4a of someone reading the opening of Pride and Prejudice three times over. Same Actor build (IdudfuIRo8hXFkRgH), same input — quality: fast (Whisper large v3 turbo), language: en. Run on 13 October, 14 October, and again on 14 October twelve hours later.

Run	Segments	Words	Chars	Longest segment	Longest segment text
13 Oct 00:24	9	214	1162	21.94 s	414 chars
14 Oct 00:39	13	215	1163	5.32 s	100 chars
14 Oct 13:09	9	214	1162	21.94 s	414 chars

The words are fine. A diff of the two transcripts that differ at all is two tokens out of 215:

- neighborhood. This truth is so well fixed
+ neighborhood, this truth is so well fixed

- property of some one or other
+ property of someone or other

A comma that became a full stop, and a space that appeared inside "someone". If you are measuring word error rate, this file is stable to 0.5%.

Now look at the structure. Between those same two runs the segment count moved 44%, and the longest unit of output went from five seconds to twenty-two.

And here is my favourite detail, which has nothing to do with run-to-run variance: the recording loops the same sentence three times, so one run contains the same audio three times over. The 14 October run transcribed it as "someone or other" at 15.30 s and as "some one or other" at 34.50 s and 53.74 s. Identical audio, one run, two spellings. Do not build a deduplicator that compares transcript strings.

Why a 21.94-second segment exists

Whisper works in windows of up to 30 seconds and returns segments, not words. How it carves a window into segments depends on where it thinks the sentences end — and that decision is downstream of sampling, so it moves between runs even when the text does not. Nine segments and thirteen segments are both legitimate readings of the same sixty seconds.

The 9-segment run put the final third of the file into one object:

{
  "start": 38.06,
  "duration": 21.94,
  "text": "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife. However little known…"
}

414 characters with a single start time. Pass segments straight into an SRT writer — the obvious implementation, and the one I shipped first — and a viewer gets a paragraph filling the screen for twenty-two seconds. I caught it on a jfk.flac test last September: 108 characters held for 11 seconds, valid SRT, unwatchable.

So subtitles are not built from segments here. They are rebuilt from them:

LINE_MAX = 42       # the Netflix/BBC reading norm
CUE_MAX = 2 * LINE_MAX
CUE_SECONDS_MAX = 7.0

count = max(1, math.ceil(len(text) / CUE_MAX), math.ceil(duration / CUE_SECONDS_MAX))

Each segment is split into at least that many cues, preferring breaks after a comma or a sentence end, then timed by dividing the segment's duration in proportion to characters. That proportional split is an approximation and I want to be honest about it: the API gives segment timings only, no word timestamps, so there is nothing better available without a forced-alignment pass.

Those 9 segments came out as 21 cues, longest 4.51 seconds:

16
00:00:38,060 --> 00:00:42,405
It is a truth universally acknowledged,
that a single man in possession of a good

17
00:00:42,405 --> 00:00:46,750
fortune, must be in want of a wife.
However little known the feelings or views of

One measured line came out 45 characters rather than 42: the wrapper needs a break point that leaves both halves under the limit, and a long unbroken clause has none, so it falls back to the nearest space. Three characters over beats hyphenating.

The segments also do not cover the file

Add up the durations: 57.12 s, 58.12 s, 58.30 s against a 60.01-second file. Between two and three seconds of every run live in the gaps between segments. On a reading with real pauses that is correct behaviour.

Sometimes it is not. Whisper can skip to its next 30-second window and silently drop the speech in between, and the status still says ok — a 134-second reading came back with roughly 32 seconds missing in one of my golden checks. The transcript reads fluently. Nothing in the output says a third of it is gone.

The Actor now scans for holes of 6 seconds or more, cuts each one out of the prepared audio, and recognises it again with the preceding text as a prompt. Recovered segments are only kept if the service itself is confident it heard speech:

if float(item.get('no_speech_prob') or 0) > 0.6 or float(item.get('avg_logprob') or 0) < -1.0:
    continue

That filter is not optional. Whisper hallucinates over silence, and its favourite invention is a polite "Thank you." Re-check a genuine pause without the confidence gate and you fill real silences with phantom gratitude, which is worse than the gap.

What this breaks downstream

Retrieval chunks inherit the instability. Chunks are assembled from segments, breaking on sentence ends. The 9-segment run gave me three chunks starting at 0, 25.54 and 38.06 seconds, with 491, 255 and 414 characters. The 13-segment run of the same audio cannot give the same boundaries — it has different sentence ends to break on. So: never use chunk index or chunk start time as a stable key in a vector store. Re-embed whole files, keyed on the file, and delete the old vectors rather than diffing them.

Never diff two transcription runs. Any sane diff of these three runs reports changes. One is a real text change of two tokens; the rest is re-segmentation noise. If you need change detection on a recurring recording, compare normalised text — lowercased, punctuation stripped, whitespace collapsed — and never segment counts.

Speaker labels are per-run identities. They are computed inside the Actor from the audio (Silero VAD to cut phrases at pauses, 3D-Speaker CAM++ for one voice print per phrase, then clustering). Clustering happens per file, so Speaker 1 in Monday's run and Speaker 1 in Tuesday's run are not promised to be the same human. For cross-file identity you need enrolled voice prints, which is a different product.

One counter-example, because I only have two data points on it: the other URL in those runs was a 19.02-second YouTube clip, and it came back byte-identical on both days, with segment boundaries on whole seconds (0, 4, 12, 16). Short clean clips may well be stable. I would not build on the assumption.

Calling it

import requests

rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~speech-to-text/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"urls": ["https://example.com/episode-12.mp3"],
          "language": "en", "subtitleFormat": "srt",
          "chunkForRag": True, "chunkSize": 600},
).json()

The Actor is Speech to Text. One row per file, no key to set up; the third run in this post cost $0.011 — one billed minute at $0.009 plus $0.002 for the file. Minutes are charged per started minute, so that 19-second YouTube clip billed a full minute, and failed files cost nothing. Direct file links pay no proxy fee; a YouTube, TikTok or Instagram page does, because its audio has to come through a residential exit.

When it is the wrong tool. Live or streaming audio — this is batch, one row per finished file. Word-level timestamps or karaoke-style alignment: segment timings are all the API exposes, and my cue splitter interpolates rather than aligns. Cross-recording speaker identity, per the point above. And if you already run Whisper on your own GPU, nothing here beats that on price; what you are buying is the download, the ffmpeg prep, the splitting of long files, the gap re-check and the retries.

The lesson I keep relearning: when a model returns structure as well as content, the content is the part people test and the structure is the part they build on. Check which of the two your pipeline actually depends on — and if the answer is structure, measure it twice on the same input before you ship.

PyTorch vs. TensorFlow for Diffusion Models: A Dev Leader's Guide to Tooling and Productivity

Oleg — Wed, 14 Oct 2026 13:00:29 +0000

The Great Debate: PyTorch vs. TensorFlow for Diffusion Models

In the rapidly evolving landscape of Artificial Intelligence, choosing the right tools is paramount for any development team. A recent GitHub Community discussion (Discussion #208944) brought to light a critical decision point for AI developers: selecting between PyTorch and TensorFlow for diffusion models. Making an informed choice here can significantly impact project velocity and output quality, factors often highlighted by effective development measurement tools.

The consensus from the community is clear: for diffusion models, PyTorch is emphatically recommended. Here’s why:

Diagram illustrating the PyTorch ecosystem for diffusion models, including Hugging Face diffusers and optimization tools.### Ecosystem & Library Dominance

The industry standard for diffusion models, Hugging Face diffusers, is built natively on PyTorch. This means that pretrained weights, pipeline schedulers (like Euler, DPM-Solver, UniPC), and fine-tuning scripts (LoRA, DreamBooth, ControlNet) are primarily optimized for PyTorch. This deep integration simplifies development and accelerates time-to-market for new features. Furthermore, PyTorch 2.x's native SDPA (Scaled Dot-Product Attention) and torch.compile offer significant speedups (2x–3x) without external dependencies, directly contributing to a more efficient development lifecycle.

Dynamic Computation Graphs for Agile Development

Diffusion sampling involves iterative denoising loops with dynamic timestep embeddings and conditioning tensors. PyTorch’s imperative design and dynamic computation graphs simplify the implementation and debugging of these multi-step reverse processes. This offers a more straightforward, intuitive experience compared to the often more rigid graph compilation workflows in TensorFlow, fostering agility and reducing debugging overhead for your engineering teams.

Research & Community Leadership

Over 90% of generative AI papers, checkpoints, and open weights released on platforms like arXiv and Hugging Face prioritize PyTorch implementations. This makes PyTorch the go-to framework for staying current with cutting-edge research and leveraging the latest advancements, ensuring your team remains at the forefront of AI innovation.

Visual representation of a classical state-space search problem, showing states, actions, and a goal path.While TensorFlow remains a viable option if your team has existing TensorFlow serving/training infrastructure, it is not the path with the broadest examples or the most current ecosystem support for diffusion models. For new diffusion projects, starting with torch + diffusers + accelerate for multi-GPU training and inference is the recommended approach.

Beyond Frameworks: Understanding State-Space Concepts in AI

The discussion also touched upon "state-space searches," a term that can have different meanings in classical AI versus modern machine learning. Clarifying this distinction is crucial for technical leaders guiding their teams.

Classical State-Space Search: The Foundation of Problem Solving

A State-Space Search is a foundational AI problem-solving paradigm where a problem is formulated as a set of discrete states, transitions (operators), and a target objective. Imagine a maze: each square is a state, moving between squares is an action, and reaching the exit is the goal.

State Space: The graph or tree of all possible configurations the system can be in.
Initial State: Where the search begins.
Actions/Transitions: Valid moves taking state S to successor state S'.
Goal Test: A function verifying if the current state satisfies the objective.
Path Cost: The cumulative cost incurred to reach a given state.

Common search strategies include:

Uninformed (Blind) Search:Breadth-First Search (BFS): Explores level-by-level; optimal if all step costs are equal.
Depth-First Search (DFS): Explores deep paths first; memory efficient but not guaranteed optimal.
Informed (Heuristic) Search:A* Search: Uses an evaluation function f(n) = g(n) + h(n), where g(n) is the actual cost so far and h(n) is an admissible heuristic estimate to the goal. A* is both complete and optimal.

Illustration comparing the linear processing of State-Space Models (Mamba) with the quadratic complexity of traditional Transformer attention.### Modern State-Space Models (SSMs): A New Paradigm for Sequence Processing

In modern generative AI and sequence modeling, the term "State Space Models" (SSMs) refers to architectures like Mamba and S4. These models are distinct from classical search algorithms. They model continuous sequence state transitions in O(L) linear time, offering significant computational advantages over Transformer-based models with their O(L2) attention mechanisms, especially for very long sequences. Understanding this distinction is vital for teams exploring cutting-edge sequence processing techniques.

Strategic Impact: Tooling Choices, Productivity, and Delivery

For dev team members, product/project managers, delivery managers, and CTOs, these technical decisions directly influence your team's productivity and project delivery metrics. Leveraging the right tools can drastically improve outcomes, making it easier to track progress with robust development measurement tools and achieve your targets on a performance KPI dashboard.

An optimal framework choice, like PyTorch for diffusion models, translates into faster iteration cycles, reduced debugging time, and quicker adoption of new research. This directly impacts your team's efficiency and the quality of your output. Furthermore, streamlined workflows, enabled by optimal framework choices, positively impact aspects like pull request analytics for GitHub, offering clearer insights into team efficiency and code velocity. By making informed decisions based on community insights and technical merits, leaders can empower their teams to build more effectively and deliver innovative AI solutions with greater confidence and speed.

Inside the ESP32-P4 Camera-to-Display Pipeline: CSI, ISP, PPA, Buffers, and DSI

ZedIoT — Wed, 14 Oct 2026 12:20:00 +0000

Inside the ESP32-P4 Camera-to-Display Pipeline: CSI, ISP, PPA, Buffers, and DSI

A camera preview that reaches the screen is not the same as a reliable camera-to-display design. The critical question is what happens to each completed frame between the sensor and display scanout: its pixel format, the memory it occupies, who may read or write it, and the event that makes it reusable. If an encoder and a GUI share the path, those questions matter more than a nominal frame-rate target.

Start with a contract for every boundary. Record the sensor's output mode, what CSI receives, what ISP writes to memory, whether PPA has a specific pixel operation to perform, and which completed buffer the display or encoder consumes. Only then can measured latency, a dropped frame, or a torn image be attributed to a stage. The architecture and API responsibilities below follow public Espressif documentation; they are not performance results from a particular board.

Draw the frame path around memory, not peripheral names

ESP32-P4 has MIPI-CSI, ISP, PPA, JPEG and H.264 hardware, MIPI-DSI, and DMA-capable paths. Their presence does not imply that a frame flows between them without an intermediate write, copy, or queue. The ESP-IDF ISP guide describes an ISP that receives data from a camera controller or system memory and writes processed output to system memory through DMA. That write is an architectural handoff, not a minor implementation detail.

The frame path, in text form:

Sensor → CSI → (system memory) → ISP → (system memory) → [PPA branch, if a pixel op is required] → display scanout (DSI) / encoder

Two notes on that path:

The direct buffer-to-display edge is conditional. Output format and display timing must actually be compatible.
PPA is a branch for required pixel work, not a mandatory stop between ISP and DSI. Likewise, the GUI is an input to a blend operation; it is not a claim that every GUI must be composited into a video frame.

At the central buffer pool, define four events for each frame: allocation, producer completion, consumer completion, and return to the free pool. A frame cannot be recycled just because one task has finished with it if another task, DMA engine, encoder, or display scanout still reads it. This contract often disappears when integration is described only as driver initialization calls.

Freeze the sensor-to-CSI contract before tuning images

A supported CSI format is only the receiver's capability

The ESP32-P4 datasheet lists a two-data-lane MIPI-CSI interface and input formats including RGB888, RGB666, RGB565, YUV422, YUV420, RAW8, RAW10, and RAW12. It does not configure a sensor for you. A valid receiver format says little about the selected sensor register table, lane rate, virtual channel, clocking, or exposure-control sequence.

Keep one reviewable mode record per sensor configuration: active resolution, line and frame lengths, CSI data type and bit depth, lane count and rate, external clock, and exposure and gain controls. If the sensor emits RAW10 but the display ultimately scans RGB565, name the stage that unpacks RAW, performs demosaicing and color processing, and quantizes the result. Otherwise an apparent "CSI works" milestone may conceal a color-order error or line-alignment problem that surfaces only after the display is added.

Use a sensor test pattern, if the sensor provides one, while establishing this boundary. A stable pattern removes scene movement and automatic exposure as competing explanations. Check frame completion and short-frame or line errors before adjusting ISP color parameters. If the pattern itself arrives with unstable rows, downstream image-quality tuning cannot repair the input contract.

CSI's 2.5 V supply belongs in the board bring-up checklist

The ESP-IDF Camera Controller guide calls out a stable 2.5 V supply for the ESP32-P4 CSI peripheral. An internal adjustable LDO can be used, but its configuration and connection must be complete before CSI driver initialization. Treat that as a schematic and power-sequencing prerequisite, not an optional software optimization.

On a custom board, check supply and clock first, then lane continuity and sensor register access, and only then complete image frames. An official development-board example is useful for API ordering, but it does not establish that a different board copied the same electrical path or that a new sensor's mode table is correct.

Make ISP output a useful downstream contract

For RAW input, the ISP can provide the image-processing stages that turn sensor samples into a usable frame: black-level correction, bad-pixel handling, lens-shading correction, Bayer filtering, demosaicing, white balance, and color correction, among others. Which stages to enable and how to tune them depend on the sensor and optics. Driver support for a stage is not a production image-quality calibration.

There is an initialization distinction worth preserving. In the stable v6.1 ISP documentation, using MIPI-CSI or ISP_DVP as the camera controller still requires an ISP processor created with esp_isp_new_processor(), even when ISP image algorithms are not needed. The bypass_isp configuration bypasses the selected algorithm pipeline; it is not a substitute for installing the driver. Recheck these names and requirements against the exact ESP-IDF release used by the product.

Work backward from the consumers

Before selecting an ISP output format, write down each consumer's requirements. The DSI/LCD path has a pixel format, line stride, address-alignment and scanout contract. A PPA operation has supported input and output formats and buffer rules. An encoder has an input format and may hold the frame after the display has moved on. A CPU algorithm may need temporary read access or a separate working copy.

An intermediate format shared by several consumers can remove a conversion, but it is not automatically optimal. Producing RGB for a display and converting it back to YUV for encoding adds memory traffic and queueing. Forcing the GUI to render against an awkward format to save that conversion can shift cost and complexity into composition. Compare complete paths—including memory reads and writes—rather than choosing a format based on one peripheral's advertised capability.

The handoff record should state whether the ISP output is immutable once marked complete. If PPA will modify an output in place where its operation permits it, distinguish that destination from the still-shared source frame. An encoder reading the original frame must not race with a GUI blend writing into it. This is an ownership decision before it is an API choice.

A buffer pool is a latency and correctness policy

Size allocations with stride, not width alone

For an uncompressed frame, begin with frame_bytes = stride_bytes × height. If stride happened to equal width × bytes_per_pixel, RGB565 would need approximately two bytes per pixel and RGB888 approximately three. Real allocations must include line alignment, DMA and driver constraints, and any format-specific planes or padding. The formula is a capacity check, not an FPS prediction.

Count all simultaneously live frames: capture or ISP outputs, PPA inputs and outputs, display framebuffers, GUI draw buffers, and encoder inputs. Then count memory transactions. A full-frame memory-to-memory conversion reads a source and writes a destination; scanout, GUI composition, encoding, and any copy add more traffic. "Does PSRAM fit the bytes?" and "Can the memory system sustain the simultaneous access pattern?" are separate questions requiring target-board measurements.

Single buffering saves capacity but exposes the same memory to producer and consumer contention. Two buffers let a producer work on a next frame while a consumer reads a completed one; they do not establish a safe display swap point. A third buffer can absorb a burst or slow consumer, yet it can leave an old frame waiting in a longer queue. More buffers are not a cure for an unspecified release rule.

Define ownership transitions explicitly

A useful design record names frame states, even if the application uses a different implementation: FREE → WRITING → COMPLETE → IN_USE → FREE. The producer alone owns a WRITING buffer. After its DMA completion and any memory-visibility work required by the configured memory region, it publishes a COMPLETE frame. Each consumer acquires a read claim; the pool recycles the frame only after the last claim is released and display scanout no longer references it.

That last condition matters when preview and H.264 encoding fork from one source. Reference counting, explicit per-consumer acknowledgments, or another equivalent mechanism can implement it. A frame should not be returned on "display finished" if encoding is still reading, nor on "encoder submitted" if encoding has only queued asynchronous work. Define whether the completion event means accepted, processed, or no longer reading memory.

Cache consistency belongs at the same transition. If the chosen CPU, DMA, and external-memory arrangement requires cache clean, invalidate, or barriers, perform the operation appropriate to the handoff direction before publishing the buffer or reusing it. Do not apply a generic cache recipe without checking current ESP-IDF memory and DMA requirements. When faults appear only with cache enabled, PSRAM placement, or higher concurrent load, first test visibility and premature reuse before labeling the symptom "insufficient bandwidth."

For a low-latency preview, an explicit policy may discard an expired completed frame and show the newest complete one. For per-frame analysis or archival, silently overwriting work is incorrect; backpressure, reduced input rate, or a documented degraded mode is needed. Log which policy fired. A display may scan the last complete frame again when its refresh rate exceeds capture rate; it must not scan a frame still under construction.

Give PPA a named pixel job

The ESP-IDF PPA guide describes scale/rotate/mirror, blending, and filling. PPA can move a required pixel operation out of general CPU code, but it still reads memory and writes a result. Its value depends on the transformation and resulting buffer traffic, not the mere fact that it is hardware acceleration.

Use it when sensor orientation differs from display orientation, when the preview window must be resized, or when a video frame and GUI surface must be blended. The PPA guide specifies separate input and output picture buffers for a scale/rotate/mirror operation; blending has different aliasing rules. Those distinctions matter when sizing a pool and deciding whether the original frame remains available to an encoder. Validate formats, block dimensions, offsets, alignment, and completion semantics for the operation actually selected.

If sensor output already matches the display's size and orientation, a whole-frame PPA pass may be unnecessary. For GUI overlays, compare two architectures: compose UI into the video frame, or let the display path manage video and UI separately where driver and hardware support that mode. Composition yields one final frame but may require substantial pixel work after each UI change. Separate layers or partial updates can avoid rewriting video pixels, but require compatible display features and careful synchronization. A claim of reduced CPU use alone does not settle the end-to-end tradeoff.

Split display and encoding into different consumer contracts

The display needs a complete frame available at the scanout deadline. The MIPI-DSI LCD example shows driver-managed framebuffer use and an LVGL draw-buffer arrangement in PSRAM. The MIPI-CSI–ISP–DSI example is a public reference for wiring modules together. Neither example proves a particular panel timing, sensor mode, UI load, or product memory layout.

Changing an address or writing a framebuffer while the display scans it can put two generations of pixels on one screen. The swap therefore needs a defined synchronization point and a buffer the producer will not overwrite. The right mechanism depends on panel timing, DSI driver behavior, and framebuffer mode; allocating two buffers without specifying the swap event is not a tearing test.

An H.264 encoder has a different deadline and output responsibilities: accepted input format, ordered frames, rate control, encoded-output buffers, and downstream storage or network consumption. Hardware encoding does not make its input lifetime zero. If preview and encoding share a source, the slow branch extends that source's lifetime. If encoding receives a separate copy, budget the extra allocation, read/write traffic, and delay. The display may prioritize freshness while the encoder requires continuity; one unbounded queue cannot satisfy both goals by accident.

Choose the contract that matches the product

Requirement or observation	Design response	Cost or boundary to verify
Lowest useful preview age	Keep the newest complete frame; document when older frames are dropped	Measure capture-to-scanout age and drop reasons, not just average FPS
Every frame must be analyzed or retained	Preserve per-frame ownership and apply backpressure or an explicit degraded mode	Queue growth, capture pressure, and recovery behavior
Rotation, scaling, or video/UI blend is required	Put the named operation on a PPA branch after checking formats and buffer rules	Extra memory transaction, destination buffer, and completion handoff
Preview and H.264 share a frame	Hold the source until both consumers have stopped reading	Encoder-induced pool starvation and display freshness
Camera format already matches display needs	Evaluate the direct completed-buffer route	DSI compatibility, scanout lifetime, and safe swap timing

This table is a decision aid, not a promise that all options can run together at a given resolution. Datasheet capabilities, reference examples, and measured end-to-end product behavior are different evidence classes.

Diagnose the first boundary that corrupts a frame

Similar screen artifacts can have unrelated causes. A color cast can originate in sensor output order, ISP color processing, RGB/BGR convention, or display interpretation. Repeating horizontal displacement points toward line length, stride, or starting address. An occasional old image suggests queue age, a missed swap, or incorrect return to the free pool. Adjusting the last module until the screen looks acceptable can hide an upstream error.

Place a comparable observation at each boundary:

With a sensor test pattern, record CSI completion and error counters.
At ISP memory output, inspect fixed rows and pixel positions along with recorded width, height, format, and stride.
Save a small set of pre- and post-PPA frames to see whether rotation, scale, or blend changed unintended areas.
At DSI, correlate the chosen framebuffer address with swap and refresh events.

Preserve the failing frame's ID, buffer address, stage, and event order so an intermittent failure can be traced to one handoff.

Do not treat visual correctness as latency evidence. Timestamp capture completion, ISP completion, PPA completion where used, display swap/refresh, and encoder completion where used. Record buffer-pool high-water mark, queue depth, dropped-frame reason, buffer location, and cache/DMA properties alongside those timestamps. A system can show an attractive average FPS while displaying increasingly stale frames or occasionally reusing a buffer too early.

Bring up the shortest observable route first

Control the sensor. Read its identity, fix one mode table, and save resolution, data type, lane configuration, clock, and power assumptions.
Prove CSI capture. Prefer a known test pattern. Log frame completion, short frames, line errors, and DMA status before adding a complete UI.
Fix ISP output. Start with unnecessary automatic adjustments disabled. Verify dimensions, stride, pixel order, and color; enable image-quality functions one by one.
Prove display ownership. With no GUI or encoder, identify producer completion, display acquisition, the swap event, and release event. Use timestamps, GPIO markers, or a logic analyzer where appropriate.
Add one PPA operation at a time. First test a required scale or rotation, then any GUI blend. Record its destination buffer, completion, and incremental memory traffic.
Add optional encoding or network consumption last. Exercise a deliberately slow consumer and observe queue high-water marks, drops or backpressure, and recovery.

At every step, retain ESP-IDF version, board and chip revision, sensor and panel part numbers, input/output formats and resolutions, buffer count, and memory placement. Without those identities, a later measurement cannot be compared fairly. Public references span stable v6.1 ISP documentation, v6.0 PPA documentation, latest Camera Controller documentation, and master-branch examples. They establish public responsibilities, not a tested single-SDK integration matrix.

Know when this is the wrong single-chip architecture

ESP32-P4 can fit products where MCU-level control, a camera input, local image processing, and an HMI must be tightly integrated. Its peripheral list alone is not a selection argument. A product needing complex multi-stream media handling, mature container support, heavy GPU composition, or large vision models may be better served by a Linux SoC and its media stack. At the other extreme, an occasional still image and simple display may not justify the power, routing, driver, and verification work of the full CSI–ISP–PPA–DSI path.

Sensor and panel ecosystems also matter. Missing stable drivers, timing data, or supply continuity cannot be fixed by the MCU's interface specification. If requirements simultaneously insist on no dropped frames, minimal preview delay, continuous encoding, and a complex GUI, first allocate explicit memory, power, thermal, and degradation budgets. Those goals may compete for the same buffers and bandwidth.

Before product review, assemble one stage-by-stage table covering format, resolution, stride, buffer size and count, memory region, DMA/cache attributes, producer, consumer, and release event. Then measure capture-to-display and capture-to-encode behavior under the fixed target workload. Public documentation supports the module responsibilities and validation method presented here; it does not establish frame rate, end-to-end latency, absence of tearing, memory bandwidth, power, temperature, or long-run stability on your hardware. Those are acceptance results to produce, not conclusions to import from an example.

Official references

ESP32-P4 Series Datasheet
ESP-IDF Camera Controller Driver
ESP-IDF Image Signal Processor, stable v6.1
ESP-IDF Pixel-Processing Accelerator, v6.0
ESP-IDF MIPI-DSI LCD Panel Example
ESP-IDF MIPI-CSI Camera via ISP to DSI Example
ESP-IDF DVP Camera Display via DSI Example, for DVP comparison only

Multi-Turn Memory, Mixed Precision, and Safer Rollouts — Serving Catches Up to Agentic Workloads

Felipe 0liveira — Wed, 14 Oct 2026 12:06:13 +0000

This edition of the LLM Inference Digest covers new developments from 2026-10-07 to 2026-10-14 across arXiv, the vLLM and Hugging Face blogs, Together AI, Modal, and Latent Space.

🔥 Highlights

Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM — lets decode nodes hand KV blocks back to prefill
DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 — three kernel tricks, 5.3x throughput under TPS limits
Endpoint Candidates: evaluate and promote inference endpoints — built-in traffic-mirrored canarying for serving changes
Vesta: Queueing-Aware LLM Scheduling for Real-World Workloads — balances cache locality vs. queueing, cuts TTFT sharply
Run fast, private agents on-device with LFM2.5-24B-A2B — sparse MoE hits 21K tok/s on one GPU

arXiv (cs.LG, cs.DC)

MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization (2026-10-13): splits a fixed KV quantization budget between layer-wise precision boosts and token-wise preservation, using lightweight offline profiling instead of uniform compression. Outperforms single-granularity baselines on Mistral/Llama/QwQ at matched memory budgets — useful when tuning KV memory limits per model rather than applying one blanket bit-width.
E8 Lattice Key Quantization for Low-Bit KV-Cache Compression (2026-10-11): replaces scalar KV quantization with vector quantization on an E8 lattice that exploits cross-dimension correlation, reaching a 2-bit keys/2-bit values config (previously needing 3-bit keys) for roughly 6x compression with decode throughput matching scalar baselines. Relevant for anyone pushing long-context serving memory limits.
The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving (2026-10-11): builds a cost model that identifies whether compute, weight loading, or KV-cache access is the real bottleneck at a given batch size/context length, then picks mixed-precision configs via dynamic programming instead of guessing from byte-width alone. Practical for choosing precision per-layer or per-component.
MLAfication: Efficiently Enabling High-Fidelity MLA for Modern LLMs (2026-10-12): converts standard multi-head attention into Multi-Head Latent Attention (DeepSeek-style), also handling QKNorm and MoE architectures that earlier conversion methods couldn't. Claims up to 5.35x serving throughput and ~95% memory reduction combined with quantization — a retrofit path for existing checkpoints instead of retraining with MLA from scratch.
FORGE: Procedural Weight Generation for Sparse MoE Inference (2026-10-12): regenerates MoE expert weights on-the-fly from compact rotor parameters instead of storing full expert tensors, cutting the memory footprint that forces multi-node parallelism with all-to-all overhead. Reports 1.3-2.8x higher throughput at matched GPU count and 10-15% lower decode latency.
Quota-Based Aging for Prediction-Guided Scheduling in LLM Serving (2026-10-13): a scheduling policy that keeps the mean-latency benefit of predicted-short-first admission while bounding tail latency for long requests via quota-based promotion, addressing the classic "shortest-job-first starves long jobs" problem. A solid read for teams tuning their scheduler's fairness/latency trade-off.
Vesta: Queueing-Aware LLM Scheduling for Real-World Workloads (2026-10-10): decouples per-session KV-cache "ownership" from per-request execution placement, balancing prefix-cache locality against queue congestion using queueing theory instead of always routing to the cache-affine node. Reports a 14% throughput gain and 57.8% mean TTFT reduction colocated, plus up to 29% P99 TTFT reduction disaggregated.
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference (2026-10-08): notes that Hessian-guided pruning calibrated on natural text mismatches the self-generated token distributions seen during actual decoding, and fixes this with decode-time calibration plus custom sparse matrix-vector kernels aimed at the memory-bound decode phase rather than prefill. Up to 1.48x wall-clock decode speedup on A100s.
Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy (2026-10-09): combines speculative decoding with a hierarchical sparse KV cache that dynamically shifts entries between GPU memory and overflow based on load instead of a fixed resident/evicted split. At 500K context and concurrency 512, reports 1.85x throughput over resident serving — relevant when long-context, speculative decoding, and memory pressure collide.
CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters (updated 2026-10-10): packs alternate candidates from a block drafter's single forward pass into a verification tree whose width adapts at runtime based on measured hardware latency, rather than a fixed, manually-tuned tree shape. Up to 43% speedup with no changes to model/drafter/decoding algorithm, provably preserving the output distribution — a drop-in upgrade for teams already running block speculative decoding.
Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing (2026-10-07): a measurement methodology, not a new serving technique — a "sequential isolation" benchmarking protocol that cut coefficient of variation from 15.2% to 2.2% on vLLM/A100 across context sizes and concurrency levels. Useful for anyone building CI-style regression gates for inference latency/throughput, where noisy benchmarks currently make such gates unreliable.
Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving (2026-10-10): four schedulers (from a greedy heuristic to a facility-level MILP) for carbon-aware request routing across geo-distributed datacenters, built on a simulator that models actual inference-engine queueing/batching dynamics. Reports up to 60% carbon reduction without violating SLOs — relevant for infra teams with multi-region deployments and sustainability targets.

vLLM Blog

Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM (2026-10-12): until now KV cache only flowed prefill → decode; this adds the reverse path, letting the decode node hand already-computed KV blocks back to the prefill node on later turns of a conversation so the prefix doesn't get recomputed, coordinated by a KV-cache-aware router with peer-to-peer tensor transfer over EFA on P5en. In multi-turn agentic workloads, this cut TTFT up to 1.6x on Qwen3-32B (778ms→460ms at 24k tokens) and up to 3.3x on GLM-5-FP8 (2334ms→702ms at 20k tokens) — a real latency/cost win for serving agents with long conversation history, no extra hardware required.
DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 (2026-10-07): three optimizations landed in three weeks — "SWA bounded replay" (recomputing only the last 128 tokens instead of caching the full sliding-window KV, +30-40% prefill), DeepSeek's own MegaAttention (1.14-1.51x) and Mega-Gate (1.18-1.31x) kernels, plus operation fusion and CUDA graphs to cut kernel-launch overhead. Net result: 1.9x speedup at low concurrency and 5.3x throughput under a 150 TPS constraint, with ~70% lower TTFT at 100K throughput — a direct cost-per-token decision for anyone serving DeepSeek-V4.1-Flash in production.
vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72 (2026-10-09): day-0 support for NVIDIA's new Vera Rubin NVL72 cluster, with "locality domains" tuned for MoE routing and FlashInfer 0.7.0 kernels adapted for Rubin while staying compatible with existing Blackwell kernels; the platform offers 5x more NVFP4 FLOPS and 2.4x more memory bandwidth than GB200. Reports 7.8x throughput per GPU over GB200 NVL72 on AgentX and 3.7x VLM throughput over GB300 on MLPerf v6.1 — relevant to next-generation hardware planning, though it's more a preview of future hardware than something actionable today.

Hugging Face Blog

Run fast, private agents on-device with LFM2.5-24B-A2B (2026-10-13): a 24B MoE model with only 2B active parameters per token (40 layers, 64 experts, top-4 routing) and context expanded from 32K to 131K, built to run on-device. Concrete throughput numbers — 184 tok/s on an Apple M5 Max and up to 21K tok/s aggregate on a single NVIDIA B300 — give a direct data point for choosing a sparse MoE over a larger dense model when serving low-cost, low-latency agents locally or on one GPU.
Other posts this week (Falcon ASR, Nemotron fine-tuning for IOI/IMO competition math, a retrospective on a model release) are model/training content without serving-performance relevance, so they're skipped here.

Together AI Blog

Nothing new in the window — the latest post ("Expanding our enterprise inference capacity with IBM Cloud and NVIDIA") predates this edition's cutoff and was already covered previously.

Modal Blog

Endpoint Candidates: evaluate and promote inference endpoints (2026-10-12): a feature for staging alternative deployments of a Dedicated Endpoint — a new checkpoint or a different model entirely — that mirrors a sample of live production traffic to measure quality, latency, throughput, and cost before promotion. The underlying engineering extends Modal's Pingora-based proxy so each mirrored candidate reads from its own bounded, deduplicated buffer with per-process memory budgets, so a slow candidate can't add latency to production traffic. A built-in, low-risk canary/A-B mechanism for quantization tweaks, model upgrades, or engine config changes, without building custom traffic-mirroring infrastructure.
Toward RL on mixed-precision, mixed-accelerator samplers (2026-10-13): describes training an open-weight model (Qwen3.6-35B-A3B) where the RL "samplers" — the inference/rollout side, which consumes 80-85% of GPUs in large RL runs — run heterogeneously across B200/B300 (NVFP4), H100/H200 (FP8/BF16), and A100/RTX PRO 6000 (BF16), each with an FP8 KV cache, while a single BF16 trainer learns from all of them via off-policy correction. Matches homogeneous-sampler quality at lower cost — a concrete recipe for exploiting idle or cheaper heterogeneous GPU capacity and low-precision serving kernels for inference-heavy workloads.

Latent Space

Nothing in the window met the inference-engineering bar — this week's posts covered agent orchestration patterns, AI news roundups, and robotics, none touching quantization, KV-cache, scheduling, or serving throughput/cost directly.

Through-line

The common thread this week is serving built around conversations, not single requests: vLLM's bidirectional KV transfer and arXiv's Vesta scheduler both attack the same problem — multi-turn, agentic sessions that keep re-paying prefill costs or get stuck behind naive queueing — from the serving-engine side and the scheduling-theory side respectively. Meanwhile Modal's Endpoint Candidates and the DeepSeek-V4.1-Flash kernel work point at the other half of the same story: once you've picked a faster serving path, you need a safe way to test it against real traffic before flipping it on for everyone.

What are you seeing in your own serving stack this week? Drop a comment below.