DEV Community: AI Tech Connect

"Design an Inference Batching System": How to Answer the 2026 AI Engineer Interview Question

AI Tech Connect — Fri, 18 Sep 2026 15:32:24 +0000

Originally published on AI Tech Connect.

Why this question is on every 2026 rubric There is one AI system-design question that keeps turning up in interview reports this year, and it is short enough to fit in a text message: design an inference batching system for a single GPU that handles up to 100 inputs per batch, while users submit requests synchronously and wait for the result. As of September 2026 it is widely reported as an Anthropic system-design question. The WashU McKelvey guide to AI engineer interview questions, published on 17 September 2026, lists it first among the recurring questions across 2026 loops and attributes it to Anthropic; Exponent's guide to the Anthropic system-design round quotes the same setup almost word for word and describes a 50 to 55 minute round that assesses abstraction, trade-off…

Read the full article on AI Tech Connect →

Write an Injection-Proof MCP Tool: shell=False, Allow-Lists and the 43% CVE Class

AI Tech Connect — Fri, 18 Sep 2026 13:32:27 +0000

Originally published on AI Tech Connect.

The 43% problem: what the 2026 CVE class looks like As of September 2026, the most common way an MCP server gets a CVE is not a clever protocol flaw. It is a tool function that takes a string from the model, glues it into a command line, and hands the result to a shell. This is the oldest bug in web security, and it has come back because MCP put a fresh generation of builders in front of subprocess with a caller that is far less trustworthy than a browser form. The figures are consistent across independent sources. Equixly's offensive assessment of popular MCP server implementations found that 43% of the servers it tested contained command injection flaws, 22% allowed reading files outside their intended directory, and 30% permitted unrestricted URL fetching. Practical DevSecOps tallied…

Read the full article on AI Tech Connect →

Review an Agent-Written Pull Request in 10 Minutes: The Checklist That Catches What Agents Get Wrong

AI Tech Connect — Fri, 18 Sep 2026 11:32:26 +0000

Originally published on AI Tech Connect.

Why review, not writing, is the 2026 bottleneck Two years ago the slow part of shipping software was writing it. That is no longer true for most teams I work with. A coding agent will produce a 600-line pull request in the time it takes to make a coffee, and then a second one, and a third. The pull requests arrive faster than anyone can read them, and the reading is now the constraint. If you run an engineering team in Bengaluru or London and your merge queue has started to look like a motorway at rush hour, you are not imagining it. The telemetry says the same thing. Faros AI's 2025 study of more than 10,000 developers across 1,255 teams found that developers on teams with high AI adoption completed 21 per cent more tasks and merged 98 per cent more pull requests, but PR review time rose…

Read the full article on AI Tech Connect →

Qualify a New Open-Weight Frontier Model in One Week

AI Tech Connect — Wed, 16 Sep 2026 15:32:26 +0000

Originally published on AI Tech Connect.

What this week decides On 11 September 2026, Shanghai Artificial Intelligence Laboratory published Atria Dawn Preview on Hugging Face. It is a 744-billion-parameter open-weight agentic mixture-of-experts model with roughly 40 billion parameters active per token, a one-million-token context window, and roughly 1.5 terabytes of published weights, built on the GLM-5.2 foundation model and released under an MIT licence. There was no blog post. There was no paper. There was no pricing announcement. A live OpenAI-compatible API appeared at api.atria-asi.ai as a free preview, the code repository went up alongside the weights, and that was the whole launch. The model's own materials claim it is "competitive with frontier agents" across 16 benchmarks spanning real-world research, engineering and…

Read the full article on AI Tech Connect →

The Two-Model Voice Agent: Full-Duplex Front End, Reasoning Back End

AI Tech Connect — Wed, 16 Sep 2026 13:32:26 +0000

Originally published on AI Tech Connect.

What this architecture actually buys you Our guide to realtime voice agents, latency budgets and barge-in answers one version of this problem: the caller speaks, the stack thinks, the agent replies, and your job is making that round trip fast enough that nobody notices the gap. That is the turn-based world, and its discipline is arithmetic — shave milliseconds out of a serial pipeline until the total lands under the threshold. This guide is about a different architecture, and the difference is not speed. It is structure. A full-duplex front end does not take turns. It listens and generates speech simultaneously, so it can acknowledge you mid-sentence, hold the floor while something else works, and yield cleanly when you cut across it. And it does not do the hard thinking: it delegates…

Read the full article on AI Tech Connect →

What to Charge as a Freelance AI Engineer in India and the UK

AI Tech Connect — Wed, 16 Sep 2026 11:32:27 +0000

Originally published on AI Tech Connect.

What this guide settles There is a conversation every independent AI engineer has, usually badly, usually on the spot, and usually for less money than they should. A client asks what you charge. You name a number you half-invented, they accept it slightly too quickly, and you spend the next eight weeks working out what the engagement actually paid you after tax, after the umbrella deduction, after the API bill you absorbed because nobody wrote down who was paying for it. This guide is about that number and the document it lives in. It deliberately does not cover finding clients — that is landing your first AI consulting clients — or shaping the first engagement, which is scoping a two-week paid AI pilot. Assume you have a buyer and a shape. What follows is the price and the terms. Four…

Read the full article on AI Tech Connect →

Sony and Warner Sue Anthropic — and Name Its Founders

AI Tech Connect — Wed, 16 Sep 2026 09:32:28 +0000

Originally published on AI Tech Connect.

What you need to know The filing. Sony Music Publishing and Warner Chappell Music filed suit against Anthropic in the US District Court for the Northern District of California on 28 August 2026. Who is named. The defendants include Anthropic PBC, chief executive Dario Amodei and co-founder Benjamin Mann, the last two as individual defendants. The core allegation. The complaint alleges Anthropic illegally downloaded, scraped and torrented copyrighted musical compositions in order to train its models. The alleged sources. According to the publishers, those sources included the pirate archives Library Genesis and the Pirate Library Mirror, and licensed lyric sites including Musixmatch and LyricFind were also scraped. The scale claimed. The complaint identifies "tens of thousands" of the…

Read the full article on AI Tech Connect →

Gemini Notebook Swaps Daily Caps for a Compute Budget

AI Tech Connect — Wed, 16 Sep 2026 07:32:35 +0000

Originally published on AI Tech Connect.

What you need to know A small documentation change with an outsized effect on how you plan a working day: The unit of account changed. From 2 September 2026, Gemini Notebook replaced fixed daily feature caps with compute-based usage limits. Same features, different meter. There are two clocks. Google's documentation states that “the quota refreshes every 5 hours until you reach your weekly limit” — a rolling quota under a weekly ceiling. Not every action costs the same. A short factual query consumes far less than a multi-source Video Overview or a full Slide Deck. The tier multipliers do not share a base. Plus and Pro are quoted against standard limits; Ultra is quoted against AI Pro. The baseline is unpublished. Google confirms a weekly limit exists but never says how many prompts…

Read the full article on AI Tech Connect →

OpenAI's GPT-Live-1 Listens and Speaks at the Same Time

AI Tech Connect — Wed, 16 Sep 2026 05:32:29 +0000

Originally published on AI Tech Connect.

What changed OpenAI launched GPT-Live-1 in the API on 10 September 2026. It is a hosted model — there are no weights to download, and it is not an open-source release. It is full duplex. The model listens and generates speech at the same time, rather than taking turns the way every mainstream voice stack has until now. Benchmarks, from OpenAI's own published figures: 80.1% full-duplex interactivity on Full Duplex Bench against 45.4% for GPT-Realtime-2.1 — a gain of roughly 30 points. Turn-taking latency of 0.8 seconds, down from 1.4. Tool-calling accuracy of 87%, up from 60%. Price: $0.05 per minute for the voice layer, billed per second rather than rounded up to a full minute. The backend model and any tools you pair with it are billed separately at their normal rates. Voices: 12 out of…

Read the full article on AI Tech Connect →

Keep Your Agent's Safety Rules Alive Through Compaction

AI Tech Connect — Tue, 15 Sep 2026 16:08:38 +0000

Originally published on AI Tech Connect.

What you need to know The window filling up is not the failure mode. Model accuracy degrades as context grows even when every relevant fact is still present. Reported degradations in the 2026 literature span roughly 13.9 percent to 85 percent depending on task and setup — a range wide enough to treat as a direction, not a number. Compaction is the answer the field converged on. Summarise the work so far, start a fresh window, continue. It rewrites the agent's logical context rather than its physical representation, which is why it works at the harness level regardless of who serves the model. The default triggers are both wrong. Reactive compaction fires at saturation, so the summarising model is already degraded when you ask it to do the most delicate job in the loop. Periodic compaction…

Read the full article on AI Tech Connect →

Prompt, Semantic or Exact-Match: Choosing Your LLM Cache

AI Tech Connect — Tue, 15 Sep 2026 14:06:17 +0000

Originally published on AI Tech Connect.

Three layers that share a name and nothing else Ask three engineers what "caching the LLM" means and you will get three answers, all correct and mutually incompatible. One means the provider skipping prefill on a repeated prompt prefix. One means a vector lookup that returns a stored answer without calling the model at all. One means a hash in Redis. They sit at different points in the request path, they save different parts of the bill, and — the part that matters most — they fail in ways that have nothing in common. One fails by quietly saving you no money. Another fails by quietly answering the wrong question. That last distinction is why "add caching" is not a decision. It is three decisions, and the order you make them in determines whether you end up with a cheaper product or a…

Read the full article on AI Tech Connect →

Prove You Can Build Evals: The Skill Gap That Gets You Hired

AI Tech Connect — Tue, 15 Sep 2026 12:02:47 +0000

Originally published on AI Tech Connect.

Almost everyone can call an API. Almost nobody can tell you if it got better Sit on the other side of an interview loop for a few months and a pattern emerges that is almost boring in its consistency. Candidates can wire up a model. They can build retrieval. Many of them can put an agent loop together, handle tool calls, stream tokens to a front end and deploy the whole thing behind a gateway. Then you ask one question — how did you know the change you made was an improvement? — and the room goes quiet. The honest answers are variations on the same theme. "It looked better." "The demo worked." "We tried a few prompts and picked the one we liked." Occasionally someone mentions a benchmark score from a model card, which is a measurement of somebody else's task, not theirs. Very rarely,…

Read the full article on AI Tech Connect →