DEV Community: Muskan Bandta

Spark 4.2 Added Native Vector Search: Do You Still Need a Vector Database?

Muskan Bandta — Fri, 24 Jul 2026 09:28:53 +0000

The headline going around is that Spark 4.2 can retire your vector database. That's half true, which is the most dangerous kind of true. Spark did add real vector search, and for some workloads it genuinely removes a whole system from your stack. For others, you'd regret dropping your vector DB. Here's the honest version.

The short answer

If your vectors already live in your data platform and your searches are batch or analytical, Spark 4.2 can absolutely replace a separate vector database. If you're serving live, low-latency retrieval for a chatbot or search box, you probably still want a dedicated one. It's a "depends on the workload" answer, and the details matter.

What Spark 4.2 actually added

Spark 4.2 brought vector search into plain SQL. No bolt-on library, no separate engine. The new primitives include vector distance and similarity functions, vector normalization, vector aggregation like sum and average, and the headline one, NEAREST BY, a top-K ranking join that finds the closest matches by distance.

In practice that means you can store embeddings in a Spark table and run a similarity search with SQL you already know. Those operations cover the real use cases: retrieval, recommendations, entity resolution, and candidate generation. Databricks is openly framing this as Spark becoming an AI serving layer, not just a batch engine.

Why this is a big deal

The value isn't that Spark invented vector search. Plenty of tools do it. The value is that you can keep your retrieval pipeline on one platform.

Think about the normal setup today. Your data sits in a lakehouse or warehouse. To do vector search, you spin up a separate vector database, then build a pipeline to copy and sync embeddings into it, and keep the two in step forever. That's a second system to run, secure, pay for, and debug at 2 a.m.

Spark 4.2 lets you skip that for a lot of cases. The embeddings stay where your data already is, and the search runs right there. Fewer moving parts is a real win, and it's the part of the hype that's genuinely earned.

When it can replace your vector database

Spark 4.2 is a strong fit when your work is batch or analytical and the data already lives in Spark. Good examples: generating recommendation candidates overnight, deduplicating millions of records with entity resolution, or building a retrieval set as part of a larger data job. If the search is part of a pipeline rather than a live user request, doing it in Spark means one less system to maintain.

When you still want a dedicated one

Here's the caveat the headlines skip. Dedicated vector databases exist because online serving is a hard, specialized problem. They answer live queries in single-digit milliseconds using purpose-built indexes.

From what's documented, Spark 4.2's vector functions don't advertise a native approximate-nearest-neighbor index like HNSW. That matters at scale, because exact similarity search across huge datasets is expensive and slower than an indexed lookup. So if you're powering a live RAG chatbot, a search-as-you-type box, or anything where a user waits on the result, a purpose-built vector database is still the safer choice for now.

The rough rule: batch and analytical retrieval, Spark can own it. Real-time, user-facing retrieval, keep the specialist.

The real trade-off

This comes down to fewer moving parts versus specialized performance. Spark 4.2 wins on simplicity, one platform, one copy of the data, one skill set. A dedicated vector database wins on low-latency serving at scale. Neither is strictly better. The question is which one your workload actually needs.

What I'd do

Start by asking where the latency requirement is. If nothing is waiting on the query in real time, try the Spark path first and enjoy deleting a system. If a user is waiting, benchmark honestly before you rip out your vector DB, because "it works in a demo" and "it holds up under live traffic" are different claims.

The bottom line

Spark 4.2 didn't kill the vector database. It absorbed the batch and analytical half of what vector databases do, which for many teams is the half they were running a whole extra system for. Keep the specialist for live, low-latency serving. Drop it where Spark can now do the job. As always, the smart move is matching the tool to the workload, not the headline.

Grok 4.5 vs Claude Opus 4.8: Same Code, a Quarter of the Tokens?

Muskan Bandta — Fri, 24 Jul 2026 09:28:40 +0000

xAI has a bold pitch for Grok 4.5: it codes about as well as Claude Opus 4.8, but does it with roughly a quarter of the tokens. That's not a "we're smarter" claim. It's a "we're just as good for far less money" claim, which in 2026 might matter more. Someone actually put it to the test, so let me walk through what the numbers say and why you should care.

The claim and the pricing

Grok 4.5 landed on July 8, 2026. xAI says it matches Opus 4.8 on coding while using about 4.2 times fewer output tokens to get there.

The sticker price already favors Grok. It runs $2 per million input tokens and $6 per million output. Opus sits at $5 and $25. That's less than half the price on both sides before you even factor in the token efficiency. Stack the two together and the cost gap gets dramatic.

What the benchmarks say

The benchmarks mostly support the marketing, with a catch. On Terminal-Bench 2.1, which measures real command-line work, Grok 4.5 scored 83.3 percent to Opus 4.8's 78.9. But on SWE-Bench Pro, the harder test of fixing real open-source bugs, Opus still comes out ahead.

So the honest read is not "Grok is better." It's "Grok is about as good, for a lot less." Different claim, and a more interesting one.

The hands-on test

Benchmarks are one thing, real work is another. The New Stack ran a head-to-head, giving both models the same three jobs in one real Rust project (the fd file-finder) inside Cursor, and tracked every token. I'm summarizing their results here, credit to them for actually measuring it.

The three tasks were a bug fix, a multi-file refactor, and a feature build. The code both models produced was nearly interchangeable, so the story came down to tokens, time, and cost.

On the small bug fix, Opus actually won. Both wrote an identical fix with all tests passing, but Opus did it faster and on fewer tokens. Grok's efficiency edge showed up on the bigger jobs. On the refactor, Grok used about 197K tokens versus Opus's 954K for the same result, roughly a fifth. On the feature build, Grok spent about 603K tokens to Opus's 3.2 million, with Opus only slightly more thorough (it wrote the man page entry too).

Add it up and the totals are striking. Across all three jobs, the reported numbers were about 1.01 million tokens and $1.00 for Grok, against 4.33 million tokens and $5.14 for Opus. That's Opus using 4.3 times the tokens, almost exactly the gap xAI advertised.

The caveats worth knowing

Before you cancel anything, the test author was upfront about the fine print, and it matters.

Those were Cursor's blended token counts, not raw API numbers, so they include context each agent re-sends every turn. Grok ran on Cursor's fast tier while Opus ran on its thinking tier, so some of Opus's token pile is extended reasoning, not pure waste. And Grok's price had a 50 percent promo, though even at full rate its three jobs came to about $2 versus Opus's $5.14. It's also three tasks in one repo, not a giant benchmark. Small sample, real signal.

What it actually means for developers

Here's the part I care about, because it's a cost story dressed as a model story.

For a lot of everyday coding, these two models are interchangeable on output. If the code comes out the same, the tiebreaker becomes tokens, time, and dollars, and on the bigger jobs Grok won those clearly. When you're doing something at scale, hundreds of tasks instead of three, a 4x token gap turns into a serious line item.

That said, Opus wasn't beaten on quality. It was a shade more thorough on documentation and it won the small, fast task. If your work leans toward hard bug-fixing or you value that extra polish, the premium may still be worth it.

The bigger shift is that "just use the most powerful model for everything" is getting expensive to justify. The smart move now is to match the model to the task and watch the cost per job, not the brand on the box.

The bottom line

Grok 4.5 didn't prove it's better than Claude Opus 4.8. It proved it can do a lot of the same coding work for roughly a quarter of the tokens and a fraction of the price. For teams past the stage of not caring what AI costs, that's the whole ballgame. Test both on your own workload, track the tokens, and let the bill help pick the model. As always, benchmarks start the conversation, your repo finishes it.

The White House Accuses Moonshot of Copying Anthropic: What AI 'Distillation' Actually Means

Muskan Bandta — Fri, 24 Jul 2026 09:28:29 +0000

The US government just accused a Chinese AI company of copying an American model, and threatened sanctions over it. Whether or not the claim holds up, it's worth understanding what's actually being alleged, because the word at the center of it, distillation, is about to come up a lot. Here's the plain version.

What distillation means

Model distillation is training a new model using the outputs of an existing one. Instead of learning purely from raw data, the new model learns by copying how a stronger "teacher" model responds. Done legitimately, it's a normal technique for making smaller, cheaper models. Done against someone else's commercial model without permission, it's a way to clone a lot of that model's behavior on the cheap, and it usually violates the terms of service.

That's the core of the accusation: that one company used another company's model as an unpaid teacher.

What the White House claims

Michael Kratsios, who leads the White House Office of Science and Technology Policy, accused Moonshot AI, a Beijing-based company, of distilling Anthropic's Fable model to build its Kimi K3. He alleged Moonshot built a sophisticated internal platform to run large-scale distillation against US models while switching between access methods to avoid detection.

The claim didn't stop there. Kratsios also said Moonshot obtained restricted Blackwell-generation Nvidia servers through Thailand, which would sidestep US export controls. Treasury Secretary Scott Bessent followed up by putting sanctions and export-control blacklisting on the table.

Why some experts are skeptical

Here's the part that keeps this from being open-and-shut. Anthropic's Fable model has only been public since July 1, and Kimi K3 shipped on July 16. That's a very short window to distill a 2.8-trillion-parameter model primarily from another model's outputs. Several experts have pointed out that the timeline makes "mostly built by distilling Fable" hard to believe.

So the honest framing is that this is a serious accusation, not a proven fact. Distillation is also genuinely hard to prove, because model outputs don't carry a watermark saying where the training signal came from.

Why this matters beyond the politics

Set aside the geopolitics and there's a real industry question underneath. As frontier models get expensive to train, distilling a competitor becomes a tempting shortcut, and the lines around what's allowed are still blurry. This won't be the last time a strong new model gets accused of standing on someone else's shoulders.

It also puts open models in an awkward spot. Kimi K3 is open-weight and cheap, which is great for developers, but open success now attracts this kind of scrutiny. Expect provenance, where a model's capabilities came from, to become a bigger part of how these tools are judged and regulated.

What it means for engineers

Practically, a couple of things. If you use or ship models, the terms of service around training on another model's outputs are worth actually reading, because that's the exact line being fought over here. And model provenance is becoming a real procurement question, especially for anyone in a regulated or government-adjacent space, where a sanctioned or blacklisted vendor is a genuine risk.

None of this changes whether Kimi K3 works. It changes whether you can safely build a business on it.

The bottom line

The White House says Moonshot distilled Anthropic's model to build Kimi K3 and dodged chip controls to do it. Moonshot's model is real and impressive, the accusation is serious, and the timeline has experts doubting the strongest version of the claim. Treat it as an unresolved fight with real stakes, not a settled verdict, and watch how it plays out, because the rules it sets will shape how the next wave of models gets built and judged.

Kimi K3 Sold Out in 48 Hours: The AI Bottleneck Just Moved to Inference

Muskan Bandta — Fri, 24 Jul 2026 09:28:19 +0000

Moonshot launched Kimi K3, and within 48 hours it had to stop taking new subscribers. Not because the model flopped, but because too many people wanted it. That's a strange kind of problem to have, and it's telling you something important about where AI's real bottleneck now sits.

What actually happened

Less than two days after Kimi K3 went live, Moonshot froze new subscriptions. The reason was blunt: demand had eaten through its available GPU capacity. In the company's own words, the model got "far more love than we expected," and in 48 hours usage pushed close to the limit of what its hardware could serve.

Moonshot handled it reasonably. Existing users kept their access, and the company said it would expand capacity and reopen signups in batches. It also split its plans into two tiers, a general Kimi Membership for web and app use, and a separate Kimi Code Membership aimed at programming work. That split is a hint about which users are burning the most compute.

Why the model drew that kind of demand

Kimi K3 isn't a minor release. It's an open-weight model at 2.8 trillion parameters, and it reportedly beat Anthropic's Fable 5 and OpenAI's GPT-5.6 Sol on front-end coding tests. Open, cheap, and competitive on real coding is exactly the combination developers pile onto. So they did.

The real story: the bottleneck moved

Here's the part worth internalizing. For years the hard, expensive problem in AI was training. That's where the giant compute bills and the headlines were. Kimi K3's freeze shows the constraint shifting to inference, the cost of actually running the model for users, every request, every day.

Agentic workloads are why. When an AI agent runs a coding task for minutes or hours instead of answering a single prompt, each user consumes far more compute than a chatbot ever did. Multiply that by a viral launch and you hit a GPU wall fast. Moonshot didn't run out of ideas. It ran out of chips to serve the ideas.

Why this matters even if you never touch Kimi

This isn't a Moonshot-only problem. It's a preview for the whole industry.

Inference capacity is becoming the scarce resource. As more products lean on long-running agents, providers will hit the same wall, and you should expect more rate limits, waitlists, tiered plans, and quiet capacity throttling across the board. The "just call the API, it's infinite" assumption is ending.

It also reshapes cost. If inference is the constraint, then compute per task is the number that matters, which is why cheaper and more efficient models suddenly look so attractive. Serving is where the money and the limits now live.

What engineers should take from it

A few practical habits. Design for rate limits and capacity failures instead of assuming endless availability, because your provider can hit a wall with no warning. Keep a fallback model or provider ready so one company's GPU shortage isn't your outage. And when you pick a model, weigh how efficiently it runs, not just how it scores, because efficiency is what decides whether you can actually get served at scale.

The bottom line

Kimi K3 selling out in 48 hours wasn't a failure, it was a signal. AI's hard problem has moved from training models to serving them, and agentic workloads are burning inference capacity faster than anyone can add GPUs. Build like capacity is finite, keep a backup path, and treat compute-per-task as a first-class number. The teams that plan for the inference wall will be the ones still running when everyone else hits it.

Amazon, Microsoft, and Google Are Building the Same Thing: The Enterprise AI Agent Lock-In Trap

Muskan Bandta — Tue, 21 Jul 2026 06:21:40 +0000

Look closely at what AWS, Microsoft, and Google shipped for AI agents this year and you notice something odd. Different names, different branding, and underneath, the exact same product. That convergence is convenient right up until you try to leave. Here's what they're all building and why it should make you a little cautious.

What all three are actually building

Strip away the marketing and the three big clouds are assembling the same five-part stack for running enterprise AI agents: a managed runtime to execute the agent, memory so it remembers context, identity so it can authenticate and be governed, tools so it can call your systems, and observability so you can see what it did.

The products are AWS Bedrock AgentCore, Microsoft's Azure AI Foundry with its Agent365 control plane, and Google's Vertex AI agent stack. They look different on the pricing page. Architecturally, they're the same idea built three times.

This isn't a coincidence. When you're solving the same problem, running agents safely inside an enterprise, you land on the same pieces. The market basically agreed on the shape of an agent platform in early 2026, and each hyperscaler raced to own it on their own cloud.

Why the convergence feels great at first

If your data and identity already live in one cloud, that cloud's agent runtime is the fastest path to production. Your storage is there, your identity system is there, your logging is there. The platform snaps into all of it. You get governance, audit trails, and deployment almost for free.

For a team that just wants agents running with proper controls, that's a real gift. I get the appeal.

The catch nobody puts on the slide

Here's the trap. When your agent's runtime, credentials, state, and telemetry all live inside one cloud, moving it somewhere else isn't a config change. It's a rebuild.

Take AWS Bedrock AgentCore. An agent built on it is wired to AWS identity, networking, and storage. Picking it up and moving it to Azure or Google isn't a matter of swapping a few integrations, because the IAM bindings, the VPC setup, and the storage hooks are all AWS-specific. You don't move the agent. You rewrite it.

That's the difference between switching APIs and switching clouds. Rewriting a few integrations is annoying. Re-implementing identity, state, and telemetry on a new platform is a project.

But don't the open protocols fix this?

Partly, and it's worth being precise here. Open standards like MCP, which connects agents to tools, and agent-to-agent protocols do standardize how agents talk to things. That's real progress.

But talking is not the whole problem. Portability has three axes: the framework your agent uses, the model it runs on, and the cloud it lives in. Open protocols mostly help the first two. They don't move your deployment, your identity setup, or your stored state to another cloud. Fail on any one of those axes and you've just traded one kind of lock-in for another.

What cloud-agnostic actually takes

Real portability means being deliberate on all three axes from the start, not bolting it on later.

Keep the agent logic in a framework you control rather than the cloud's proprietary primitives, so the brain of the agent isn't welded to one vendor. Keep the model swappable behind an abstraction, since prices and quality shift monthly. And keep a control plane above the clouds, an agent gateway or management layer that governs, observes, and audits agents no matter which cloud or framework produced them.

The good news is this layer is emerging. Governance tools like Microsoft Purview, Databricks Unity Catalog, and Google's Agent Governance Framework are starting to give you audit trails and data lineage across agents. The catch is that using one cloud's governance tool can quietly pull you back into that cloud. Neutral is the goal.

What I'd do about it

You don't have to avoid the hyperscaler platforms. They're genuinely good, and for a single-cloud shop the lock-in may be a price worth paying for speed. Just make it a decision, not an accident.

Before you build on one, ask the boring questions. If we had to move this agent to another cloud in a year, what breaks? Where does its state live? Who owns its identity? If the honest answer is "everything breaks," you're choosing lock-in, and that's fine as long as you chose it on purpose.

The bottom line

Amazon, Microsoft, and Google are building the same enterprise agent platform, and each one is easiest to adopt inside its own cloud and hardest to leave. Open protocols standardize how agents communicate, but they don't make your deployment portable. If moving clouds later matters to you, keep your agent logic, your model, and your control plane vendor-neutral from day one. Convenience now, or freedom later. Just make sure you know which one you picked.

OpenAI Just Bought Gitpod: The AI IDE Wars Are Officially On

Muskan Bandta — Tue, 21 Jul 2026 06:12:18 +0000

OpenAI just bought a cloud company, and if you only read the headline you'd miss the point. This deal tells you exactly where AI coding is heading: out of your editor and into the cloud. Here's what happened and why it matters.

What OpenAI actually bought

On June 11, 2026, OpenAI announced it's acquiring Ona, a German startup you might know by its old name, Gitpod. Terms weren't disclosed, and the deal still needs regulatory approval before it closes.

Ona builds secure cloud environments where code can run. That's the whole reason OpenAI wanted it. Its Codex coding agent has grown fast, now past 5 million weekly active users, up from 3 million in April, and the jobs those agents run have gotten much longer. Tasks that used to take a few minutes now stretch into hours, sometimes days.

An agent that works for hours can't live on your laptop. Close the lid and it dies. It needs a persistent place in the cloud to keep running. That's what Ona gives Codex, and notably, Ona's platform can run inside a customer's own cloud, which matters for enterprises that won't send code elsewhere. Ona's enterprise usage reportedly grew 13-fold this year, with clients like major banks and pharma companies.

Why this is a bigger deal than it looks

For years, AI coding meant an assistant inside your editor. Autocomplete, a chat panel, quick edits. The editor was the product.

That era is ending. When agents run for hours on their own, the important question isn't "how good is the autocomplete." It's "where does the agent run, and for how long." The battleground is shifting from the editor to persistent cloud execution, and OpenAI just bought its way into that fight.

The wider war

OpenAI isn't alone in this move, which is why it feels like a real war now.

Cursor already added cloud agents that run tasks without tying up your machine. Devin, from Cognition, was built cloud-first as an autonomous engineer from day one. Anthropic's Claude Code runs long, multi-step jobs and connects into your tools. Every serious player is racing toward the same place: agents that run for a long time, in the cloud, ideally inside your infrastructure.

Buying Ona is OpenAI making sure Codex can play that game. The company that owns the execution layer, not just the model, controls how far these agents can go.

What it means for engineers

A few practical shifts to expect.

Your workflow is changing from "assist me while I type" to "go do this and report back." That's a different skill. You'll spend less time writing lines and more time scoping tasks and reviewing results.

Where the agent runs will matter as much as how smart it is. For anyone in a regulated or security-conscious shop, "does it run in our own cloud" is about to be a standard question. That's exactly why Ona's in-your-cloud model was attractive.

And lock-in moves up a level. It's no longer just which model you use, but whose execution platform your agents live on. Keep that in mind before you wire everything to one vendor's cloud.

The bottom line

OpenAI buying Gitpod isn't really about an editor. It's about owning the place where long-running AI agents actually run. Coding is moving from autocomplete in your editor to autonomous jobs in the cloud, and every big player is now fighting for that layer. If you write software, get comfortable delegating and reviewing instead of just typing, and start asking where your agents run, because that question is about to matter a lot.

The Pentagon Called Anthropic a Security Risk: What It Means for AI in Regulated Industries

Muskan Bandta — Tue, 21 Jul 2026 06:12:05 +0000

Something happened in 2026 that should be on every engineer's radar, especially if you work in finance, healthcare, government, or anywhere regulated. The US government labeled an American AI company a national security supply-chain risk. That had never happened before. Whatever you think of the politics, the professional lesson underneath it is big, so let me walk through it plainly.

What actually happened

The Department of Defense terminated a contract with Anthropic worth an estimated $200 million. The breakdown wasn't about price or performance. It was about how the military could use the AI.

Anthropic's CEO, Dario Amodei, objected to a clause that allowed the military "any lawful use" of the model. Anthropic wanted the contract to explicitly rule out things like domestic mass surveillance and autonomous weapons, and refused to sign without those limits. The government wanted fewer restrictions.

When talks broke down, the Secretary of War, Pete Hegseth, formally designated Anthropic a "Supply-Chain Risk to National Security." That label is normally reserved for foreign adversaries and hostile telecom vendors, think of how the US treated Huawei. This was the first time it had ever been applied to an American company, and reportedly the first time it was used because a company wouldn't agree to certain contract terms. OpenAI then stepped in to take the Pentagon work Anthropic had walked away from.

There's a legal coda worth noting for accuracy. In March 2026, a federal judge granted a preliminary injunction against the government, writing that the designation looked like retaliation for Anthropic drawing public scrutiny, which she framed as a First Amendment problem. So the label is being contested in court, not settled fact.

Why this matters beyond the headlines

Strip away the politics and this is a case study in something every regulated team deals with: what happens when your AI vendor's policies collide with your requirements.

Anthropic drew a hard line on use cases. The customer wanted flexibility. Neither is unreasonable, but the clash blew up a $200 million relationship overnight. If a deal that size can unravel over usage terms, so can yours.

The real lessons for regulated industries

If you deploy AI where compliance and governance matter, take a few things from this.

Your AI vendor has opinions, and they can change. Model providers set usage policies, and those policies can restrict what you're allowed to build, or shift under you with little notice. Read the acceptable-use terms as carefully as the pricing.

Single-vendor dependence is a real risk. Anthropic's customer lost access to a model it had built around. If your critical workflow runs on exactly one provider and that relationship breaks, you're stuck. That's the AI version of vendor lock-in, and it now has a very public example.

Contract terms are an engineering concern, not just legal fine print. What the model may be used for, where it runs, and who can access the data are questions that shape your architecture. Get them answered before you build, not after.

What I'd actually do about it

The practical defense is boring but effective. Abstract your model layer so you can swap providers without rewriting everything. Keep a second vendor qualified, even if you don't use it daily. And treat your AI provider like any other critical dependency, with a plan for what you do if it disappears tomorrow.

None of that is exciting, but it's the difference between a vendor dispute being a headline you read and a headline you live.

The bottom line

The Anthropic-Pentagon fight is being sorted out in court, and reasonable people disagree on who's right. But the lesson for the rest of us is clear and apolitical: AI vendors come with policies, limits, and risks, and building critical systems on a single provider is a bet. Read the usage terms, keep a backup path, and design so you can move. In regulated work, the vendor you can't replace is the vendor that can hurt you most.

The AI Coding Price War Is a Bloodbath: GPT-5.6, Claude Sonnet 5 & Kimi K3 in One Month

Muskan Bandta — Fri, 17 Jul 2026 06:33:00 +0000

In roughly one month, three of the biggest names in AI shipped new models, and the story wasn't the capabilities. It was the price. The floor under AI is falling fast, and if you pay for these tools, that's the most important trend of 2026. Here's what happened and what it means.

What just launched

Three launches in quick succession tell the whole story.

Anthropic released Claude Sonnet 5 on June 30 and made it the default for every free and Pro user the next day. It's the most agentic Sonnet yet, and at intro pricing it costs about $2 per million input tokens and $10 per million output, before rising to $3 and $15 after August 31.

OpenAI shipped its GPT-5.6 family in three sizes, Luna, Terra, and Sol, priced from roughly $1 to $5 per million input tokens, spreading the lineup across budget and premium tiers.

And Moonshot AI dropped Kimi K3 on July 16, a 2.8-trillion-parameter open model. That last one matters for a different reason: it's open, which puts price pressure on everyone charging for closed models.

Add Meta's Muse Spark from earlier in the month, at $1.25 input and $4.25 output, and you have four serious models fighting on price in a single window.

The floor is collapsing

Put the numbers next to each other and the trend is obvious. Not long ago, top-tier models cost $5 input and $25 to $30 output per million tokens. Now competitive models are landing at a quarter of that, and open models are threatening to take the price to near zero for anyone willing to run them.

This is a classic price war. Each player is willing to compress margins to win developers, because whoever becomes the default tool captures years of spend.

Why it's happening now

Two forces are colliding.

First, real competition. Anthropic pulled ahead largely on the strength of Claude Code, OpenAI is pushing hard on enterprise with Codex, and Google and Microsoft are using their cloud businesses and balance sheets to catch up. When four giants want the same developers, price becomes a weapon.

Second, buyers finally started counting. After two years of spending on AI without checking the bill, companies are under real pressure to justify the cost. A cheaper model that's good enough is suddenly very attractive, and vendors know it. Cheap models are arriving exactly as buyers get cost-conscious, and that's not a coincidence.

Who wins and who sweats

Buyers win, clearly. If you're paying for AI, your per-token cost is dropping while quality climbs. That's a rare combination.

The vendors are in a harder spot. The AI coding market is worth around $4 billion, with GitHub Copilot, Claude Code, and Cursor each reportedly past $1 billion in annual revenue, and Cursor raising at a $50 billion-plus valuation. Those numbers assume growth, but a price war compresses the margins underneath them. Someone eventually has to make money, not just market share.

Open models like Kimi K3 are the wild card. If open weights keep closing the quality gap, they cap how much anyone can charge for the middle of the market.

What this means for you

A few practical takeaways if you build or buy AI.

Don't lock in. Prices and rankings are changing month to month, so avoid long contracts that assume today's pricing is permanent. Revisit your model choice quarterly, not yearly.

Pick by fit and cost, not brand. For a lot of routine work, a cheaper model is genuinely good enough. Save the premium models for the tasks that need them, and route the rest to whatever is cheapest that clears the bar.

Watch the open models. Kimi K3 and its peers may not top every benchmark, but "free to run and good enough" reshapes budgets fast.

The bottom line

Three launches in a month, each cheaper than the last, is not a coincidence. It's a price war, and for once the people paying the bills are the ones winning. The smart move isn't picking a favorite. It's staying flexible, choosing models by fit and cost, and revisiting that choice often, because whatever is cheapest and good enough today will probably be undercut again next month.

The Real Cost of Running Your Own LLM vs Calling an API

Muskan Bandta — Fri, 17 Jul 2026 06:32:46 +0000

"Just self-host an open model, it's free." I've heard this a lot, and it's one of the most expensive sentences in tech. Open weights are free. Running them is not. Here's the honest cost comparison between hosting your own LLM and just calling an API, with real 2026 numbers.

The short answer

For most teams, calling an API is cheaper, and it stays cheaper until you're pushing serious volume, roughly 5 to 10 million tokens a month on a premium model. Past that, self-hosting can win on raw compute, but hidden costs like engineering time often erase the savings anyway. Scale and specific needs decide it, not vibes.

How API pricing works

The API model is simple: you pay per token, and someone else owns the hardware. Prices in 2026 keep falling. Claude Sonnet 5 runs about $2 per million input tokens and $10 per million output, OpenAI's GPT-5.6 family starts near $1 per million input, and cheaper frontier models like Meta's Muse Spark sit around $1.25 input and $4.25 output.

The appeal is that you pay for exactly what you use and nothing when you're idle. No servers to babysit, no 3 a.m. GPU failures. For spiky or low-volume workloads, that's hard to beat.

How self-hosting actually costs money

Self-hosting flips the model. You rent or buy GPUs and pay for them whether you're using them or not.

A single NVIDIA H100 runs about $2 to $3 per hour on specialized GPU clouds like Lambda, RunPod, or CoreWeave, and notably more on the big hyperscalers, closer to $7 on AWS and $12 on Azure. Run one H100 around the clock and you're looking at roughly $1,800 to $2,000 a month, for one GPU. Serious models need more than one.

That's the number people quote. It's also the number that lies to you.

The break-even math

Let's do the honest comparison. On a single on-demand H100 running a 70-billion-parameter open model efficiently, self-hosting starts to beat premium API pricing somewhere around 5 to 10 million tokens per month, or very roughly 500,000 tokens a day sustained.

Below that, the API wins easily, because you'd be paying $2,000 a month for a GPU that mostly sits idle. Above it, and especially at 100 million-plus tokens a month, the math tilts hard toward self-hosting, where large deployments can save real money.

But that's only the compute line.

The costs nobody puts in the spreadsheet

Here's where the "free model" story falls apart. Raw GPU cost is only about 30 to 40 percent of the true cost of running your own LLM. Plan for a 2.5 to 3x multiplier once you add networking, storage, redundancy, and monitoring.

Then there's the big one: people. Keeping a self-hosted model reliable takes real engineering. Estimates put a minimum viable team at 1.5 to 2 full-time engineers, which is $270,000 to $550,000 a year in salary alone. That "free" open-source model can quietly cost half a million dollars a year in engineering time before you count a single GPU.

An API has none of that. The provider eats the ops burden. You're paying a premium per token precisely so you don't need that team.

When self-hosting actually wins

It's not always about cost. Self-hosting makes sense when:

You're at real scale, processing tens or hundreds of millions of tokens a month consistently, so the GPUs stay busy. Idle GPUs are what kill the economics.

You have hard data or compliance requirements, where sending data to a third-party API isn't allowed. Sometimes control, not cost, is the reason.

You need predictable latency or heavy customization that a shared API can't give you.

If none of those apply, you're probably paying a lot to feel in control.

The hybrid reality

Most teams that get this right don't pick one and commit forever. They start on APIs because it's fast and cheap at low volume, then move the highest-volume, most stable workloads to self-hosting once the numbers clearly justify it, while keeping everything else on APIs. Cost follows usage instead of ideology.

The bottom line

Open weights are free. Running an LLM is not. APIs win for most teams because you skip the GPUs, the ops, and the 2 extra engineers, and you only pay for what you use. Self-hosting wins at real scale or when compliance forces your hand, but only if you count the full cost, not just the GPU sticker price. Do the token math first, add the hidden 60 percent, and you'll make the call for the right reasons instead of the "it's free" myth.

I Let an AI Agent Run My Cloud Ops for a Week: Here's What Broke

Muskan Bandta — Fri, 17 Jul 2026 06:00:25 +0000

Everyone keeps saying AI agents can run your operations now. So I gave one seven days to try, on a real environment, not a demo. Here's what actually happened, including the part that nearly went wrong.

The setup

I connected an AI agent to my monitoring stack and cluster with read access first, then limited write access for a few safe actions like restarting a stuck pod or scaling a deployment. It could see logs, metrics, and recent deploys, and it could suggest or take a small set of pre-approved actions. Anything bigger needed my sign-off.

The rule I set for myself: treat it like a new junior engineer on their first on-call. Useful, fast, and absolutely not trusted with the keys yet.

What went well

The triage speed was the real surprise. When an alert fired, the agent had already pulled the related logs, the metrics around the spike, and the last deploy before I even opened my laptop. On a normal week, gathering that context is the slow part. It just handed it to me.

Over the seven days there were 11 alerts worth looking at. For the routine ones, a pod stuck in a crash loop, a node running hot, the agent correctly identified the cause about 8 out of 11 times. My rough mean time to resolution dropped from around 45 minutes to closer to 20 on those incidents, mostly because the boring detective work was already done.

It also caught something I would have missed: a slow memory climb on a service that hadn't alerted yet. It flagged the trend and suggested a restart before it became a 2 a.m. page. That one save alone made the week feel worth it.

What broke

Now the honest part, because the hype posts skip this.

It was confidently wrong 3 times out of 11. Once it blamed a deploy for an issue that was actually a downstream database problem. The explanation was clean, professional, and completely wrong. If I hadn't checked, I would have rolled back a perfectly good release and still had the outage.

It also struggled with anything it hadn't seen before. For known patterns it was great. For a weird networking issue that needed tribal knowledge about our setup, it flailed and kept suggesting generic fixes. It didn't know what it didn't know, which is exactly the trait you don't want in ops.

The near-miss

Here's the part that stuck with me. During one incident, the agent proposed scaling a deployment from 3 replicas up to 30 to "handle load." The load wasn't the problem, a bad config was. If that action had run automatically, it would have multiplied a broken service tenfold and spiked the bill, all while fixing nothing.

It didn't run, because I'd kept that kind of action behind manual approval. But it was a clear lesson: the agent's confidence and its correctness are two different things, and the gap between them is where you get hurt. The guardrail did its job.

What it cost

Worth mentioning, since AI spend is under scrutiny this year. The token usage for a week of this came to roughly $30. Small next to the time saved, but easy to imagine that climbing fast if an agent runs loose across a big environment with no limits. Budget caps aren't optional here.

What I actually learned

Three things.

First, the value is in the boring 80 percent. Context gathering, correlation, first-pass triage, that's where an agent genuinely saves time. Let it do that.

Second, keep the risky 20 percent behind a human. Anything that changes production state, scaling, rollbacks, deletes, should need approval until you deeply trust the setup. My near-miss would have been a real incident without that rule.

Third, it doesn't replace knowing your systems. The agent was a force multiplier for an engineer who could sanity-check it. For someone who couldn't, it would have been a very fast way to make confident mistakes.

Would I keep it

Yes, but exactly the way I ran it. Read access wide open, write access narrow and approved, budget capped, and me still in the loop for anything that matters. In that shape it made my week genuinely easier and caught something I'd have missed.

The bottom line

An AI agent can't run your cloud ops on its own yet, and anyone claiming otherwise hasn't watched one confidently suggest scaling a broken service by 10x. But as a triage partner that does the tedious first 80 percent while you keep control of the dangerous 20, it's already useful today. Give it the boring work, keep the guardrails tight, and never confuse how sure it sounds with how right it is.

What Is CI/CD? Explained with a Real Pipeline Example

Muskan Bandta — Wed, 15 Jul 2026 05:33:31 +0000

CI/CD is one of those terms that gets used constantly and explained rarely. If you've been nodding along in meetings without being totally sure what it means, this is for you. I'll explain it in plain terms, then show you a real pipeline you can read line by line.

What CI/CD actually means

CI/CD stands for Continuous Integration and Continuous Delivery (or Deployment). In short, it's automation that takes your code from "I just pushed a change" to "it's tested and released" without you doing it all by hand.

It splits into two halves.

Continuous Integration (CI) means every time someone pushes code, it's automatically built and tested. The goal is to catch problems early, while the change is small and fresh in your mind, instead of days later when it's tangled up with everyone else's work.

Continuous Delivery (CD) means that once the code passes, it's automatically prepared for release, and often deployed. Continuous Delivery usually means it's ready to ship at the click of a button. Continuous Deployment goes one step further and ships it automatically, with no human in the loop.

Put together, CI/CD is a conveyor belt for your code: push, build, test, deploy, with machines doing the repetitive parts.

Why it matters

Before CI/CD, releasing software was a scary, manual event. Someone would run tests on their laptop, build by hand, copy files to a server, and hope. Releases were rare and stressful, so bugs piled up between them.

CI/CD flips that. When shipping is automated and safe, you ship small changes often. The best-performing teams, by the industry's DORA research, deploy many times a day, while struggling teams deploy once a month or less. Small, frequent releases are easier to test and easier to roll back when something goes wrong. That's the real payoff: less risk, not just more speed.

A real pipeline example

Here's a working GitHub Actions pipeline for a Node.js app. GitHub Actions has been generally available since 2019 and is free to start with, which makes it a great first tool. Save this as .github/workflows/ci.yml in your repo:

name: CI/CD Pipeline

on:
  push:
    branches: [ main ]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test

  deploy:
    needs: build-and-test
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to production
        run: ./deploy.sh

Reading the pipeline stage by stage

Let's walk through what this actually does, because once you can read one pipeline, you can read most of them.

The on block is the trigger. This one runs whenever code is pushed to the main branch. That's the "continuous" part, it fires automatically, not when you remember to run it.

The build-and-test job is the CI half. It checks out your code, installs Node.js version 20, installs dependencies with npm ci, and runs your tests. If the tests fail, the pipeline stops here and the change never reaches production. That's the safety net.

The deploy job is the CD half. The needs: build-and-test line is important: it means deploy only runs if the tests passed first. No green tests, no deploy. When it does run, it executes your deploy script and ships the change.

So the full story of one push: you commit to main, the pipeline builds and tests automatically, and if everything is green, it deploys. If a test breaks, you find out in minutes and nothing bad ships.

Common stages you'll add later

Real pipelines grow more steps over time. A linter to catch style issues, a security scan, a build step that creates a Docker image, and a staging deploy before production are all common. But the shape stays the same: trigger, test, and only deploy if the tests pass. Everything else is detail.

Tips if you're just starting

Start tiny. A pipeline that only runs your tests on every push is already worth it, and you can add deployment later. Keep your steps fast, because a pipeline that takes 30 minutes is one people learn to ignore. And never let deploy run without tests passing first, that one rule prevents most bad releases.

The bottom line

CI/CD is automation that turns a code push into a tested, released change. CI is the "build and test everything automatically" half, CD is the "release it safely" half. Together they let you ship small changes often instead of big scary ones rarely. Start with a simple test-on-every-push pipeline like the one above, get it green, and grow it as you go. It's one of the most useful things you can add to any project.

What Is Infrastructure as Code? A Beginner's Guide

Muskan Bandta — Wed, 15 Jul 2026 05:33:30 +0000

If you've ever set up a server by clicking through a cloud console, you already know the problem Infrastructure as Code solves. You click for an hour, it works, and three months later nobody remembers exactly what you did. Infrastructure as Code fixes that. Here's what it means, in plain terms.

What Infrastructure as Code actually is

Infrastructure as Code (IaC) is the practice of managing your infrastructure, servers, networks, databases, load balancers, through code and config files instead of clicking around by hand.

Instead of opening the AWS console and creating a server manually, you write a file that describes the server you want. A tool reads that file and creates it for you. The file becomes the single source of truth, and you can save it, share it, and rerun it any time.

That's the whole idea. Your infrastructure stops being a thing you build by memory and becomes a thing you write down.

The problem it solves

Manual setup breaks down in three ways, and every team hits all three eventually.

First, it isn't repeatable. Setting up one server by hand is fine. Setting up the same thing for dev, staging, and production, three times, by memory, is how tiny differences sneak in. Those differences cause the classic "it works in staging but not production" nightmares.

Second, there's no record. When a server is set up by clicking, the only documentation is whatever someone remembered to write down, which is usually nothing. If that person leaves, the knowledge leaves with them.

Third, things drift. Someone makes a "quick fix" in the console at 2 a.m. during an incident, and now reality doesn't match anything. Over months, your servers become snowflakes, each slightly different and impossible to reproduce.

IaC kills all three. The code is the record, it runs the same way every time, and you can spot drift by comparing reality to the file.

How it works

Most modern IaC tools are declarative, which is the key concept to understand. You describe the end state you want, not the steps to get there. The tool figures out the steps.

Here's a tiny Terraform example that creates a storage bucket:

resource "aws_s3_bucket" "reports" {
  bucket = "my-company-reports"
}

You don't tell it how to make a bucket. You say "a bucket named my-company-reports should exist," and the tool makes reality match. Run it once, it creates the bucket. Run it again, it sees the bucket already exists and does nothing. That property, where running the same file repeatedly is safe, is called idempotency, and it's what makes IaC trustworthy.

The usual workflow has two steps. You run a plan to preview what will change, then an apply to actually make the changes. The plan step is a lifesaver, because you see exactly what's about to happen before it happens.

The tools you'll hear about

A few names come up constantly. Terraform, released in 2014, is the most widely used and defines infrastructure in a language called HCL. OpenTofu is its open-source fork and works almost identically. Pulumi lets you use real programming languages like Python or TypeScript instead. Ansible, from 2012, leans more toward configuring existing servers. AWS CloudFormation is Amazon's own, locked to AWS.

If you're starting fresh, Terraform or OpenTofu is the safest first choice, because the concepts you learn there carry over everywhere.

Why it's worth learning

Once your infrastructure is code, good things follow almost for free. You can put it in Git, so every change has history and an author. You can review infrastructure changes in a pull request, the same way you review app code. You can spin up an identical environment in minutes instead of a day. And you can tear it all down and rebuild it exactly, which is priceless when something breaks.

How to start

Don't try to codify your whole company on day one. Pick one small, real thing, a single bucket, one server, a DNS record, and write it in Terraform. Run the plan, read what it says, then apply. Change one value and run it again so you see how updates work. That loop, edit, plan, apply, is the entire skill, and an afternoon with it teaches more than a week of reading.

The bottom line

Infrastructure as Code means describing your infrastructure in files instead of building it by hand. It makes your setup repeatable, reviewable, and recoverable, and it turns tribal knowledge into something you can actually read. Start with one tool and one small resource, get comfortable with the plan-and-apply loop, and grow from there. It's one of the highest-leverage skills you can pick up in cloud and DevOps.