DEV Community: Arun Kumar

MCP doesn't replace function calling. Your MCP client still emits one.

Arun Kumar — Thu, 24 Sep 2026 15:23:08 +0000

"Do we still need APIs after MCP" keeps coming up, and the framing is off in a way that's worth unpicking. MCP and function calling aren't two options you choose between. They're two layers of the same request, and a system built on MCP runs function calling underneath it every single time.

The two gaps

A model can't act on its own. It reads text and predicts more text. Two separate gaps sit between it and anything useful, and each needs its own fix.

The first gap is expression — the model needs a structured way to say what it wants done. Function calling fills that. It's a capability of the model provider's API, not a protocol: your application sends tool definitions with the request, and the model returns a structured call naming one of them.

The part people skip: the model doesn't run anything. It hands back a request. Your code executes the function and feeds the result into the next turn.

The second gap is reach. Your application still has to find the tool, get its schema, and call it correctly — usually across a lot of tools. That's MCP's job. Client-server, JSON-RPC 2.0, discovery at runtime via tools/list.

Function calling links the model to your app. MCP links your app to the tool server.

What actually happens when both are in play

Five steps, fixed order:

The MCP client calls tools/list on a server and gets tool definitions back, each with a JSON Schema.
Your application hands those definitions to the model through the provider's function-calling mechanism.
The model emits a function call — tool name, arguments.
Your application routes that call to the MCP server as tools/call.
The server executes it, and the result goes back into the conversation.

Steps 2 and 3 are function calling. Adopting MCP doesn't remove them, it feeds them. Note also that the model never talks to the MCP server directly — your application sits in the middle the whole time, translating between a function call and a tools/call.

So what is MCP actually for

Not model capability. Models could call functions long before MCP existed.

It's the N×M problem. Without a shared protocol, wiring N applications to M tools means N×M integrations — three apps each needing five tools is fifteen separate pieces of work, none of which carry over. Wrap each tool once as a server, implement the client once per application, and the total becomes N+M.

That's the real argument, and it's a good one. It just has nothing to do with whether a model can express a tool call.

The part that gets overstated

This is the bit I'd most want people to check themselves, because plenty of write-ups claim MCP ships approval workflows and audit logging out of the box.

Here's the specification text:

the protocol itself does not mandate any specific user interaction model. For trust & safety and security, there SHOULD always be a human in the loop with the ability to deny tool invocations.

SHOULD. The spec states up front that its key words are to be read per RFC 2119, so that's a recommendation aimed at client implementers, not something the protocol enforces. Nothing in MCP stops a client skipping the check.

What you do get is a seam. Tools sit behind a server boundary, which is a natural place to put authentication, logging and policy. That part is real and genuinely useful. But you build it yourself. And the boundary means trusting a new party — the server is now part of your threat model, which isn't a free win.

Where I'd leave it

Reach for function calling whenever a model needs to act. That's every agentic flow, MCP or not.

Reach for MCP once tools need reuse across applications, or once they live behind a server you don't control. A single script calling one local function probably doesn't need it.

Most production systems end up running both, because they're solving different problems. "Do we still need APIs after MCP" has the same answer as "do we still need function calling after MCP" — yes, underneath, every time.

Longer version with the full comparison table at diffstudy.com. Spec quote from the MCP specification, 2025-06-18, Server/Tools.

GRPO doesn't remove the reward model. It removes the critic.

Arun Kumar — Fri, 18 Sep 2026 04:00:00 +0000

Every time GRPO comes up I see the same slip — someone says it "gets rid of the reward model". It doesn't. It gets rid of the value model. Those are two different networks doing unrelated jobs, and telling them apart is most of understanding the algorithm.

Worth going back to what the DeepSeekMath paper actually claims, because it's more modest than the version that travels.

It's a variant, not a successor

Shao and colleagues introduce GRPO as "a variant of Proximal Policy Optimization (PPO)". Not a replacement. A variant. It keeps PPO's clipped ratio, and it keeps the reference model. Three things change: the critic goes, the baseline becomes group-relative, and the KL term moves.

That's narrower than "GRPO replaces PPO", which is how I usually see it written up.

What the critic was costing

PPO is actor-critic, so the critic gets trained alongside the policy. The paper is blunt about the bill. That value function is "typically another model of comparable size as the policy model", and so "it brings a substantial memory and computational burden".

You're training two large models to ship one.

There's a second complaint I found more interesting than the memory one. In language-model RL, "usually only the last token is assigned a reward score by the reward model". A signal that sparse "may complicate the training of a value function that is accurate at each token". So the critic is expensive and hard to fit well. Both halves matter, and only the first one usually gets mentioned.

What replaces it

Sampling. For each question GRPO samples a group of outputs, scores every one with a reward model — still present, still doing its job — and uses the group's average as the baseline. Those rewards get "normalized by subtracting the group average and dividing by the group standard deviation".

So an output's advantage answers a relative question: was this better or worse than its siblings? The paper argues that suits how reward models are built in the first place, since they're trained on comparisons between outputs for the same question.

Model count goes four to three:

PPO: policy, reference, reward, value
GRPO: policy, reference, reward

The reward model is right there in the second list. It never left.

The trade that isn't in the headline

None of this is free. You now need several generations per question instead of one. DeepSeek-R1-Zero sampled 16 outputs per question.

GRPO trades memory for generation. Memory-bound, that's a good deal. Throughput-bound, maybe not.

The KL term moved, it didn't vanish

Second thing I see stated wrong: that GRPO drops KL regularisation. PPO adds "a per-token KL penalty from a reference model in the reward at each token". GRPO works "instead of adding KL penalty in the reward" and "regularizes by directly adding the KL divergence" to the loss.

Same idea, different address. GRPO also uses an unbiased estimator "which is guaranteed to be positive".

The finding I didn't expect

This one rarely survives into summaries, and it's the part I'd most want people to take away.

The DeepSeekMath authors measured Pass@K and Maj@K before and after RL. The result was asymmetric: "RL enhances Maj@K's performance but not Pass@K".

Read that slowly. The model got better at reliably surfacing an answer it could already produce. It did not start producing answers that were previously out of reach. The authors draw the conclusion themselves — the gain is "attributed to boosting the correct response from TopK rather than the enhancement of fundamental capabilities".

That is a considerably narrower claim than the one usually made for RL fine-tuning.

Two things worth keeping straight

The value model went, the reward model stayed. If someone says GRPO removed the reward model, they've merged two different networks.

R1's rule-based rewards were a separate decision. DeepSeek-R1 did drop neural reward models for reasoning tasks, but for another reason entirely: "neural reward models are susceptible to reward hacking during large-scale reinforcement learning". That's a robustness call, not a memory one, and it applied to reasoning tasks specifically. It gets folded into the GRPO story where it doesn't belong.

Longer version with every quote sourced at diffstudy.com. Sources: arXiv 2402.03300 (DeepSeekMath), arXiv 2501.12948 (DeepSeek-R1), arXiv 1707.06347 (PPO).

Muon doesn't replace AdamW. Every Muon run still has AdamW in it.

Arun Kumar — Mon, 14 Sep 2026 04:00:00 +0000

Muon gets written up as the optimizer that finally beats AdamW. Reading Keller Jordan's original post, that framing falls apart in the first section — and what's actually going on is more interesting.

It was never meant to cover the whole model

Here's the definition, from the post that introduced it:

Muon is an optimizer for 2D parameters of neural network hidden layers.

Hidden layers. 2D. That's the scope, stated up front. What about everything else?

When training a neural network with Muon, scalar and vector parameters of the network, as well as the input and output layers, should be optimized by a standard method such as AdamW.

So your biases, your LayerNorm gains, your scalars — AdamW. And the input and output layers too, which is the part I found genuinely surprising:

Empirically, we find that it is also important to optimize input and output parameters using AdamW, even though these are typically 2D.

Read that again. The embedding matrix and the classifier head are 2D. They fit Muon's stated scope perfectly. And you should still use AdamW on them, because it works better. Jordan is explicit that for transformers, AdamW "should be used for the embedding and final classifier head layers in order to attain the best performance".

A transformer's embedding and unembedding are a serious fraction of its parameters. So "we switched from AdamW to Muon" almost always means "we run both, and Muon takes the middle."

Then it borrowed AdamW's defining feature

This is the part that made me laugh.

AdamW exists because of one idea: decouple weight decay from the gradient update. That's the whole paper — it's in the title, Decoupled Weight Decay Regularization. Take that away and you have Adam.

Now here's Moonshot AI's paper on scaling Muon to real model sizes. Their two contributions:

We identify two crucial techniques for scaling up Muon: (1) adding weight decay and (2) carefully adjusting the per-parameter update scale.

Contribution number one is weight decay. Muon needed AdamW's signature idea bolted on before it would scale.

That isn't a criticism of Muon — it's how optimizer research works, and the result is good. But it does make "Muon replaces AdamW" a strange thing to say about an optimizer that runs alongside AdamW and adopted its main contribution.

What it actually buys you

The numbers are real and worth knowing precisely, because they get inflated in the retelling.

From Jordan's results:

CIFAR-10 to 94% accuracy: speed record improved from 3.3 to 2.6 A100-seconds
NanoGPT speedrunning to 3.28 val loss: faster by a factor of 1.35x
A 1.5B transformer to GPT-2 XL level on HellaSwag: 10 8xH100-hours, against 13.3 for AdamW

And from Moonshot at larger scale, with scaling-law experiments behind it:

Muon achieves ∼ 2× computational efficiency compared to AdamW with compute optimal training

They trained Moonlight, a 3B/16B MoE, on 5.7T tokens with it. So this isn't a toy-scale result any more.

But notice the spread: 1.35x on NanoGPT, roughly 1.33x on the 1.5B run, around 2x in Moonshot's scaling-law setting. "2x faster than AdamW" is the number that travels, and it's the top of the range, from one paper's specific setup.

The mechanism, briefly

AdamW keeps two running averages per parameter and rescales each weight individually. Every parameter is treated as its own little scalar problem.

Muon treats a weight matrix as a matrix. It takes the momentum update and orthogonalises it using a Newton-Schulz iteration, so the update doesn't collapse toward a few dominant directions. That's why it only makes sense for 2D hidden layers — the whole idea depends on there being a meaningful matrix structure to condition.

Which also explains the embedding exception. An embedding table is 2D in shape, but it isn't doing matrix-multiply work in the same sense, and empirically it prefers AdamW.

What I'd take away

Muon is a hidden-layer optimizer, not a whole-model one. If someone says they replaced AdamW, ask what's optimizing their embeddings.

The headline number is a range, not a constant. 1.35x on speedrunning, around 2x in Moonshot's scaling-law experiments.

It's additive, not a replacement. The current best setup is Muon on 2D hidden layers plus AdamW on everything else — which is exactly what the original post recommends.

Longer version with every quote sourced at diffstudy.com. Sources: Keller Jordan's Muon post, arXiv 2502.16982 (Moonshot AI), arXiv 1711.05101 (AdamW).

"RoPE extrapolates to longer contexts" — someone finally measured it, and it doesn't

Arun Kumar — Thu, 10 Sep 2026 04:00:00 +0000

You've read this sentence, or something close to it: RoPE handles longer sequences than absolute positional encoding because rotations are periodic.

I believed it too. Then I went looking for where it was measured, and found the opposite.

The claim has a four-paper history, and following it in order is more interesting than the claim itself.

2017: a hedge

The original Transformer paper picks sinusoidal encodings over learned ones, and gives a reason:

We chose the sinusoidal version because it may allow the model to extrapolate to sequence lengths longer than the ones encountered during training.

May. Vaswani and colleagues were careful. It's a hypothesis, offered as a tiebreaker between two options that — by their own testing — "produced nearly identical results".

That word survives almost nowhere downstream. It got quoted into oblivion as a property.

2021: RoPE arrives

Su and colleagues introduce rotary embeddings and list "the flexibility of sequence length" among its properties.

Nothing wrong with that. It's a design claim about the mechanism, made before anyone had measured the ceiling. But between 2017's may and 2021's property list, "might extrapolate" quietly became "extrapolates".

2022: someone measures it

This is the paper that should have ended the folklore, and mostly didn't.

The ALiBi paper (Press, Smith and Lewis, ICLR 2022) exists to test length extrapolation, and it tests the incumbents first. On sinusoidal encodings — the thing the 2017 paper hoped would extrapolate — the verdict is that it "in practice has very limited extrapolation capabilities".

On rotary:

Though the rotary position method improves over the sinusoidal one, it still does not achieve satisfying results.

Better than sinusoids. Still not good. And they put numbers on it: a model trained at length 512 keeps improving perplexity for about 200 extra tokens. At length 1024, about 100.

Two hundred tokens. Not 2x, not 10x. And the paper notes that even this "comes at the cost of slower training and inference".

2023: the patch that proves the point

If RoPE extrapolated, Position Interpolation wouldn't need to exist.

Chen and colleagues open by naming the problem directly — many pre-trained LLMs, LLaMA included, "use positional encodings that have weak extrapolation properties" — and RoPE is the example they name. Their fix doesn't extrapolate at all. It interpolates: squeeze the position indices down into the range the model already saw.

Why not just push past the trained length? Because doing so "may lead to catastrophically high attention scores that completely ruin the self-attention mechanism".

That's the strongest evidence available. A whole technique exists because the free-extrapolation story isn't true.

The other thing everyone gets backwards

While I was in the RoFormer paper I hit a second surprise. People describe RoPE as relative positional encoding, in contrast to absolute. Here's the abstract:

the proposed RoPE encodes the absolute position with a rotation matrix and meanwhile incorporates the explicit relative position dependency in self-attention formulation

It's both. Simultaneously. And once you see the mechanism it's obvious why: each vector rotates by an angle set by its own absolute index, and when you take the dot product of a rotated query and a rotated key, the absolute angles cancel and only the offset survives.

Absolute goes in. Relative comes out. One operation.

The 2017 side wasn't purely absolute either, incidentally — the sinusoidal choice was motivated by the hope the model would "easily learn to attend by relative positions". The clean absolute/relative split was never real.

What I'd actually keep

None of this makes RoPE a bad choice. It dominates modern stacks for good reasons — it composes with the KV cache, it never touches the value vector, and it has a provable long-term decay property that sinusoids never claimed.

But three corrections stuck with me:

It doesn't extrapolate for free. Measured at ~200 extra tokens past a 512 training length. If you need a longer window you need Position Interpolation, or YaRN, or retraining. The encoding alone doesn't give it to you.

It isn't "relative instead of absolute". It's both, by construction.

The 2017 authors weren't wrong. They wrote may, and hedged correctly. The folklore is downstream of people dropping the hedge.

Longer version with every quote sourced at diffstudy.com. Papers: 1706.03762 (Transformer), 2104.09864 (RoFormer), 2108.12409 (ALiBi), 2306.15595 (Position Interpolation).

Speculative decoding won't change your model's distribution. It might still change your output.

Arun Kumar — Sun, 06 Sep 2026 05:35:06 +0000

There's a thread on the DeepSeek-R1 model page that's been sitting unresolved since March last year, and it bothered me enough to go read the papers.

Someone had tried speculative decoding and reported that the output got worse — "very low quality words for the given context, words it would never generate by itself". Someone else replied that this is impossible, because the main model verifies and corrects everything the draft model proposes. Neither budged.

They're both right. That took me a while to see.

The guarantee is real

The claim that speculative decoding can't hurt quality isn't marketing. It's proven, twice, independently.

Leviathan and colleagues at Google introduced the technique in 2022 and state it flatly: "A strong property of Algorithm 1 is that the output distribution is guaranteed to remain unchanged". Chen and colleagues at DeepMind arrived at the same place a few months later through a different proof — theirs is "a novel modified rejection sampling scheme which preserves the distribution of the target model within hardware numerics".

So the person insisting it cannot degrade quality has the papers behind them.

The guarantee is also narrower than it sounds

Here's the part almost every explainer skips, and it's sitting right there in the DeepMind paper:

because the different computation graphs lead to different numerics, we cannot not expect identical outputs

(The doubled "not" is theirs. I left it.)

Read that against the guarantee and the argument dissolves. The distribution is preserved. The sample is not the same sample. You're drawing from the same distribution through a different computation graph, with different floating-point behaviour in the batched verification pass than in a plain decode.

So on any single prompt, you can absolutely get different text. Sometimes worse-looking text. That isn't the method failing — it's what same-distribution-but-different-draw means in practice.

Which is exactly what the person reporting bad output experienced. And exactly why the person quoting the theory couldn't reproduce it.

Worth noting: Leviathan's own abstract does claim "identical outputs" for their T5X experiments. That's true for their setup — argmax sampling, one implementation. It's a property of that configuration, not a law.

It does more work, not less

This is the bit that genuinely surprised me.

I'd absorbed the idea that speculative decoding is a clever way to do less computation. It's the opposite. Leviathan has an entire section titled Number of Arithmetic Operations, and it says the number of concurrent arithmetic operations "grows by a factor of" gamma plus one. Every rejected draft token is compute you spent and threw away.

Total FLOPs go up.

It's faster anyway, because arithmetic was never what you were waiting on. From the same paper: inference from large models is "often not bottlenecked on arithmetic operations, but rather on memory bandwidth and communication". Chen puts it in five words — "Transformer sampling is typically memory bandwidth bound".

Your accelerator is sitting idle waiting on memory. Speculative decoding spends that idle compute to avoid serial steps. It's a trade, and it only pays because one side of the ledger was free.

That reframing changed how I think about the whole category. Same shape as FlashAttention, incidentally — a technique everyone describes as making attention cheaper, when it is really about memory movement.

What the numbers actually look like

Chen's Table 1, Chinchilla 70B, batch size 1, K=4:

	Result	Time per token	Speedup
Autoregressive, XSum	0.112	14.1 ms	1x
Speculative, XSum	0.114	7.52 ms	1.92x
Autoregressive, HumanEval	45.1%	14.1 ms	1x
Speculative, HumanEval	47.0%	5.73 ms	2.46x

Time per token roughly halves. Now look at the quality column — HumanEval goes up, 45.1 to 47.0.

Don't read that as an improvement. Elsewhere in the same table, XSum's greedy ROUGE-2 goes down, 0.157 to 0.156. Small movements in both directions across metrics is precisely what a distribution-preserving method should produce. If speculative decoding reliably improved scores, the theory would be wrong.

Where the speedup goes away

Both papers are more careful than the blog posts about them.

You need spare compute — Leviathan is explicit that the algorithm assumes "we have enough compute resources to support the increased concurrency". No headroom, no win.

You need a high acceptance rate. Every rejected token is wasted work.

And the draft model has to be genuinely cheap. This is the counterintuitive one: against an 11B target, Leviathan found T5-small (77M) beat both T5-base (250M) and T5-large (800M). The best draft model was the smallest one tested, not the most accurate.

One more caveat that gets dropped constantly: both papers measured at batch size 1. That's a latency benchmark. Chen notes that other techniques are the ones aimed at throughput "(at larger batch sizes)". If you're serving large batches, don't expect these numbers.

So who was right in that thread?

Both, and the disagreement was never really about speculative decoding.

The theory says your distribution is untouched. Your eyes say the text came out different. Both statements are true simultaneously, and the papers are clear about why, and nobody in the thread had read that far.

If your output is reproducibly worse across many prompts, that's an implementation problem — there's a 2025 paper showing some engines break the distribution guarantee outright once batch size exceeds one. If it's one bad generation, that's just a different draw.

I went through both papers line by line for a longer write-up at diffstudy.com, with every quote sourced. Papers are arXiv 2211.17192 (Leviathan et al.) and arXiv 2302.01318 (Chen et al.) if you'd rather go straight to them.