DEV Community

jidonglab profile picture

jidonglab

1 project a week. Building and sharing the entire process — from idea to shipped product in 7 days. Currently: AI news automation.

Attention Sinks: Why Streaming LLMs Break When You Evict Token 0

Attention Sinks: Why Streaming LLMs Break When You Evict Token 0

Comments
6 min read

Want to connect with jidonglab?

Create an account to connect with jidonglab. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
Why Temperature 0 Doesn't Make Your LLM Deterministic

Why Temperature 0 Doesn't Make Your LLM Deterministic

Comments
6 min read
Speculative Decoding: Why a Great Draft Model Still Caps Speedup

Speculative Decoding: Why a Great Draft Model Still Caps Speedup

Comments
6 min read
Constrained Decoding: Force Valid JSON Without Wrecking Accuracy

Constrained Decoding: Force Valid JSON Without Wrecking Accuracy

Comments
6 min read
GRPO Explained: Why DeepSeek Dropped the Critic in RLHF

GRPO Explained: Why DeepSeek Dropped the Critic in RLHF

Comments
6 min read
Online Softmax: How FlashAttention Skips the N N Matrix

Online Softmax: How FlashAttention Skips the N N Matrix

Comments
7 min read
Late Interaction Retrieval: Why ColBERT Beats Single-Vector RAG

Late Interaction Retrieval: Why ColBERT Beats Single-Vector RAG

Comments
7 min read
Why repetition_penalty Quietly Corrupts Your Code Generation

Why repetition_penalty Quietly Corrupts Your Code Generation

Comments
6 min read
DPO Likelihood Displacement: When Preferred Answers Get Rarer

DPO Likelihood Displacement: When Preferred Answers Get Rarer

Comments
6 min read
Why Token Logprobs Beat Asking Your LLM How Confident It Is

Why Token Logprobs Beat Asking Your LLM How Confident It Is

Comments
7 min read
Multi-Token Prediction: DeepSeek's Built-In Draft Model

Multi-Token Prediction: DeepSeek's Built-In Draft Model

Comments
6 min read
AWQ: How Activation-Aware Quantization Saves 4-bit LLMs

AWQ: How Activation-Aware Quantization Saves 4-bit LLMs

Comments
6 min read
PagedAttention: Why Static Batching Wastes Your KV Cache

PagedAttention: Why Static Batching Wastes Your KV Cache

Comments
6 min read
11 Claude Agents Audited My Kit. 2 'Major Bugs' Were Fake

11 Claude Agents Audited My Kit. 2 'Major Bugs' Were Fake

Comments 1
6 min read
Semantic Entropy: Detect LLM Hallucinations Without Ground Truth

Semantic Entropy: Detect LLM Hallucinations Without Ground Truth

Comments
6 min read
Contextual Retrieval: Fix the RAG Chunk That Lost Its Context

Contextual Retrieval: Fix the RAG Chunk That Lost Its Context

Comments
6 min read
Prefilling Claude's Response: Steer Output Without JSON Mode

Prefilling Claude's Response: Steer Output Without JSON Mode

Comments
6 min read
I Cut My Claude Code Prompt Overhead 84% Per Turn (v1.4)

I Cut My Claude Code Prompt Overhead 84% Per Turn (v1.4)

Comments
7 min read
Multi-head Latent Attention: The KV Cache Trick Beyond GQA

Multi-head Latent Attention: The KV Cache Trick Beyond GQA

Comments
7 min read
Why Your Mixture-of-Experts Model Silently Drops Tokens

Why Your Mixture-of-Experts Model Silently Drops Tokens

Comments
7 min read
Embedding Anisotropy: Why Cosine Similarity Never Hits Zero

Embedding Anisotropy: Why Cosine Similarity Never Hits Zero

Comments
6 min read
Return Claude's Thinking Blocks or Your Agent Breaks

Return Claude's Thinking Blocks or Your Agent Breaks

Comments
6 min read
Binary Quantized Embeddings: 32x Smaller Vectors, Recall Intact

Binary Quantized Embeddings: 32x Smaller Vectors, Recall Intact

Comments
7 min read
Prefill/Decode Disaggregation: Stop Serving LLMs on One GPU

Prefill/Decode Disaggregation: Stop Serving LLMs on One GPU

Comments
6 min read
Chunked Prefill: Why One Long Prompt Freezes Your LLM Server

Chunked Prefill: Why One Long Prompt Freezes Your LLM Server

Comments 1
7 min read
How I Made Opus 4.8 Act Like Fable 5 (64% 97%, Measured)

How I Made Opus 4.8 Act Like Fable 5 (64% 97%, Measured)

Comments
7 min read
FP8 KV Cache Quantization: The Memory Math and the Accuracy Cliff

FP8 KV Cache Quantization: The Memory Math and the Accuracy Cliff

Comments
6 min read
Why LLM Decoding Is Memory-Bound: Prefill vs Decode Roofline

Why LLM Decoding Is Memory-Bound: Prefill vs Decode Roofline

Comments
6 min read
Min-p Sampling: Why Top-p Breaks at High Temperature

Min-p Sampling: Why Top-p Breaks at High Temperature

Comments
7 min read
Matryoshka Embeddings: Truncate Vectors 12x Without Losing Recall

Matryoshka Embeddings: Truncate Vectors 12x Without Losing Recall

Comments
6 min read
The Hidden Token Tax of Tool Use in LLM Agents

The Hidden Token Tax of Tool Use in LLM Agents

Comments
7 min read
Grouped-Query Attention: The KV Cache Math Behind Long Context

Grouped-Query Attention: The KV Cache Math Behind Long Context

Comments
6 min read
RoPE Scaling: How LLMs Stretch From 8K to 128K Context

RoPE Scaling: How LLMs Stretch From 8K to 128K Context

Comments
7 min read
Prompt Caching With Claude: Where the Cache Breakpoint Goes

Prompt Caching With Claude: Where the Cache Breakpoint Goes

Comments
6 min read
Attention Sinks: Why Evicting Your LLM's First Token Breaks It

Attention Sinks: Why Evicting Your LLM's First Token Breaks It

Comments
7 min read
Why Your LLM-as-Judge Disagrees With Itself (And How to Fix It)

Why Your LLM-as-Judge Disagrees With Itself (And How to Fix It)

Comments
7 min read
Speculative Decoding: Why Two Models Decode Faster Than One

Speculative Decoding: Why Two Models Decode Faster Than One

Comments
7 min read
Structured Output Isn't Free: The Constrained-Decoding Tax

Structured Output Isn't Free: The Constrained-Decoding Tax

Comments
6 min read
71,700 Stars and 60 Rust Crates: Inside OpenAI's Codex CLI Source

71,700 Stars and 60 Rust Crates: Inside OpenAI's Codex CLI Source

Comments
6 min read
Pentagon Blacklisted Anthropic From 8 Classified AI Deals

Pentagon Blacklisted Anthropic From 8 Classified AI Deals

Comments
7 min read
Anthropic $900B: 2.4x in 90 Days, 48-Hour Window

Anthropic $900B: 2.4x in 90 Days, 48-Hour Window

Comments
5 min read
Symphony: Why OpenAI's PRs Jumped 500% in 3 Weeks

Symphony: Why OpenAI's PRs Jumped 500% in 3 Weeks

Comments
5 min read
GPT Image 2 Inside Codex: My New Frontend Workflow

GPT Image 2 Inside Codex: My New Frontend Workflow

1
Comments
6 min read
GPT-5.5-Codex vs 5.3: A 200-Task Bench Result

GPT-5.5-Codex vs 5.3: A 200-Task Bench Result

Comments
6 min read
Codex Is No Longer a CLI. Embed It in Your App.

Codex Is No Longer a CLI. Embed It in Your App.

Comments
6 min read
I Gave Codex My Mouse for a Day. Here's What Broke.

I Gave Codex My Mouse for a Day. Here's What Broke.

Comments
6 min read
OpenAI's Super App Play: Why Spud + Duct Tape Matter for Builders

OpenAI's Super App Play: Why Spud + Duct Tape Matter for Builders

Comments
8 min read
Why OpenAI Shipped GPT-5.5 Just 6 Weeks After 5.4

Why OpenAI Shipped GPT-5.5 Just 6 Weeks After 5.4

Comments
8 min read
OpenCode Hit 140K Stars. Why Terminal Agents Won 2026.

OpenCode Hit 140K Stars. Why Terminal Agents Won 2026.

Comments
7 min read
How a Markdown File Hit 16K Stars: Skills in 2026

How a Markdown File Hit 16K Stars: Skills in 2026

1
Comments
7 min read
MIT Tech Review Just Split Its \"Breakthrough\" List. That's the Story.

MIT Tech Review Just Split Its \"Breakthrough\" List. That's the Story.

Comments
7 min read
Stellantis Just Outsourced Its AI Moat to Microsoft. Expect GM, Ford, and VW to Follow.

Stellantis Just Outsourced Its AI Moat to Microsoft. Expect GM, Ford, and VW to Follow.

Comments
8 min read
44% of New Music on Deezer Is AI. Only 0.5% of Streams Are. Read That Twice.

44% of New Music on Deezer Is AI. Only 0.5% of Streams Are. Read That Twice.

Comments
7 min read
Adobe Just Made MCP an Enterprise Procurement Line Item

Adobe Just Made MCP an Enterprise Procurement Line Item

Comments
7 min read
Claude Opus 4.7 Hit 87.6% on SWE-bench. The Story Is What It Didn't Charge You.

Claude Opus 4.7 Hit 87.6% on SWE-bench. The Story Is What It Didn't Charge You.

Comments
7 min read
I Built an AI Newsletter for Myself. 11 Subscribers, 49 Posts, Zero Regrets.

I Built an AI Newsletter for Myself. 11 Subscribers, 49 Posts, Zero Regrets.

Comments
6 min read
DataForge, Atropos, and a 30K-Token Guillotine: Reverse-Engineering Hermes 4's Training Stack

DataForge, Atropos, and a 30K-Token Guillotine: Reverse-Engineering Hermes 4's Training Stack

Comments
8 min read
The Honest Hermes 4 Production Checklist (April 2026 Edition)

The Honest Hermes 4 Production Checklist (April 2026 Edition)

Comments
7 min read
66.5% of My Claude Code Tokens Were Wasted. A 200-Line Wrapper Got Them Back.

66.5% of My Claude Code Tokens Were Wasted. A 200-Line Wrapper Got Them Back.

1
Comments
7 min read
Hermes 4's Tool-Calling Is Trained as a Separate Skill. Here's Why Your Agent Cares.

Hermes 4's Tool-Calling Is Trained as a Separate Skill. Here's Why Your Agent Cares.

Comments
6 min read
loading...