DEV Community: WonderLab

RAG Series (4): Document Processing — From Raw Files to High-Quality Chunks

WonderLab — Sat, 02 May 2026 03:43:58 +0000

Why "How You Cut" Matters as Much as "What You Cut"

In the first three articles, we built a working RAG pipeline and tuned the core parameters. But if you look closely at the retrieval results, you may notice a strange phenomenon:

The answer is clearly in the document, yet the Retriever can't find it. Or it finds it, but the answer is cut in half — the LLM only sees the first half of the sentence.

The problem usually lies in the chunking step.

Chunking is essentially an information splitting strategy — how you divide a 500-page book, how large each piece is, and where you make the cuts directly determines whether the reader (here, the Retriever) can quickly find what they need.

In this article, we'll process the same technical document with four different strategies so you can see the dramatic differences that "how you cut" makes.

📎 Source Code: All experiment code is open-sourced at llm-in-action/04-chunking-strategies. Clone it to reproduce the results.

Four Chunking Strategies at a Glance

Before diving in, here's a quick reference table to build intuition:

Strategy	Core Idea	Pros	Cons
Fixed Size	Cut at fixed character intervals, like scissors cutting paper	Simple, uniform chunk sizes	May cut through sentences, poor semantic integrity
Recursive Character	Try separators in priority order: paragraph → line → sentence → word	Balances semantics and uniformity	Limited Chinese support (uses English punctuation)
Semantic Chunking	Compute semantic similarity between adjacent sentences, cut where similarity drops	Highly semantically coherent chunks	Requires Embedding API, higher cost
Document Structure	Split by Markdown/HTML heading hierarchy	Preserves document structure, retrieved chunks carry chapter context	Only works for structured documents

Experimental Design

Test Document and Source Code

The full runnable code is available at llm-in-action/04-chunking-strategies, including:

chunking_compare.py — The 4-strategy comparison script
data/sample-tech-doc.md — Sample Markdown technical document
.env.example — Environment variable template (SemanticChunker requires an Embedding API)

Test Document

We'll use a ~5,400-character Markdown technical document titled "Microservices Architecture Design Guide," containing 7 top-level chapters with multiple level-2 and level-3 headings, covering service decomposition, communication protocols, data consistency, observability, security, and deployment.

Strategy Configurations

Strategy	Key Configuration
Fixed Size	`CharacterTextSplitter(chunk_size=512, chunk_overlap=50)`
Recursive Character	`RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=50, separators=["\n\n", "\n", ". ", " ", ""])`
Semantic	`SemanticChunker(embeddings, breakpoint_threshold_type="percentile", breakpoint_threshold_amount=85, sentence_split_regex=r"(?<=[。！？.?!])\s+", buffer_size=0)`
Document Structure	`MarkdownHeaderTextSplitter(headers_to_split_on=[("#", "Header 1"), ("##", "Header 2"), ("###", "Header 3")])`

About buffer_size=0: SemanticChunker defaults to concatenating neighboring sentences before computing embeddings (buffer_size=1 means 1 sentence on each side). But SiliconFlow's BGE model limits single inputs to < 512 tokens, so concatenation often exceeds this. Setting it to 0 makes each sentence independent — we lose some context, but it runs stably.

Strategy 1: Fixed-Size Chunking

The Principle

The most brute-force approach: cut at fixed-length intervals regardless of content.

Imagine using scissors to snip every 512 characters. Simple and efficient, but you might cut right through the middle of a sentence.

The Code

from langchain_text_splitters import CharacterTextSplitter

splitter = CharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    length_function=len,
    separator="\n",  # Prefer line breaks; hard-cut if none exist
)
chunks = splitter.split_documents(documents)

Results

Metric	Value
Chunk count	12
Average length	453.5 chars
Max length	506 chars
Min length	128 chars

First 3 chunks:

Chunk 1 (489 chars):
# Microservices Architecture Design Guide This article covers...

Chunk 2 (504 chars):
- **Read Service vs Write Service**: In read-heavy scenarios...

Chunk 3 (457 chars):
**gRPC** is based on HTTP/2 and Protocol Buffers. Advantages:...

The Problem:

Notice how Chunk 2 starts: - **Read Service vs Write Service**... — this is the middle of a list item. Fixed-size chunking brutally cut off the list at the end of the previous chunk, so Chunk 2 starts with an incomplete list item. If the user asks "What are the advantages of read-write separation?", the Retriever might return this chunk, but the LLM sees incomplete information.

Strategy 2: Recursive Character Chunking

The Principle

Slightly smarter than fixed-size: it has a priority list of separators and tries them in order — first by paragraph (\n\n), then by line (\n), then by sentence (.), and finally by word ().

Like an experienced editor: prefer cutting at paragraph boundaries, fall back to sentence boundaries if necessary, and never cut in the middle of a word.

The Code

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    length_function=len,
    separators=["\n\n", "\n", ". ", " ", ""],
)
chunks = splitter.split_documents(documents)

Results

Metric	Value
Chunk count	13
Average length	431.5 chars
Max length	507 chars
Min length	88 chars

First 3 chunks:

Chunk 1 (441 chars):
# Microservices Architecture Design Guide  This article covers...

Chunk 2 (452 chars):
### 1.2 Split by Technical Characteristics  Besides business boundaries...

Chunk 3 (457 chars):
The most common synchronous communication methods between microservices are...

Improvement over fixed-size:

Chunk 2 now starts with ### 1.2 Split by Technical Characteristics — a complete heading. Recursive character chunking successfully cut at a heading boundary instead of slicing through a list item.

But note that the separators list uses . (English period + space), so for Chinese documents, it won't split on Chinese periods (。). Its behavior on Chinese text is therefore close to fixed-size, relying mainly on \n\n and \n.

Strategy 3: Semantic Chunking

The Principle

The previous two strategies cut by length. Semantic chunking cuts by meaning.

Here's how it works:

Split the document into sentences
Compute each sentence's embedding (semantic vector)
Compare semantic similarity between adjacent sentences
If similarity suddenly drops (below the threshold), cut there

Imagine watching a movie where the scene suddenly shifts from an office to a beach — that's a semantic boundary. Semantic chunking recognizes these "scene changes" and ensures each chunk discusses one coherent topic.

The Code

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="BAAI/bge-large-zh-v1.5",
    api_key=os.getenv("EMBEDDING_API_KEY"),
    base_url=os.getenv("EMBEDDING_API_BASE", "https://api.siliconflow.cn/v1"),
    chunk_size=32,  # SiliconFlow limits batch_size to 32
)

# Key: Custom Chinese sentence-splitting regex, or SemanticChunker defaults to English punctuation only
splitter = SemanticChunker(
    embeddings=embeddings,
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=85,
    sentence_split_regex=r"(?<=[。！？.?!])\s+",
    buffer_size=0,  # Avoid exceeding 512-token limit when concatenating sentences
)
chunks = splitter.split_documents(documents)

Pitfalls We Hit

Implementing semantic chunking, we ran into three issues:

Pitfall 1: Batch size exceeded

ValueError: input batch size 1000 > maximum allowed batch size 32

→ Fix: OpenAIEmbeddings(chunk_size=32)

Pitfall 2: Single-input token limit exceeded

Error code: 413 - input must have less than 512 tokens

→ Fix: Set buffer_size=0 to prevent SemanticChunker from concatenating neighboring sentences

Pitfall 3: Empty strings cause 400 errors

Error code: 400 - The parameter is invalid

→ Fix: Subclass SemanticChunker and override _get_single_sentences_list to filter empty strings

class FilteredSemanticChunker(SemanticChunker):
    def _get_single_sentences_list(self, text: str) -> List[str]:
        sentences = re.split(self.sentence_split_regex, text)
        return [s for s in sentences if s.strip()]

Results

Metric	Value
Chunk count	9 (fewest)
Average length	590.9 chars
Max length	2047 chars
Min length	17 chars

Key Finding:

Semantic chunking produces the fewest chunks (9), but with extreme size variation — smallest 17 chars, largest 2047 chars. This confirms it's truly grouping by semantic boundaries: semantically similar sentences are merged into large chunks, while topic transitions become tiny chunks.

For example, the entire "Service Communication" chapter (REST vs gRPC vs message queues) was aggregated into one 1,189-character chunk — because it all discusses the same topic. Transition sentences between chapters became tiny fragments (like a 28-character decision tree snippet).

Strategy 4: Document Structure Chunking (Markdown Header)

The Principle

The first three strategies are like "blind folding" — they don't know the document structure and purely use text features. Document structure chunking, in contrast, "keeps its eyes open": it recognizes Markdown #, ##, ### headings and splits strictly by heading hierarchy.

Each chunk's boundary is a heading boundary: starts at one heading, ends before the next heading at the same or higher level.

The Code

from langchain_text_splitters import MarkdownHeaderTextSplitter

splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=[
        ("#", "Header 1"),
        ("##", "Header 2"),
        ("###", "Header 3"),
    ],
    strip_headers=False,  # Keep headings inside chunk content
)
chunks = splitter.split_text(text)

Results

Metric	Value
Chunk count	20 (most)
Average length	266.5 chars
Max length	402 chars
Min length	71 chars

Key Finding:

Document structure chunking produces the most chunks (20), but each one carries an "ID card" — metadata records which heading level it belongs to:

chunk.metadata = {
    "Header 1": "Microservices Architecture Design Guide",
    "Header 2": "1. Service Decomposition Strategy",
    "Header 3": "1.1 Split by Business Boundary (DDD)"
}

This means during retrieval, you get not just the content but also its chapter origin. This is extremely valuable for citation tracing ("The answer comes from Chapter X of the document").

Side-by-Side Comparison

Statistics Summary

Strategy	Chunks	Avg Length	Median	Max	Min
Fixed Size	12	453.5	476.5	506	128
Recursive Character	13	431.5	457.0	507	88
Semantic	9	590.9	422.0	2047	17
Document Structure	20	266.5	259.0	402	71

Retrieval Difference for the Same Query

Suppose the user asks: "What are the anti-patterns of microservice decomposition?"

Strategy	Retrieved Chunk	Issue
Fixed Size	Chunk 4 (contains partial anti-pattern content, but starts mid-sentence)	List item starts in the middle; LLM lacks full context
Recursive Character	Chunk 5 (fully contains "1.3 Common Anti-patterns" section)	Good, but may truncate if the section is long
Semantic	Chunk 3 (aggregates anti-patterns + some following content)	May include irrelevant content
Document Structure	Chunk 6 (exactly matches "### 1.3 Common Anti-patterns")	Best — precise structural match

Strategy Selection Decision Matrix

Scenario	Recommended Strategy	Reasoning
General technical docs (PDF/Word)	Recursive Character	Most reliable baseline, no special formatting required
Markdown / Papers / Books	Document Structure	Preserves chapter structure, retrievable with provenance
Terminology-dense docs (legal/medical)	Semantic Chunking	Semantically coherent chunks, reduces cross-topic noise
Ultra-high-speed chunking (real-time)	Fixed Size	Zero computation overhead, pure string operations
Code documentation	Recursive Character + custom separators	Split by function/class boundaries

Selection Advice

Step 1: Start with recursive character chunking as your baseline
    ↓
Step 2: If documents are Markdown/HTML, try document structure chunking
    ↓
Step 3: If retrieval quality is unsatisfactory, upgrade to semantic chunking
         (highest cost but best quality)

Summary

This article used the same document and four strategies to show you how "how you cut" affects RAG quality:

Fixed Size: Simple but brutal. Good for rapid prototyping.
Recursive Character: The most universal baseline. Sufficient for 80% of scenarios.
Semantic Chunking: Best quality but highest cost. Use when precision is critical.
Document Structure: Best choice for structured documents. Retrieved chunks carry built-in context.

Key Takeaway: There is no perfect chunking strategy — only the strategy that fits your document type and business scenario. In real projects, use the comparison script from this article, run it on your own documents, and let the data guide your decision.

RAG Series (3): Tuning These 4 Parameters to Go From 'It Works' to 'It Works Well'

WonderLab — Sat, 02 May 2026 02:49:49 +0000

Why Does Your RAG Give Wrong Answers When Someone Else's Doesn't?

In the first two articles, we built a RAG pipeline that runs. But many people find that while the code works, answer quality is inconsistent — sometimes spot-on, sometimes missing information that's clearly in the document, sometimes drifting off-topic even when the right chunks were retrieved.

The problem is usually not the code. It's the parameters.

RAG has four core parameters, like four knobs on a radio:

Chunk Size: How long is each text chunk?
Chunk Overlap: How much do adjacent chunks overlap?
Top-K: How many chunks does the retriever return?
Embedding Model: How is text converted into vectors?

The combination of these four parameters directly determines whether the system can find relevant information and whether that information is enough to answer the question. In this article, we'll use a controlled-variable experiment so you can see the effect of different parameters with your own eyes.

Parameter 1: Chunk Size — How Long Is Each Chunk?

What Is Chunk Size?

Imagine you're organizing a 500-page technical manual. Chunk Size is how many pages you read at a time — 1 page, 5 pages, or 50 pages?

In RAG, Chunk Size is the maximum number of characters (or tokens) in each text chunk. The document is cut into many chunks, each no longer than this limit.

Why Does It Matter?

Chunk Size directly impacts two metrics:

Chunk Size	Retrieval Precision	Context Completeness	Analogy
Small (128)	High	Poor	Like reading dictionary entries — precise but isolated
Medium (512)	Balanced	Balanced	Like reading a paragraph — enough context without bloat
Large (2048)	Low	Good	Like reading an entire chapter — complete but noisy

What's wrong with too small? Suppose the document says: "The system uses Redis for caching with a default TTL of 3600 seconds. If this timeout is exceeded, data is automatically purged." If Chunk Size=128, this sentence might be split into two chunks: "The system uses Redis for caching with a default TTL of 3600 seconds." and "If this timeout is exceeded, data is automatically purged." When the user asks "What happens when Redis cache expires?", the retriever might only return the first chunk. The LLM sees "3600 seconds" but doesn't know about "automatically purged" — the answer is incomplete.

What's wrong with too large? Suppose Chunk Size=2048, and one chunk contains five unrelated topics. When the user asks a specific question, this chunk gets retrieved, but the LLM's attention is scattered by irrelevant content — like trying to hear one person speak in a noisy marketplace.

How to Choose?

There's no silver bullet, but there's a rule of thumb:

Chunk Size ≈ 1.5 ~ 2 × the expected answer length

Document Type	Recommended Chunk Size	Reasoning
FAQ / Q&A pairs	256 ~ 384	Short answers, precision matters more
Technical docs / API manuals	512 ~ 768	Medium-length answers, need some context
Papers / book chapters	1024 ~ 1536	Argument-heavy, need large context
Legal contracts / medical records	768 ~ 1024	Dense terminology, need inference from context

Heuristic: Start with 512, then observe retrieval results. If you notice "answers are cut off", increase it. If you notice "retrieved chunks contain too much irrelevant content", decrease it.

Parameter 2: Chunk Overlap — How Much Should Adjacent Chunks Overlap?

What Is Chunk Overlap?

Back to that technical manual. If you read 5 pages at a time, Overlap is how many pages from the previous chunk you keep when starting the next one. For example, Overlap=1 means: first read pages 1-5, then read pages 5-9 (page 5 appears twice).

Why Is Overlap Needed?

Without overlap, critical information can get "cut at the seam":

Chunk A: "The system uses Redis for caching with a default TTL of 3600 seconds."
Chunk B: "If this timeout is exceeded, data is automatically purged."

If the user asks "What happens when Redis cache expires?", the embedding model might think Chunk B is more relevant (both mention "expires"), and only return Chunk B. But Chunk B starts with "If this timeout is exceeded" — without Chunk A, the LLM doesn't know what "this timeout" refers to.

With Overlap=50, Chunk B starts with the last 50 characters of Chunk A:

Chunk B (with overlap): "...default TTL of 3600 seconds. If this timeout is exceeded, data is automatically purged."

Now even if only Chunk B is retrieved, the LLM can infer "this timeout = 3600 seconds".

How Much Overlap?

Generally set to 10% ~ 20% of Chunk Size:

Chunk Size	Recommended Overlap	Notes
256	25 ~ 50	Short text, small overlap preserves context
512	50 ~ 100	The sweet spot for general use
1024	100 ~ 200	Long text needs more overlap to preserve continuity

Note: More overlap is not always better. Too much overlap leads to storing massive amounts of duplicate content in the vector database, increasing storage cost and deduplication burden during retrieval.

Parameter 3: Top-K — How Many Chunks to Retrieve?

What Is Top-K?

Top-K is the number of text chunks the retriever returns each time. K=4 means "give me the 4 most relevant chunks", K=10 means "give me the 10 most relevant chunks".

Why Does It Matter?

K too small = missing information. K too large = introducing noise.

Scenario A: K=2, missing critical information

User asks: "How do I configure the database connection pool and log level?" This question involves two topics. If K=2, the retriever might only return two chunks about "database connection pool" and completely miss "log level" — the LLM can only answer half the question.

Scenario B: K=20, noise drowning out the answer

User asks: "What's the default timeout?" The document has a clear answer. But K=20 retrieves 20 chunks, 19 of which discuss unrelated topics. The LLM's context window is filled with irrelevant content, and it can't find that simple number.

How to Choose?

Top-K = Number of topics the answer is expected to cover × 2 ~ 3

Query Type	Recommended K	Reasoning
Single-point fact ("What's the default port?")	3 ~ 5	Focused answer, fewer is better
Multi-condition ("How do I configure A and B?")	5 ~ 8	Might involve multiple topics
Comprehensive summary ("Summarize Chapter 3")	8 ~ 12	Need to cover multiple points

Heuristic: Start with K=4. If you notice "the answer is missing a part", increase it. If you notice "the answer contains irrelevant content", decrease it.

Parameter 4: Embedding Model — Who Does the "Semantic Translation"?

Embedding Is RAG's "Translator"

What an embedding model does is simple: convert text into a sequence of numbers (a vector). Semantically similar texts have vectors that are close together; semantically dissimilar texts have vectors that are far apart.

The retriever relies on this — it converts the user's question into a vector, then finds the nearest vectors in the database.

How Big Is the Difference Between Models?

Very big. For the same question, different models can return completely different results.

Model	Strong Language	Dimensions	Positioning	Best For
text-embedding-3-small	English	1536	Cheap & fast	English docs, budget-sensitive
text-embedding-3-large	English	3072	High precision	English docs, precision-first
BAAI/bge-large-zh-v1.5	Chinese	1024	Best for Chinese	Chinese docs, China-first choice
BAAI/bge-m3	Multilingual	1024	Multilingual	Mixed Chinese-English, cross-lingual

A Real Comparison Experiment

We use the same Chinese technical document (Automotive SPICE PAM v4.0), the same question, and compare retrieval results between text-embedding-3-small and BAAI/bge-large-zh-v1.5:

Question: "What is process capability level 1?"

Model	1st Retrieved Result	2nd Retrieved Result	Assessment
text-embedding-3-small	Page 12: paragraph about project management	Page 89: paragraph about risk assessment	❌ Neither mentions "process capability level"
BAAI/bge-large-zh-v1.5	Page 45: Definition of process capability level 1	Page 46: Example practices for level 1	✅ Direct hit

Reason: OpenAI's models are primarily trained on English corpora. Their understanding of Chinese technical terminology is not as strong as BGE, which is specifically fine-tuned on Chinese corpora.

How to Choose an Embedding Model?

Decision tree:

What language is your document?
    ├─ Pure English → text-embedding-3-small (best value)
    │                  or text-embedding-3-large (best precision)
    ├─ Pure Chinese → BAAI/bge-large-zh-v1.5 (China-first choice)
    │                  or BAAI/bge-m3 (if mixed Chinese-English)
    └─ Mixed Chinese-English → BAAI/bge-m3 (best multilingual support)

Switching models is a one-line change: Just change model="..." in the build_embeddings() function. Everything else stays the same — that's the beauty of LangChain.

Hands-On: Controlled-Variable Experiment

Let's run an experiment: same document, same question, only changing Chunk Size, and observe how answer quality changes.

Experimental Design

"""
RAG Parameter Controlled-Variable Experiment
Fixed: document, question, embedding model, Top-K, LLM
Variable: Chunk Size
"""

import os
from pathlib import Path
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_chroma import Chroma
from langchain_community.document_loaders import PyPDFLoader
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI, OpenAIEmbeddings

# Load document
doc = PyPDFLoader("./data/Automotive-SPICE-PAM-v40.pdf").load()

# Embedding (fixed)
embeddings = OpenAIEmbeddings(
    model="BAAI/bge-large-zh-v1.5",
    api_key=os.getenv("EMBEDDING_API_KEY"),
    base_url="https://api.siliconflow.cn/v1",
    chunk_size=32,
)

# LLM (fixed)
llm = ChatOpenAI(
    model="glm-4-flash",
    api_key=os.getenv("LLM_API_KEY"),
    base_url="https://open.bigmodel.cn/api/paas/v4",
    temperature=0,
)

# Test different Chunk Sizes
def test_chunk_size(chunk_size, overlap):
    print(f"\n{'='*50}")
    print(f"Chunk Size={chunk_size}, Overlap={overlap}")
    print(f"{'='*50}")

    # Split
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=overlap,
        length_function=len,
    )
    chunks = splitter.split_documents(doc)
    print(f"Generated {len(chunks)} chunks")

    # Build vector store
    persist_dir = f"./chroma_db_{chunk_size}"
    if os.path.exists(persist_dir):
        import shutil
        shutil.rmtree(persist_dir)

    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory=persist_dir,
    )

    # Build RAG Chain (LCEL style)
    retriever = vector_store.as_retriever(search_kwargs={"k": 4})

    prompt = ChatPromptTemplate.from_messages([
        ("system", "Answer based on reference content. Reference:\n{context}"),
        ("human", "{question}")
    ])

    rag_chain = (
        {"context": retriever | (lambda docs: "\n\n".join(d.page_content for d in docs)),
         "question": RunnablePassthrough()}
        | prompt | llm | StrOutputParser()
    )

    # Ask question
    question = "What is process capability level 1?"
    answer = rag_chain.invoke(question)
    print(f"\nAnswer: {answer[:200]}...")

    # Print retrieved sources
    sources = retriever.invoke(question)
    print(f"\nRetrieved {len(sources)} sources:")
    for i, s in enumerate(sources[:3], 1):
        print(f"  [{i}] Page {s.metadata.get('page', '?')}: {s.page_content[:80]}...")

# Run three experiments
test_chunk_size(chunk_size=128, overlap=20)
test_chunk_size(chunk_size=512, overlap=50)
test_chunk_size(chunk_size=1024, overlap=100)

Expected Results

Chunk Size	Number of Chunks	Retrieval Quality	Typical Observation
128	Many (~4000)	High precision but broken context	Retrieved chunks have the keyword "process capability level" but lack sufficient context; LLM answers are fragmented
512	Medium (~1000)	Best balance	Retrieved chunks contain complete definitions + examples; LLM answers are coherent and accurate
1024	Few (~500)	Complete context but low precision	Retrieved chunks contain lots of irrelevant content (e.g., descriptions of other levels); LLM answers are verbose

Key insight: Chunk Size is not "bigger is better" nor "smaller is better". 512 characters is a safe starting point for most Chinese technical documents.

The 5 Most Common Pitfalls

Pitfall 1: Setting Chunk Size by Token Count, but length_function Uses Character Count

# ❌ Wrong: You think chunk_size=512 means 512 tokens
splitter = RecursiveCharacterTextSplitter(chunk_size=512)

# Actually the default length_function=len counts characters!
# 512 characters ≈ 256 tokens (Chinese), so chunks are half the size you expected

Fix: If you want to count by tokens, explicitly specify a tokenizer:

import tiktoken

def token_length(text):
    return len(tiktoken.encoding_for_model("gpt-4").encode(text))

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    length_function=token_length,  # ✅ Count by tokens
)

Pitfall 2: Overlap Too Large, Causing 30% Duplicate Content in the Vector Store

Overlap is not free. Every overlapping character requires an embedding computation and takes up storage space in the vector database. Overlap=100 with Chunk Size=200 means 50% of your storage is redundant.

Fix: Set Overlap to 10%~15% of Chunk Size, never exceed 20%.

Pitfall 3: Swapped Embedding Model Without Clearing the Old Vector Store

# ❌ Wrong: Yesterday you built the index with BGE, today you switch to OpenAI
# and reuse the same chroma_db/ directory
vector_store = Chroma.from_documents(documents=chunks, embedding=new_embeddings)
# Result: Query vectors and index vectors come from different models — completely mismatched

Fix: When switching embedding models, always delete the old vector store and re-index:

if os.path.exists(persist_directory):
    shutil.rmtree(persist_directory)  # ✅ Clear old data

Pitfall 4: Hardcoding Top-K Without Adjusting for Question Complexity

Using K=4 for all questions, but "What's the default port?" (simple fact) and "Summarize all key points from Chapter 3" (comprehensive overview) require vastly different amounts of information.

Fix: Simple questions use K=3~4, complex questions use K=8~10. A more advanced approach is to use an LLM to judge question complexity first, then dynamically decide K (covered in a later article).

Pitfall 5: Not Monitoring "Zero Retrieval"

Sometimes the retriever returns 0 relevant chunks (e.g., the user asks about something completely absent from the document), but you don't know. The LLM has no choice but to hallucinate from memory.

Fix: Add a threshold filter after retrieval — if the similarity score of the most relevant chunk is below a threshold (e.g., 0.6), directly tell the user "The document doesn't contain relevant information" instead of feeding irrelevant chunks to the LLM:

# Add a filter layer after retrieval
docs = retriever.invoke(question)
if not docs or max_similarity < 0.6:
    return "Sorry, I cannot answer this question based on the available documents."

Parameter Selection Cheat Sheet

Condense everything above into one table, tape it next to your monitor:

Parameter	Default for Beginners	When to Increase	When to Decrease
Chunk Size	512	Answer needs large context (books/papers)	Answer is short (FAQ/config items)
Chunk Overlap	50 (~10%)	Sentences often span pages/paragraphs	Document is highly structured with clear boundaries
Top-K	4	Question involves multiple topics	Question is very specific with a unique answer
Embedding	BGE (Chinese) / OpenAI (English)	Chinese professional documents	English general-purpose documents

Summary

In this article, we covered the four core parameters of RAG:

Chunk Size: Determines how long each text chunk is. Default 512. Use 256 for short answers, 1024 for long arguments.
Chunk Overlap: Determines how much adjacent chunks overlap. Default 10% of Chunk Size. Prevents cross-chunk information from being severed.
Top-K: Determines how many chunks to retrieve. Default 4. Increase to 8 for complex questions, decrease to 3 for simple facts.
Embedding Model: Chinese documents use BGE, English documents use OpenAI. Remember to clear the vector store when switching models.

Through the controlled-variable experiment, we demonstrated that parameters are not "bigger is better" nor "smaller is better" — the key is finding the balance that suits your document type and query patterns.

Next up, we enter Part 2: Core Components — a deep dive into 4 chunking strategies (Fixed Size, Recursive Character, Semantic Chunking, Document Structure), thoroughly unpacking the "how to cut" problem.

References

LangChain Text Splitters Documentation — Official chunking strategy guide
BGE Embedding Models GitHub — Best practices for Chinese embeddings
MTEB Leaderboard — Authoritative embedding model ranking
ChromaDB Distance Metrics — Cosine similarity vs Euclidean distance

RAG Series (1): Why LLMs Need External Memory

WonderLab — Sat, 02 May 2026 02:47:17 +0000

Two Root Causes Behind LLM "Hallucinations"

Anyone who has worked with large language models has run into these two situations:

Situation 1: Knowledge Cutoff

You: What were our company's Q1 sales figures?
GPT: I'm sorry, my training data only goes up to early 2024 and I have
     no access to your company's internal data.

Situation 2: Hallucination

You: How do I use LangChain's RunnablePassthrough?
GPT: RunnablePassthrough can be enabled by calling .with_config(pass_through=True)...
     (This parameter doesn't exist.)

Both problems share the same root cause: an LLM's knowledge is frozen at training time.

The moment training completes, the model's "memory" is locked in place. It has no idea what happened today, knows nothing about your internal documents, and won't go look things up—it can only answer from memory. When memory runs dry, it either admits ignorance or invents a plausible-sounding answer.

That's where hallucinations come from: the model uses fluent language to fill in the gaps in its knowledge.

Three Solutions and How They Compare

There are three engineering approaches to this problem:

Approach	Mechanism	Best For	Cost
Fine-tuning	Retrain on new data, "bake" knowledge into parameters	Fixed-domain language style, output format	Expensive, slow to update, limited factual recall
Long Context	Stuff all documents into the prompt	Small document sets, one-off queries	Token cost grows exponentially; quality degrades at extreme lengths
RAG	Dynamically retrieve relevant content at query time, inject into prompt	Large knowledge bases, continuously updated data	Requires retrieval infrastructure

A common misconception: fine-tuning is not good at injecting new facts.

Fine-tuning changes a model's behavioral patterns and language style—it doesn't "store a book" inside the parameters. Experiments consistently show that fine-tuning on specific Q&A pairs produces limited accuracy gains on related questions, and if training data contains errors, the model confidently repeats those errors.

RAG's core advantage is separating "what to know" from "how to say it":

Knowledge lives in an external database and can be updated anytime
The model focuses purely on understanding and generation, not memorization

When should you use long context instead of RAG?
When the total document volume is under ~100K tokens, the query is one-off (not recurring), and API costs are acceptable, long context is often simpler. Claude and Gemini's extended context windows make "stuffing a whole book in" genuinely viable. But for enterprise knowledge bases—thousands of documents, continuous updates, multiple concurrent users—RAG remains the more sensible architecture.

What RAG Is: An Open-Book Exam Analogy

RAG = Retrieval-Augmented Generation.

The most intuitive way to think about it: turning a closed-book exam into an open-book exam.

Closed-book (pure LLM): The student answers purely from memory. Anything not memorized gets guessed.

Open-book (RAG): The student can consult reference materials, but still needs to understand the question, find the relevant content, and compose the answer. The reference materials are the external knowledge base; looking things up is the retrieval step.

This analogy reveals two key properties of RAG:

Knowledge lives outside the model — it can be swapped and updated independently
The model handles understanding and generation — after retrieval, the model still needs to "read" the content and produce a coherent response

The Complete RAG Pipeline

RAG operates in two distinct phases: the indexing phase (one-time, offline) and the query phase (real-time, per request).

The two-phase RAG architecture — top: offline indexing pipeline; bottom: real-time query pipeline; both share the same Vector DB

Indexing Phase

This phase completes before any user query arrives. It's a one-time preprocessing step.

Raw Documents → Document Loading → Text Splitting → Embedding → Vector Database

Step 1: Document Loading

Convert raw content from various formats into plain text. PDFs, Word docs, Markdown, web pages, code—each format has its own parsing challenges (tables and images in PDFs are notoriously tricky).

Step 2: Text Splitting (Chunking)

Cut long documents into smaller chunks. This step has a significant impact on final quality—chunks too large reduce retrieval precision; chunks too small lose semantic coherence. (Chunking strategies are covered in depth in a later article; for now, just understand why we split.)

Step 3: Embedding

Use an embedding model to convert each text chunk into a high-dimensional vector. This vector captures the semantic meaning of the text—semantically similar texts produce vectors that are close together in the vector space.

Step 4: Store in Vector Database

Store all vectors along with their original text in a vector database that supports similarity search (Chroma, Qdrant, Weaviate, etc.).

Query Phase

Executed in real time for every user request.

User Question → Embedding → Similarity Search → Retrieved Chunks → Prompt Assembly → LLM → Answer

Step 1: Query Embedding

Convert the user's question into a vector using the same embedding model.

Step 2: Similarity Search

Find the Top-K most similar text chunks in the vector database. Similarity is measured by distance in vector space (cosine similarity, etc.).

Step 3: Prompt Assembly

Combine the retrieved text chunks with the user's question into a complete prompt and send it to the LLM. A typical format:

You are a knowledge assistant. Answer the user's question based solely on
the reference content provided below.

Reference content:
[Retrieved chunk 1]
[Retrieved chunk 2]
...

User question: [original question]

Please base your answer on the reference content. If the reference content
does not contain relevant information, say so clearly.

Step 4: LLM Generation

The LLM generates an answer grounded in the provided context, rather than from its internal parameters alone.

Hands-On: A Minimal RAG in 100 Lines

No frameworks—just the OpenAI API. Let's implement a working RAG from scratch. The goal is to see exactly what each step does, without the abstraction layers of a framework hiding the details.

"""
Minimal RAG implementation — no frameworks, OpenAI API only.
Demonstrates the complete RAG pipeline: indexing + querying.
"""

import numpy as np
from openai import OpenAI

client = OpenAI()  # requires OPENAI_API_KEY environment variable

# ─────────────────────────────────────────
# Simulated knowledge base: 5 technical docs
# ─────────────────────────────────────────
DOCUMENTS = [
    "LangChain is a framework for building LLM applications, providing chaining, memory management, and tool integration.",
    "Vector databases enable semantic search by converting text into high-dimensional vectors. Common options include Chroma, Qdrant, Weaviate, and Pinecone.",
    "RAG (Retrieval-Augmented Generation) reduces LLM hallucinations by retrieving relevant documents before generation, improving answer accuracy.",
    "Embedding models convert text into fixed-dimension vectors, where semantically similar texts are positioned closer together in vector space.",
    "Fine-tuning retrains a model on specific data to adjust its behavior — it's best suited for changing output style, not injecting new factual knowledge.",
]


# ─────────────────────────────────────────
# Indexing phase: convert documents to vectors
# ─────────────────────────────────────────
def build_index(documents: list[str]) -> list[dict]:
    """
    Convert each document into a vector.
    Returns [{text, embedding}, ...]
    In production, store these in a vector database.
    """
    print(f"Indexing {len(documents)} documents...")

    index = []
    for doc in documents:
        response = client.embeddings.create(
            model="text-embedding-3-small",
            input=doc
        )
        embedding = response.data[0].embedding
        index.append({"text": doc, "embedding": embedding})

    print("Index built successfully.")
    return index


def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
    """Cosine similarity between two vectors. Range: -1 to 1, higher = more similar."""
    a = np.array(vec_a)
    b = np.array(vec_b)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))


# ─────────────────────────────────────────
# Query phase: retrieve + generate
# ─────────────────────────────────────────
def retrieve(query: str, index: list[dict], top_k: int = 2) -> list[str]:
    """
    Embed the query and find the top_k most similar documents.
    """
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=query
    )
    query_embedding = response.data[0].embedding

    scored = []
    for doc in index:
        score = cosine_similarity(query_embedding, doc["embedding"])
        scored.append((score, doc["text"]))

    scored.sort(key=lambda x: x[0], reverse=True)
    return [text for _, text in scored[:top_k]]


def generate(query: str, context_docs: list[str]) -> str:
    """
    Assemble retrieved docs + user question into a prompt and call the LLM.
    """
    context = "\n".join([f"- {doc}" for doc in context_docs])

    prompt = f"""You are a knowledge assistant. Answer the user's question based
solely on the reference content below. If the reference content does not contain
relevant information, say so clearly — do not make anything up.

Reference content:
{context}

User question: {query}

Answer:"""

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content


def rag_query(query: str, index: list[dict]) -> str:
    """Full RAG query pipeline."""
    print(f"\nQuestion: {query}")

    docs = retrieve(query, index, top_k=2)
    print(f"Retrieved {len(docs)} relevant documents:")
    for i, doc in enumerate(docs, 1):
        print(f"  [{i}] {doc[:60]}...")

    answer = generate(query, docs)
    print(f"\nAnswer: {answer}")
    return answer


# ─────────────────────────────────────────
# Run the demo
# ─────────────────────────────────────────
if __name__ == "__main__":
    # Build the index (only needs to be done once in a real system)
    index = build_index(DOCUMENTS)

    # Test with a few questions
    rag_query("What is a vector database?", index)
    rag_query("What's the difference between RAG and fine-tuning?", index)
    rag_query("What is Python's GIL?", index)  # Not in the knowledge base — testing refusal

Sample output:

Indexing 5 documents...
Index built successfully.

Question: What is a vector database?
Retrieved 2 relevant documents:
  [1] Vector databases enable semantic search by converting text into high-dim...
  [2] Embedding models convert text into fixed-dimension vectors, where semant...

Answer: A vector database is a database system that enables semantic search by
converting text into high-dimensional vectors. Common examples include Chroma,
Qdrant, Weaviate, and Pinecone...

Question: What is Python's GIL?
Retrieved 2 relevant documents:
  [1] LangChain is a framework for building LLM applications...
  [2] Fine-tuning retrains a model on specific data...

Answer: Based on the provided reference content, I cannot answer your question
about Python's GIL — the reference material does not contain relevant information.

Notice the last question: the knowledge base has nothing about Python's GIL, and the LLM explicitly says it can't answer rather than inventing a response. This is how RAG controls hallucinations: a constraint in the prompt instructs the model to answer only from retrieved content.

Limitations of This Implementation

The 100 lines above demonstrate the complete RAG pipeline, but there are obvious shortcomings:

Problem	Cause	Engineering Solution
Vectors live in memory, lost on restart	No persistence	Vector database (Chroma / Qdrant)
Long documents passed in directly will exceed token limits	No chunking	Text Splitter strategies (next article)
Poor keyword matching; pure vector retrieval only	No hybrid search	Hybrid search (later in series)
No quality measurement	No evaluation	RAGAS evaluation framework (later in series)

Each limitation maps directly to a topic covered in the upcoming articles.

Summary

This article addressed three core questions:

Why RAG? — LLM knowledge cutoff and hallucinations both stem from knowledge being frozen in model parameters
What is RAG? — Dynamically retrieve external knowledge at query time, inject it into the prompt, and let the LLM answer based on evidence
RAG vs. alternatives — Fine-tuning changes behavior; long context works for small documents; RAG is built for large-scale, continuously-updated knowledge bases

Next up: the first deep dive into RAG's core components — text chunking strategies. Why does the chunking approach have such a dramatic impact on quality, and how do you choose between the four main strategies?

References

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Original RAG paper (Lewis et al., 2020)
OpenAI Embeddings Documentation
LangChain RAG Tutorial

One Open Source Project a Day (No. 54): Warp - The AI-Native Rust Terminal

WonderLab — Sat, 02 May 2026 02:42:27 +0000

Introduction

"The terminal hasn't fundamentally changed in 40 years. It's time it did." — The Warp Team

This is the No.54 article in the "One Open Source Project a Day" series. Today, we are exploring Warp.

The terminal is perhaps the most ancient part of a developer's daily toolkit. While every other tool has evolved for the AI era, most of us still interact with our command line as a simple "stream of text" based on 40-year-old logic. Warp is here to change that. Rebuilt from the ground up in Rust, Warp is not just a high-performance terminal emulator, but a collaborative, AI-native Agentic Development Environment (ADE).

What You Will Learn

How Warp transforms terminal output into actionable "Blocks."
How AI integration in the terminal has evolved from simple completion to "Agentic Mode."
Warp Drive: Managing and sharing team workflows like code repositories.
The tech behind its high-performance GPU-accelerated UI rendering in Rust.

Prerequisites

Basic command-line experience.
Familiarity with modern development tools like VS Code.

Project Background

Project Introduction

Warp is a modern, high-performance terminal emulator. It breaks away from the linear text-stream model of traditional terminals, reimagining the interface as a series of distinct "Blocks." Recently, Warp hit a major milestone by open-sourcing its client codebase and fully embracing "Agentic Development," allowing AI agents to debug, refactor, and deploy directly from the command line.

Author/Team Introduction

Team: The Warp team, based in the USA.
Background: Backed by top-tier firms like Sequoia Capital and GV, with investors including Sam Altman (CEO of OpenAI) and Dylan Field (CEO of Figma).
Founded: 2020 (Client codebase open-sourced in 2024).

Project Data

⭐ GitHub Stars: 23,000+
🍴 Forks: 1,200+
📦 Core Language: Rust (98.2%)
📄 License: AGPL v3 (Client) / MIT (UI Framework)
🌐 Official Website: warp.dev

Main Features

Core Utility

Warp upgrades the boring command-line interaction into an IDE-like collaborative experience. It lowers the barrier to entry for complex CLI tools using AI and eliminates organizational "knowledge silos" through cloud-based sharing features.

Use Cases

AI-Assisted Debugging
- When a command fails, use the built-in AI to explain the error and generate a fix with one click.
Workflow Sharing (Warp Drive)
- Save complex deployment scripts or operational commands as parameterized "Workflows" and share them with your team.
Immersive Command Editing
- Edit commands like you edit text: support for mouse positioning, undo, and standard editor keybindings.

Quick Start

Warp is available for macOS, Linux, and Windows (v1).

# macOS users can install via Homebrew
brew install --cask warp

# Linux users can download .deb, .rpm, or AppImage from the website

Once installed, use CMD+P to search, or start a command with # to use natural language to command conversion.

Core Characteristics

Block-based UI
- Every command and its output is encapsulated in a "Block." You can copy output, share a link to a specific block, or filter output with AI.
Warp AI
- Deeply integrated with models like Claude 3.5 Sonnet and GPT-4o. It acts as a "tech lead" capable of managing sub-agents for complex tasks.
Warp Drive
- A cloud-based "safe" for storing, searching, and sharing common workflows and interactive notebooks.
Exceptional Performance
- Fully GPU-accelerated UI rendering built in Rust, ensuring low latency even with high-volume log outputs.
Modern Editor Experience
- The input bar features syntax highlighting, smart completions, and multi-cursor editing.

Project Advantages

Feature	Warp	iTerm2 / Alacritty	VS Code Terminal
AI Integration	Native & Deep	Plugin-based / Minimal	Basic Completion
Collaboration	Cloud Sync & Sharing	None	Very Limited
Interaction Model	Block-level Objects	Continuous Text Stream	Text Stream
Rendering	GPU Accelerated (Rust)	Mixed	Generally Lower

Why Choose Warp?

Eliminate Context Switching: No need to leave the terminal to search for regex syntax or error details.
Knowledge Retention: Turn scattered team knowledge into reusable digital assets via Warp Drive.
Zero Configuration: Comes with most modern developer features out of the box.

Detailed Analysis

1. From Terminal to ADE

Warp is positioning itself as an Agentic Development Environment (ADE). Its architecture leverages the Model Context Protocol (MCP), allowing it to connect terminal sessions to external data (GitHub, Jira, DBs) and AI agents seamlessly.

2. GPU-Accelerated Rendering

Warp built a custom Rust UI framework called warpui.

Rendering: Fully rendered on the GPU using modern graphics APIs.
Benefit: Extremely low CPU overhead and zero "flicker" or lag when handling heavy log processing.

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/warpdotdev/warp
📚 Documentation: https://docs.warp.dev/
📄 Open Source Roadmap: Why Warp is Going Open Source

Community

Discord: Active community for developers.
Warp Drive: Explore how to codify team workflows.

Target Audience

Professional developers seeking ultimate efficiency.
DevOps engineers who need to share operational assets.
Terminal veterans tired of the 80s-style command line experience.

Find more useful knowledge and interesting products on my Homepage

One Open Source Project a Day (No. 53): pi-mono - Minimalist & High-Performance AI Coding Agent

WonderLab — Sat, 02 May 2026 02:39:38 +0000

Introduction

"Simplicity is the ultimate sophistication." — Leonardo da Vinci

This is the No.53 article in the "One Open Source Project a Day" series. Today, we are looking at pi-mono (pi).

In an era where AI coding tools are becoming increasingly bloated—with massive binaries and complex sub-agent architectures—Mario Zechner, the author of libGDX, has taken a diametrically opposite path. pi-mono is a TypeScript-based monorepo containing a lean yet powerful CLI coding assistant named pi. It eschews flashy GUIs for a custom "differential rendering" TUI framework, delivering the smoothest AI collaboration experience directly in your terminal.

What You Will Learn

The minimalist design philosophy of pi-mono.
How "differential rendering" achieves a flicker-free terminal UI.
High-efficiency cross-provider LLM switching.
Why the "YOLO mode" (no permission confirmation) dramatically boosts productivity.
A deep comparison with heavy-duty agents like Claude Code.

Prerequisites

Basic Node.js/TypeScript environment setup.
Basic understanding of LLM Tool Calling.
API Keys for AI providers like Anthropic or OpenAI.

Project Background

Project Introduction

pi-mono is a suite of AI programming agent tools designed specifically for "power users." It consists of a core agent engine, a unified AI interface layer, and a terminal UI with its own autonomous rendering engine. Its core objective is to provide the fastest response times and the cleanest user experience without sacrificing context control.

Author/Team Introduction

Author: Mario Zechner
Background: Founder of the popular open-source game framework libGDX and former founder of RoboVM. He has deep expertise in high-performance cross-platform development and the open-source community.
Project Status: Under rapid iteration, currently performing exceptionally well on benchmarks like Terminal-Bench.

Project Data

⭐ GitHub Stars: 430+ (Early stage, growing fast)
🍴 Forks: 30+
📦 Package Manager: pnpm
📄 License: MIT
🌐 Repository: badlogic/pi-mono

Main Features

Core Utility

As a "harness," pi-mono connects LLMs (like Claude 3.5 Sonnet) to your local development environment. It can autonomously read files, execute Bash commands, rewrite code, and provide feedback on results.

Use Cases

Fast Refactoring
- It understands the full context of a codebase and can perform cross-file interface renaming with a single prompt.
Hard Bug Fixing
- By observing error logs, it autonomously executes search commands and applies targeted fixes.
Minimalist Development
- For developers who prefer working in Terminal/Vim, pi provides IDE-like interactions while remaining lightweight.

Quick Start

# Install the pi coding agent
npm install -g @mariozechner/pi-coding-agent

# Set up API Key (e.g., Anthropic)
export ANTHROPIC_API_KEY=your_key_here

# Start in the project root
pi

Core Characteristics

Differential Rendering TUI
- Frustrated by terminal flickering, the author developed a rendering engine inspired by React's diffing algorithm, making Markdown parsing and syntax highlighting extremely smooth.
Minimalist System Prompt
- Unlike other agents using thousands of tokens for instructions, pi uses less than 1000 tokens, saving context window and increasing response speed.
Seamless Multi-Model Switching
- You can switch models mid-conversation (e.g., from Claude to GPT-4o), and it automatically migrates the conversation history.
YOLO Mode
- The project embraces a "trust through execution" philosophy. It doesn't nag you with permission popups for running ls or read commands.

Project Advantages

Feature	pi-mono (pi)	Claude Code	Cursor / Windsurf
Size	Tiny (Node-based)	Large (Multi-layer deps)	Heavy (IDE level)
Extensibility	High (Bash-based)	Medium (MCP-constrained)	Low (Closed-box)
Start Speed	Instant	Slower	Slow
Control	100% Transparent	Limited	Lower

Why Choose This Project?

Performance Beast: Extreme optimization of terminal rendering and network I/O makes it feel 2x faster than similar tools.
Transparency: You can clearly see every character the AI generates and every simple tool call it executes.
Developer-Friendly API: If you want to build your own agent, its pi-ai package is one of the best-wrapped cross-platform AI libraries available.

Detailed Analysis

1. Differential Rendering Engine (pi-tui)

This is the most technically impressive part of pi-mono. Standard Terminal UIs usually perform full redraws, causing noticeable flickering during long text outputs. pi-tui borrows from Virtual DOM concepts:

It maintains a buffer of the terminal state.
It calculates the difference (Diff) between old and new states.
It only sends the necessary escape sequences to Stdout.

2. Tool Calling Model (The "Bash-only" Philosophy)

While other agents try to integrate dozens of APIs, pi insists: If an AI can use Bash well, it can do anything.
Its toolbox consists of only four atomic tools:

read(path, startLine, endLine): Read file segments.
write(path, content): Overwrite files.
edit(path, oldStr, newStr): Local search and replace (the most stable way to edit code).
bash(command): Execute any shell command.

This design ensures pi remains incredibly robust in almost any environment.

Project Links & Resources

Official Resources

🌟 GitHub: https://github.com/badlogic/pi-mono
📦 NPM: @mariozechner/pi-coding-agent
💬 Discord: Visit GitHub for the invite link.

Related Resources

Terminal-Bench Leaderboard - pi is consistently a top performer.

Target Audience

Terminal natives looking for a high-speed experience.
Developers with high requirements for code privacy and agent transparency.
Learners who want to understand how to build high-performance Agent TUIs from scratch.

Find more useful knowledge and interesting products on my Homepage

RAG Series (2): Building Your First RAG Pipeline with LangChain

WonderLab — Fri, 01 May 2026 02:08:30 +0000

From 100 Lines to a Production Pipeline

In the last article, we built a minimal RAG system in 100 lines of pure Python. It worked. It demonstrated the core idea. But if you tried to productionize that code, you'd quickly run into a wall.

Loading a PDF? You need PyPDF2 or pdfplumber, and you'll discover that tables, headers, and footers are a nightmare to parse cleanly.

Splitting text? Your naive text.split("\n\n") will cut sentences in half, break code blocks, or create chunks so large they blow past the token limit.

Swapping the vector database? Say goodbye to your afternoon — every database has a different API, different distance metrics, different metadata handling.

Switching LLM providers? OpenAI's client, Anthropic's client, local models via llama.cpp — each has its own message format, its own token counting, its own error handling.

This is exactly why LangChain exists. It doesn't do anything magical. It doesn't replace the underlying models or databases. What it does is simple and valuable: it gives you a uniform interface for plugging components together, so you can focus on the logic of your RAG system instead of the plumbing.

In this article, we'll rebuild our RAG pipeline using LangChain's modern API. By the end, you'll have a complete, runnable project that loads PDFs, splits them intelligently, stores them in ChromaDB, and answers questions using a multi-provider LLM — with about 30 lines of actual pipeline code.

LangChain Version Note: The code in this article is based on langchain 1.x (current stable). LangChain 1.x performed a breaking reorganization of 0.3.x — high-level APIs like create_retrieval_chain were removed. We use LCEL native syntax (| pipe operator) instead, which is functionally equivalent and version-agnostic. Full source code: https://github.com/chendongqi/llm-in-action/tree/main/02-langchain-basic

The Six Moving Parts of a RAG Pipeline

LangChain decomposes RAG into six components. Understanding what each one does — and where the quality risks hide — is the key to debugging RAG systems later.

Component	Role	The Quality Risk
Document Loader	Reads raw files (PDF, Word, Markdown, HTML) and extracts text	Tables, images, and weird formatting get mangled
Text Splitter	Cuts long documents into semantically coherent chunks	Chunks too large = low precision; too small = lost context
Embedding Model	Converts text chunks into high-dimensional vectors	Poor model choice = semantically unrelated texts cluster together
Vector Store	Persists vectors and enables fast similarity search	Wrong distance metric or no metadata filtering = bad retrieval
Retriever	Accepts a query, searches the vector store, returns relevant chunks	Top-K too low = missing information; too high = noisy context
Chain	Orchestrates the full flow: query → retrieve → prompt → LLM → answer	Prompt design and context assembly determine answer quality

Think of these six components as an assembly line in a factory. The Document Loader is the raw material intake. The Text Splitter is the precision cutting station. The Embedding Model and Vector Store form the warehouse and inventory system. The Retriever is the picker who fetches the right parts. The Chain is the foreman who coordinates everything and delivers the final product.

If any station on the line is misconfigured, the final product suffers — and the tricky part is that the failure often looks like an LLM problem when it's actually a retrieval problem.

Hands-On: A Complete LangChain RAG Project

Let's build it. We'll create a RAG system that reads a directory of PDF files, indexes them, and lets you ask questions in natural language.

Project Structure

rag-project/
├── requirements.txt
├── data/
│   └── sample.pdf          # Your PDF documents go here
└── rag_pipeline.py

Step 0: Dependencies

langchain>=0.3.0
langchain-text-splitters>=0.3.0
langchain-openai>=0.2.0
langchain-chroma>=0.1.0
langchain-community>=0.3.0
pypdf>=4.0.0
python-dotenv>=1.0.0

Full source code (ready to run): https://github.com/chendongqi/llm-in-action/tree/main/02-langchain-basic
Supports Zhipu AI / OpenAI / Ollama for LLM, SiliconFlow and local Ollama for Embedding.

Install them:

pip install -r requirements.txt

Configure your API keys (copy .env.example to .env):

cp .env.example .env
# Edit .env to fill in LLM_API_KEY and EMBEDDING_API_KEY

Supported Providers:

LLM: Zhipu AI (default), OpenAI, SiliconFlow, Ollama, Azure
Embedding: SiliconFlow (default), OpenAI, Ollama

Step 1: Load Documents

The PyPDFLoader handles PDF parsing for us. It extracts text page by page and returns a list of Document objects, each containing the page content and metadata (page number, source file, etc.).

from langchain_community.document_loaders import PyPDFLoader
from pathlib import Path

def load_pdfs(data_dir: str = "./data"):
    """Load all PDF files from the data directory."""
    documents = []
    pdf_paths = list(Path(data_dir).glob("*.pdf"))

    for pdf_path in pdf_paths:
        loader = PyPDFLoader(str(pdf_path))
        pages = loader.load()
        documents.extend(pages)
        print(f"Loaded '{pdf_path.name}': {len(pages)} pages")

    print(f"Total documents loaded: {len(documents)}")
    return documents

Real-world note: PDFs are the wild west of document formats. If your PDFs contain scanned images, you'll need OCR (via pdfplumber with Tesseract or Azure Document Intelligence). If they contain tables, consider UnstructuredPDFLoader which preserves table structure better than raw text extraction.

Step 2: Split Documents into Chunks

Remember the chunking problem from Part 1? LangChain's RecursiveCharacterTextSplitter is the industry default for good reason. It tries to split on natural boundaries — paragraphs first, then newlines, then sentences, then words — so it avoids cutting mid-sentence whenever possible.

from langchain_text_splitters import RecursiveCharacterTextSplitter

def split_documents(documents, chunk_size=200, chunk_overlap=30):
    """
    Split documents into overlapping chunks.
    chunk_overlap ensures context continuity between adjacent chunks.
    """
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        length_function=len,
        separators=["\n\n", "\n", ". ", " ", ""]
    )

    chunks = splitter.split_documents(documents)
    print(f"Split into {len(chunks)} chunks (chunk_size={chunk_size}, overlap={chunk_overlap})")
    return chunks

Why chunk_overlap matters: If a key concept spans two chunks — say, "The API rate limit is 100 requests per minute. Exceeding this limit returns a 429 status code" — an overlap of 50 characters ensures the second chunk still contains "100 requests per minute" as context. Without overlap, the retriever might fetch only one chunk and miss the causal relationship.

Chunk size trade-offs:

Chunk Size	Precision	Context	Best For
256 tokens	High	Minimal	Fact lookup, Q&A over structured docs
512 tokens	Balanced	Moderate	General-purpose RAG (good default)
1024 tokens	Lower	Rich	Long-form summarization, narrative documents
2048+ tokens	Low	Very rich	Only when the LLM context window is large and queries are broad

For most use cases, 512 tokens with 50-token overlap is a safe starting point.

Step 3: Embed and Store in ChromaDB

Now we convert each chunk into a vector and store them. ChromaDB is our choice here because it's persistent (data survives restarts), supports metadata filtering, and requires zero setup — it runs locally as an embedded database.

from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
import os

persist_directory = "./chroma_db"

def build_vector_store(chunks):
    """Create embeddings and store in ChromaDB."""
    embeddings = OpenAIEmbeddings(
        model="BAAI/bge-large-zh-v1.5",  # SiliconFlow Chinese model
        api_key=os.getenv("EMBEDDING_API_KEY"),
        base_url="https://api.siliconflow.cn/v1",
        dimensions=1024,
        chunk_size=32  # SiliconFlow limit: max 32 per batch
    )

    if os.path.exists(persist_directory):
        import shutil
        shutil.rmtree(persist_directory)

    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory=persist_directory
    )
    print(f"Vector store built: {vector_store._collection.count()} vectors persisted")
    return vector_store

Embedding model note: We use BAAI/bge-large-zh-v1.5 on SiliconFlow (excellent for Chinese). Access via the OpenAI-compatible interface in langchain_openai. Switch to text-embedding-3-small for OpenAI. The chunk_size=32 parameter is critical — it's SiliconFlow's batch limit (max 32 per request), while most other providers default to 1000.

Step 4: Build the Retriever

The retriever is a thin wrapper around the vector store that handles the search logic. By default, it performs similarity search and returns the top-K most relevant chunks.

def get_retriever(vector_store, search_k=4):
    """Configure retriever with similarity search."""
    retriever = vector_store.as_retriever(
        search_type="similarity",
        search_kwargs={"k": search_k}
    )
    return retriever

Step 5: Build the RAG Chain

Here's where LangChain's modern API shines. Instead of the older RetrievalQA class, we use LCEL (LangChain Expression Language) to compose a chain that is explicit, debuggable, and easy to modify.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

def build_rag_chain(retriever):
    """
    Build the full RAG chain using LCEL (langchain 1.x compatible).
    retrieve → format_docs → assemble prompt → LLM → StrOutputParser
    """
    llm = ChatOpenAI(
        model="glm-4-flash",  # Zhipu AI, via SiliconFlow or direct
        api_key=os.getenv("LLM_API_KEY"),
        base_url="https://open.bigmodel.cn/api/paas/v4",
        temperature=0
    )

    # System prompt. {context} filled by format_docs, {question} by the user's input.
    system_prompt = (
        "You are a precise knowledge assistant. Answer the user's question "
        "based solely on the provided reference content below. "
        "If the reference content does not contain the answer, say so clearly "
        "— do not make anything up.\n\n"
        "Reference content:\n{context}"
    )

    prompt = ChatPromptTemplate.from_messages([
        ("system", system_prompt),
        ("human", "{question}")
    ])

    # Helper: convert Document list to a single string for {context}
    def format_docs(docs: list) -> str:
        return "\n\n".join(doc.page_content for doc in docs)

    # LCEL Chain: compose components with the pipe operator
    # 1. {"context": retriever | format_docs, "question": RunnablePassthrough()}
    #    → retriever fetches docs, format_docs converts them to a string
    # 2. | prompt → assembles into a full prompt
    # 3. | llm → generates the answer
    # 4. | StrOutputParser() → returns plain text (not an AIMessage object)
    rag_chain = (
        {
            "context": retriever | format_docs,
            "question": RunnablePassthrough()
        }
        | prompt
        | llm
        | StrOutputParser()
    )

    return rag_chain

What's happening here? We're using LCEL (LangChain Expression Language) native syntax — the pipe | operator — instead of the high-level create_retrieval_chain (which was removed in langchain 1.x). The key line is retriever | format_docs: the retriever outputs a list of Document objects, format_docs converts them to a string that fills the {context} placeholder. RunnablePassthrough() passes the user's raw question through to the {question} placeholder. Three lines of declarative code that are functionally equivalent to the 50 lines of imperative Python from Part 1.

Step 6: Query the Pipeline

def query(rag_chain, question: str, retriever):
    """Run a question through the RAG pipeline, print answer and sources."""
    print(f"\nQuestion: {question}")

    answer = rag_chain.invoke(question)  # LCEL chain returns plain text directly
    print(f"\nAnswer:\n{answer}")

    # Retrieve docs separately to show sources (rag_chain doesn't expose them)
    docs = retriever.invoke(question)
    print("\nRetrieved sources:")
    for i, doc in enumerate(docs, 1):
        source = doc.metadata.get("source", "unknown")
        page = doc.metadata.get("page", "?")
        preview = doc.page_content[:120].replace("\n", " ")
        print(f"  [{i}] {source} (page {page}): {preview}...")

    return answer

Putting It All Together

if __name__ == "__main__":
    # 1. Load
    docs = load_pdfs("./data")

    # 2. Split (smaller chunk_size for short PDF pages)
    chunks = split_documents(docs, chunk_size=200, chunk_overlap=30)

    # 3. Embed & Store
    vector_store = build_vector_store(chunks)

    # 4. Retrieve
    retriever = get_retriever(vector_store, search_k=4)

    # 5. Build Chain (LCEL)
    rag_chain = build_rag_chain(retriever)

    # 6. Interactive Q&A
    while True:
        user_input = input("\nYour question (quit to exit): ").strip()
        if user_input.lower() in ("quit", "exit", "q"):
            break
        if user_input:
            query(rag_chain, user_input, retriever)

Running the Pipeline

Place a PDF in data/sample.pdf and run:

python rag_pipeline.py

Sample output:

RAG Pipeline 启动
  LLM Provider    : zhipu
  LLM Model       : glm-4-flash
  Embedding       : openai / BAAI/bge-large-zh-v1.5
  数据目录        : ./data
  向量库          : ./chroma_db
==================================================
已加载 'Automotive-SPICE-PAM-v40.pdf'：153 页
共加载 153 个文档片段
切分为 2000 个块（chunk_size=200，overlap=30）
[Embedding] Provider: openai | Model: BAAI/bge-large-zh-v1.5 | Base: https://api.siliconflow.cn/v1
已清除旧向量库：./chroma_db
向量库构建完成：2000 个向量已持久化
==================================================
RAG Pipeline 构建完成！输入问题开始问答（输入 'quit' 退出）
==================================================

你的问题：What is Automotive SPICE?

==================================================
问题：What is Automotive SPICE？
==================================================

答案：
Automotive SPICE（Automotive Software Process Improvement and
Capability Determination）是一种用于评估和改进汽车软件
开发过程能力的框架。它定义了软件开发生命周期中的关键
过程域，并建立了过程能力的等级评估标准...

检索到的来源：
  [1] ./data/Automotive-SPICE-PAM-v40.pdf（第 5 页）：Automotive
      SPICE Process Assessment Model The Process Assessment Model
      (PAM) defines the processes...

Notice how the answer includes citations — we know exactly which pages the information came from. This traceability is critical for production RAG systems where users need to verify claims.

What Changed From Our 100-Line Version?

Let's compare the hand-written RAG from Part 1 with our LangChain pipeline:

Aspect	Hand-written (Part 1)	LangChain (Part 2)
PDF loading	Not supported	One-line `PyPDFLoader`
Text splitting	None (whole docs)	`RecursiveCharacterTextSplitter` with smart boundaries
Vector persistence	In-memory only, lost on restart	ChromaDB persists to disk
Embedding model swap	Rewrite API calls	One-line parameter change
LLM swap	Rewrite client code	One-line parameter change
Vector DB swap	Rewrite storage + search	Swap `Chroma` for `Qdrant` / `Pinecone`
Prompt engineering	Raw string formatting	Templated prompts with `ChatPromptTemplate`
Source citations	Manual tracking	Automatic metadata propagation
Chain construction	Manual `retrieve + generate`	LCEL `
Lines of pipeline code	~80	~25

The abstraction doesn't hide complexity — it isolates it.

LangChain Version Compatibility: The code in this article targets {% raw %}langchain 1.x (current stable). LangChain 1.x performed a breaking reorganization of 0.3.x — create_retrieval_chain and create_stuff_documents_chain were removed. We use LCEL native syntax (| pipe operator) instead, which is functionally equivalent and version-agnostic. When you need to debug retrieval quality, you know exactly which component to tune. When you need to swap the embedding model for a cheaper alternative, you change one line. When your data grows beyond ChromaDB's capabilities, you switch to Qdrant without touching the rest of the pipeline.

Common Pitfalls at Each Stage

Loader Pitfall: "My PDF has tables and they come out garbled"

Raw PDF text extraction flattens tables into a stream of numbers. For table-heavy documents, use UnstructuredPDFLoader or AzureAIDocumentIntelligenceLoader which preserves structural relationships.

Splitter Pitfall: "The answer is split across two chunks and the model only sees half"

Increase chunk_overlap to 100-150 tokens, or reduce chunk_size so that key concepts fit within a single chunk. Better yet, use Parent-Document Retrieval (covered in a later article) which retrieves small chunks but returns the full parent document for context.

Embedding Pitfall: "Questions and documents don't match even though they should"

This is the "asymmetric retrieval" problem. A user asks "How do I reset my password?" but the document says "To reset your password, navigate to Settings → Security." The question and the answer embed to different vectors because their surface text differs. Solutions: use a model fine-tuned for Q&A retrieval (like BGE-M3), or generate hypothetical answers for retrieval (HyDE — also covered later).

Retriever Pitfall: "Top-K=4 isn't enough for complex questions"

If a question requires synthesizing information from five different sections of a document, k=4 will miss one. But increasing k blindly adds noise. A better approach: use Multi-Query Retrieval (generate 3 variants of the question, retrieve for each, deduplicate) or Reranking (retrieve 20, then use a cross-encoder to pick the best 5).

Chain Pitfall: "The model ignores the context and hallucinates"

Your prompt matters. The system prompt must explicitly instruct the model to use only the provided context. Adding "If the reference content does not contain the answer, say so clearly — do not make anything up" dramatically improves faithfulness. We'll measure this quantitatively with RAGAS in the evaluation articles.

Summary

In this article, we took the raw RAG concept from Part 1 and wrapped it in a production-ready framework. Here's what we covered:

The six components of a LangChain RAG pipeline — Loader, Splitter, Embedding, Vector Store, Retriever, and Chain — and what quality risk hides in each one.
A complete, runnable project that loads PDFs, splits them with RecursiveCharacterTextSplitter, embeds them with OpenAI, stores them in ChromaDB, and answers questions via a LangChain LCEL chain.
The chunk size trade-off — in real projects, PDF pages may be very short (e.g., 200 chars). A chunk_size=512 default can produce 0 chunks. 200 + 30 overlap is the safe default.
Common pitfalls at each pipeline stage, from mangled PDF tables to asymmetric retrieval mismatches.

The code in this article is a solid foundation. It handles real PDFs, persists data, and gives you source citations. But it's still a naive RAG pipeline — one query, one retrieval pass, one answer. In the next articles, we'll add the components that separate toy demos from production systems: hybrid search, reranking, query optimization, and evaluation frameworks.

References

LangChain RAG Tutorial — Official LangChain RAG quickstart
LangChain Expression Language (LCEL) — Why and how to use LCEL for composable chains
ChromaDB Documentation — Vector store setup, persistence, and querying

One Open Source Project a Day (No.52): Tank-OS - A Red Hat Engineer Baked an AI Agent Into a Bootable Linux Image Over a Weekend

WonderLab — Thu, 30 Apr 2026 11:59:02 +0000

Introduction

"When an AI Agent starts deleting emails, accessing databases, and calling external APIs, are you certain it can't go out of bounds?"

This is article No.52 in the "One Open Source Project a Day" series. Today's project is Tank-OS (GitHub).

In April 2026, TechCrunch reported on a project that one engineer built over a single weekend. The engineer is Sally O'Malley, Principal Software Engineer at Red Hat's Office of the CTO and a core maintainer of OpenClaw. The project answers a question that becomes more pressing as AI Agents get more capable: when you need to deploy a fleet of AI Agents across a company, how do you ensure every machine is isolated, secure, and consistently updatable?

Tank-OS's answer: pack the Agent, its runtime, the OS, Systemd units, and the upgrade mechanism into a single OCI container image, then boot entire machines directly from that image.

In cloud-native circles, this pattern (called bootc — Boot Container) isn't new. But applying it specifically to solve AI Agent enterprise deployment security? Tank-OS is the most concrete, complete open-source reference implementation available.

TechCrunch headline: "Red Hat's OpenClaw maintainer just made enterprise Claw deployments a lot safer."

What You'll Learn

What bootc is: why "OS as a container" is the next Linux deployment paradigm
rootless Podman Quadlet: how to run containers with no root privileges and no daemon
Tank-OS's layered architecture: how the immutable OS layer and mutable state layer stay separate
Transactional system updates: rolling back an entire machine's OS state with one command
Credential management: how API keys never touch the filesystem in plaintext
Tank-OS vs. NVIDIA NemoClaw: two enterprise AI Agent security approaches compared

Prerequisites

Basic understanding of Linux containers (images, OCI spec)
Familiarity with Podman or Docker
Basic Systemd service management knowledge
Familiarity with OpenClaw (AI Agent framework, 40,000+ GitHub Stars)

Project Background

What Is It?

The core problem Tank-OS addresses can be stated in one sentence: AI Agents are powerful enough to be dangerous, and most deployment approaches haven't taken that seriously.

Sally O'Malley's project documentation describes real incidents: OpenClaw Agents in production have deleted emails and accidentally modified user data. When an enterprise deploys dozens or hundreds of Agent-running machines, the traditional "apt install + config files" approach has four fundamental problems:

Credential security: API Keys live in config files, readable by other processes or accidentally leaked
Update consistency: Different OS versions and library versions across machines cause inconsistent Agent behavior
Failure isolation: A crashed or compromised Agent process can affect the host OS
Fleet management: No unified mechanism to push updates across the whole fleet

Tank-OS uses a bootc + rootless Podman Quadlet stack to systematically address all four.

About the Author

Sally Ann O'Malley (GitHub: sallyom; LobsterTrap is the account she created for this project)

Role: Red Hat Principal Software Engineer, Emerging Technologies, Office of the CTO
OpenClaw connection: Core maintainer of OpenClaw, collaborating with founder Peter Steinberger on enterprise use cases and Red Hat Linux ecosystem integration
Background: Long-time contributor to Red Hat container technologies; deep user and contributor to Podman, bootc, and related Red Hat open-source projects
Origin of Tank-OS: Built in a single weekend. O'Malley anticipated "what happens when OpenClaw enters the enterprise" and wanted a reference architecture ready

Red Hat's official blog also published a companion article: "Building a hardened, image-based foundation for AI agents" — signaling this is not just a personal project but Red Hat's formal exploration of AI Agent infrastructure.

Project Stats

⭐ GitHub Stars: 104
🍴 Forks: 11
🔤 Language: Shell (81.7%), Dockerfile (18.3%)
📄 License: MIT
📦 Pre-built image: quay.io/sallyom/tank-os:latest (amd64 / arm64)
📰 Press: TechCrunch (2026-04-28), Decrypt, Yahoo Tech
📅 Released: April 2026

Key Features

The Core Architecture Shift

Tank-OS elevates "how to safely run an AI Agent" from a software configuration problem to an operating system architecture problem.

Traditional deployment:

Host OS (mutable)
  └── Userspace (mutable)
        └── OpenClaw process (can access host filesystem)
              └── API Keys (plaintext config file)

Tank-OS deployment:

bootc image (immutable OS layer, read-only)
  └── openclaw user (no root privileges)
        └── rootless Podman Quadlet (container, no daemon)
              └── OpenClaw process (isolated from host)
                    └── API Keys (Podman secret store, encrypted)
  └── ~/.openclaw/ (mutable state layer, persisted but isolated from OS)

Use Cases

1. Enterprise AI Agent fleet management
Dozens or hundreds of machines running the same OpenClaw version, kept consistent via unified image update — no "configuration drift" causing behavioral differences across machines.

2. Dev/prod environment parity
Developers run the exact same Tank-OS image locally in a VM, eliminating "works on my machine" problems.

3. Read-only OS security hardening
Even if an Agent process is compromised or has a bug, it cannot modify the host OS filesystem — the OS layer is immutable.

4. Cloud instances and edge devices
SSH key injection via cloud-init enables fast Agent instance startup on AWS EC2, GCP VMs, or Raspberry Pi devices.

5. Fast version rollback
One command to roll back to the previous OS + Agent version — no manual uninstall/reinstall of dependencies.

Quick Start

Option 1: Use the pre-built image (recommended)

No local compilation needed. Use Podman Desktop's BootC extension or bootc-image-builder to generate a virtual disk:

# Create output directory
mkdir -p out-tank-os

# (Optional) Prepare SSH key injection config
cat > config.json << 'EOF'
{
  "blueprint": {
    "customizations": {
      "user": [
        {
          "name": "openclaw",
          "groups": ["wheel"],
          "key": "ssh-ed25519 AAAA...your-public-key..."
        }
      ]
    }
  }
}
EOF

# Build QCOW2 disk image (requires rootful Podman)
sudo podman run \
  --rm \
  --privileged \
  --pull=newer \
  -v ./config.json:/config.json \
  -v ./out-tank-os:/output \
  -v /var/lib/containers/storage:/var/lib/containers/storage \
  quay.io/centos-bootc/bootc-image-builder:latest \
  --type qcow2 \
  --config /config.json \
  quay.io/sallyom/tank-os:latest

# Result: out-tank-os/qcow2/disk.qcow2

SSH in and interact with OpenClaw:

# Log in as openclaw user (not root)
ssh openclaw@<vm-ip>

# Check Agent status
openclaw gateway status --deep

# Run health check
openclaw doctor

# Get Dashboard URL
openclaw dashboard --no-open

# List connected devices
openclaw devices list

Update OS + Agent (transactional):

# First update: switch to latest image
sudo bootc switch --apply quay.io/sallyom/tank-os:latest

# Subsequent updates: pull new version and restart
sudo bootc upgrade --apply

Core Features

Immutable OS — Root filesystem mounted read-only; Agent processes and userspace programs cannot modify system files. Configuration drift is structurally impossible.
rootless Podman Quadlet — OpenClaw runs in a Podman container with no root privileges, lifecycle managed by Systemd Quadlet units. Compromised container process cannot escalate to host OS.
Transactional updates — bootc's atomic update mechanism: updates either succeed completely or roll back completely. No "half-updated broken machine" state.
Secure credential management — API Keys stored in rootless Podman's secret store (encrypted), never appearing in plaintext config files or environment variables. SSH keys injected via cloud-init at first boot, not baked into the image.
Multi-instance support — Single machine can run multiple OpenClaw instances (e.g., one for work, one for research), fully isolated with separate containers, ports, data directories, and secret namespaces.
Multi-architecture — Pre-built images support both linux/amd64 and linux/arm64, covering x86 servers and Apple Silicon/ARM devices.
cloud-init native integration — Standard cloud-init support for AWS, GCP, Azure instance initialization.

How It Compares

Dimension	Tank-OS	Bare metal OpenClaw	Docker compose	NVIDIA NemoClaw
OS immutability	Complete (bootc read-only root)	None	None (host OS mutable)	None
Container privileges	rootless (no daemon)	No containers	Root Docker daemon	K3s + Docker daemon
Transactional updates	Atomic rollback	Not supported	Not supported	Not supported
Credential storage	Podman secret store	Plaintext files	Docker secrets or plaintext	L7 proxy injection
Fleet management	Suitable (unified image)	Difficult	Limited	Suitable (more complex)
Deployment complexity	Low (single image boot)	Low	Medium	High (K3s cluster)
Security policy granularity	Medium (OS-level isolation)	Low	Low	High (multi-layer policy engine)

Deep Dive

Core Technology 1: bootc (Boot Container)

bootc is a Red Hat-led open-source project that implements a counterintuitive idea: manage the entire operating system as an OCI container image.

Traditional Linux update model:

System running
  → apt upgrade / dnf update (replacing files one by one)
  → partial success → system in intermediate state → rollback is hard

bootc update model:

Currently running: image v1.0 (read-only mounted)
  → bootc switch quay.io/sallyom/tank-os:v1.1
  → Download new image layers, write to separate partition
  → Reboot → atomic switch to v1.1
  → Error → bootc rollback → back to v1.0

This is analogous to the A/B partition mechanism in phone OTA updates: new version writes to standby partition, reboots to switch, keeps original partition if something fails.

Tank-OS's bootc/Containerfile builds from quay.io/fedora/fedora-bootc:latest, Fedora's official bootc base image:

FROM quay.io/fedora/fedora-bootc:latest

# Install required packages
RUN dnf install -y \
    podman \
    openssh-server \
    cloud-init \
    python3 \
    shadow-utils \
    sudo \
    vim

# Create openclaw user (UID/GID 1000)
# Configure subuid/subgid range (100000-165535) for rootless Podman
RUN useradd -m -u 1000 -g 1000 openclaw && \
    usermod -aG wheel openclaw && \
    echo "100000:65536" > /etc/subuid && \
    echo "100000:65536" > /etc/subgid

# Enable cloud-init service family
RUN systemctl enable \
    cloud-init-local.service \
    cloud-init-network.service \
    cloud-init.service \
    cloud-config.service \
    cloud-final.service \
    sshd.service

# Inject tank-os scripts and Quadlet units
COPY bootc/ /

Core Technology 2: rootless Podman Quadlet

Why Podman instead of Docker?

Podman is Red Hat's daemonless container tool. The core security advantages:

No central daemon: Docker requires a root-privileged dockerd running continuously; Podman forks container processes directly, no daemon required
rootless mode: Regular users can run full containers; container "root" is an unprivileged user on the host
User namespace isolation: Via subuid/subgid mapping — container UID 1000 maps to host UID 101000

OpenClaw in Tank-OS runs as the openclaw user (UID 1000). Even if the OpenClaw process is exploited:

It cannot access host OS system files (OS layer is read-only)
It cannot escalate to root (rootless Podman namespace isolation)
It cannot affect other users' data

What is Quadlet?

Quadlet (introduced in Podman 4.4+) lets you declare container runtime configuration using Systemd unit file syntax, with Systemd managing the container lifecycle directly:

# /etc/containers/systemd/users/1000/openclaw.container
[Unit]
Description=OpenClaw AI Agent Service
After=network-online.target

[Container]
Image=ghcr.io/openclaw/openclaw:latest
ContainerName=openclaw
UserNS=keep-id:uid=1000,gid=1000
Volume=%h/.openclaw:/home/openclaw/.openclaw:z
Secret=openclaw-api-key,type=env,target=OPENCLAW_API_KEY

[Service]
Restart=always

[Install]
WantedBy=default.target

Systemd reads this file and automatically generates the corresponding .service unit — pulling the image, creating the container, managing restart policy. OpenClaw becomes a standard Systemd service: systemctl --user status openclaw for status, journalctl --user -u openclaw for logs.

State Layer Architecture

Tank-OS strictly separates immutable and mutable layers:

Immutable layer (OS image, read-only)
├── /usr/           ← System software
├── /etc/           ← System config (baked into image)
├── /opt/tank-os/   ← tank-os scripts and tools
└── Quadlet units   ← Container declaration files

Mutable layer (runtime state, persisted)
├── ~/.openclaw/            ← OpenClaw session state, plugins, history
├── ~/.config/containers/   ← Podman user config
└── Podman secret store     ← Encrypted API keys

When you run bootc upgrade, only the immutable layer is replaced. OpenClaw's state, conversation history, and API keys in the mutable layer are fully preserved. Upgrading the Agent version while keeping all session history and configuration intact.

Credential Flow

The API key injection flow is a security highlight:

1. First boot: cloud-init executes
   ↓
2. Reads user-data (from cloud provider metadata or local config)
   ↓
3. Runs tank-os bootstrap script
   ↓
4. Writes API key into rootless Podman secret store (encrypted)
   ↓
5. Quadlet unit starts OpenClaw container
   ↓
6. Secret= directive injects secret as environment variable into container
   ↓
7. API key never appears in plaintext on the filesystem or in process environment

Compared to a traditional .env file approach: Podman's secret store uses system keychain encryption; ordinary file reads cannot retrieve the contents.

Tank-OS vs. NVIDIA NemoClaw

The most common technical question after Tank-OS launched: "How does this differ from NVIDIA's NemoClaw reference architecture?"

Dimension	Tank-OS	NVIDIA NemoClaw
Core technology	Fedora bootc + rootless Podman	Docker + embedded K3s cluster
OS immutability	Complete (bootc partition-level)	None
Credential isolation	Podman secret store	L7 proxy injection (key never touches Agent filesystem)
Security policy	OS-level isolation	seccomp + Landlock + network namespace multi-layer policy
Update mechanism	Atomic image replacement (rollback capable)	Standard container update
Target scenario	Enterprise fleet (standard IT ops toolchain)	Research environments, fine-grained policy control
Deployment complexity	Low (single image, standard VM tools)	High (K3s cluster operations)

Conclusion: Tank-OS fits "treat AI Agent as standard IT infrastructure" enterprise scenarios. NemoClaw fits "need multi-layer fine-grained security policy" research or high-security-requirement environments.

Resources

Official

🌟 GitHub: https://github.com/LobsterTrap/tank-os
📦 Pre-built image: quay.io/sallyom/tank-os (amd64 / arm64)
📚 Build docs: docs/build.md
📚 CLI docs: docs/cli.md

Press & Related

📰 TechCrunch: Red Hat's OpenClaw maintainer just made enterprise Claw deployments a lot safer (Julie Bort, 2026-04-28)
📝 Red Hat blog: Building a hardened, image-based foundation for AI agents (Sally O'Malley)
📖 Fedora bootc docs: docs.fedoraproject.org/en-US/bootc
📖 OpenClaw project: github.com/openclaw/openclaw

Summary

Key Takeaways

Clear problem definition: Tank-OS targets real enterprise AI Agent deployment security problems — credential leakage, configuration drift, update consistency, failure isolation — not technology stack-stacking for its own sake
Mature technology choices: bootc + rootless Podman + Systemd Quadlet are all Red Hat production-grade tools with full enterprise support and documentation. Not experimental.
The layered design is the insight: Strict separation of immutable OS layer and mutable state layer is what lets Tank-OS simultaneously achieve "security isolation" and "state persistence." This design principle is worth borrowing for any stateful AI Agent deployment scenario.
Transactional updates change the ops paradigm: A single bootc upgrade command updates the entire machine's OS + Agent, with atomic rollback on failure. For IT teams managing AI Agent fleets, this is a qualitative change.
The weekend project lesson: Tank-OS uses ~500 lines of Shell scripts and Dockerfile to solve a real enterprise pain point. A reminder that many apparently complex infrastructure problems only need existing mature tools composed correctly.

Who Should Use This

Enterprise IT administrators: Deploying OpenClaw Agent fleets internally with unified management and security compliance requirements
DevOps/SRE engineers: Interested in bootc and immutable infrastructure; looking for an AI Agent deployment reference architecture
Red Hat/Fedora users: Want to integrate AI Agents seamlessly into existing RHEL infrastructure
AI security researchers: Studying AI Agent isolation, credential management, and enterprise security hardening

Where to Start

Start by understanding bootc's core concepts (Fedora bootc docs), then read Tank-OS's bootc/Containerfile and docs/build.md, following the documentation to run an instance in a local VM. The whole process takes under 2 hours, but you'll come away with a very concrete feel for the "OS as container" paradigm.

If you're already deploying OpenClaw in an enterprise environment, Tank-OS can be used almost directly as a production reference architecture. Sally O'Malley is OpenClaw's enterprise maintainer — every design decision in this project comes from real enterprise requirements.

A Question Worth Sitting With

Tank-OS's central premise is worth sitting with: when should AI Agent security be solved at the OS architecture level rather than the application configuration level?

Most current AI Agent deployments treat security as a software problem — use the right API key management library, set the right file permissions, configure the right network rules. Tank-OS argues that once an Agent is powerful enough to delete emails, modify databases, and call external APIs, software configuration isn't enough. The OS itself needs to be part of the security model.

That's a significant architectural claim. The fact that it came from a Red Hat engineer on a weekend — not a multi-year research project — suggests the underlying tools were already ready. Someone just needed to put them together.

Visit my personal site for more useful knowledge and interesting products

One Open Source Project a Day (No.51): VibeVoice - Microsoft's Speech AI That Processes 90 Minutes of Audio in a Single Pass

WonderLab — Wed, 29 Apr 2026 02:32:29 +0000

Introduction

"The fundamental limit of traditional speech AI isn't model quality — it's architecture. They were never designed for long audio."

This is article No.51 in the "One Open Source Project a Day" series. Today's project is VibeVoice (GitHub).

In August 2025, Microsoft Research quietly pushed a repository to GitHub. No launch event. No press release.

The capability it demonstrated: synthesizing 90 minutes of natural multi-speaker conversation — 4 speakers, consistent voices throughout — in a single model pass. For context, ElevenLabs tops out around 5 minutes per call. OpenAI's TTS has similar constraints. The open-source alternatives before this couldn't touch an hour of audio without stitching together segments.

The mechanism behind this is a single architectural decision: a 7.5 Hz ultra-low framerate tokenizer that compresses 90 minutes of audio into ~40,500 tokens — small enough to fit inside an LLM's context window. That's a 3,200x compression ratio compared to the raw audio.

44,900+ Stars, 5,000+ Forks. ICLR 2026 Oral Presentation (top ~2% acceptance rate at the field's premier venue).

What You'll Learn

Why 7.5 Hz tokenization is the key architectural insight: the math behind 3,200x compression
The LLM + diffusion head hybrid architecture: how semantic understanding and acoustic generation divide responsibilities
Three model comparison: ASR-7B, TTS-1.5B, Realtime-0.5B — what each is for
Benchmark numbers: how VibeVoice ASR compares to Gemini 2.5 Pro and Whisper
The misuse incident: why Microsoft pulled TTS access in September 2025 and current availability status

Prerequisites

Basic familiarity with speech AI concepts (TTS, ASR, WER)
Python and Hugging Face Transformers experience
Basic understanding of LLM inference and diffusion models (optional but helpful for architecture sections)

Project Background

What Is It?

VibeVoice is a family of speech AI models from Microsoft Research that uses a unified architecture — 7.5 Hz continuous speech tokenizer + LLM backbone + diffusion head — to address the fundamental limitation of existing speech AI systems: they were built for short audio.

Traditional TTS synthesizes a few minutes at most. Traditional ASR (like Whisper) splits long audio into 30-second chunks and processes them sequentially — meaning speaker tracking breaks at every boundary and global semantic context is lost. Try to process a one-hour podcast or generate a 45-minute audiobook and these systems essentially give up.

VibeVoice's answer: redesign at the architecture level, so the LLM operates directly on compressed audio tokens over the full audio length.

About the Team

Organization: Microsoft Research
Academic recognition: VibeVoice-TTS paper "Expressive Podcast Generation with Next-Token Diffusion" accepted as ICLR 2026 Oral Presentation
Technical report: arXiv 2508.19205
First published: August 2025

Project Stats

⭐ GitHub Stars: 44,900+
🍴 Forks: 5,000+
📝 Commits: 134
📄 License: MIT
🌐 Language: Python (100%)
🤗 HuggingFace: microsoft/vibevoice collection

Key Features

Three Models, One Architecture

Model	Parameters	Core capability	Use case
VibeVoice-ASR	7B	60-min long audio recognition + speaker diarization	Meeting transcription, podcast-to-text
VibeVoice-TTS	1.5B	90-min 4-speaker speech synthesis	Podcast generation, audiobook production
VibeVoice-Realtime	0.5B	~300ms first-chunk latency streaming	Real-time voice assistants, dialogue systems

Core Technical Features

1. Single-pass 60-minute ASR
No segmentation, no concatenation — one forward pass over the entire audio. Speaker tracking doesn't break, global semantic context is maintained end-to-end.

2. 90-minute multi-speaker TTS
Up to 4 speakers. Single synthesis of a complete podcast. Voice consistency is maintained for each speaker across the full duration.

3. 7.5 Hz ultra-low framerate tokenizer
This is the key innovation. At 7.5 Hz vs. Encodec's 25-50 Hz, the compression ratio improves 80x. 90 minutes of audio becomes ~40,500 tokens — fitting comfortably inside a 64K context window.

4. Custom hotword injection
ASR supports dynamic domain vocabulary injection at inference time (medical, legal, product names) without retraining.

5. Expressive synthesis with natural spontaneous speech
TTS generates contextually appropriate emotional variation — laughter, pauses, interjections, natural dialogue prosody.

6. LoRA fine-tuning support
ASR supports LoRA fine-tuning for accent adaptation or domain specialization with minimal labeled data.

7. vLLM inference backend
ASR supports vLLM acceleration for high-throughput production deployment.

Quick Start

ASR model (integrated into HuggingFace Transformers since March 2026):

pip install transformers torch soundfile

from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch
import soundfile as sf

# Load model (GPU required, float16 inference)
processor = AutoProcessor.from_pretrained("microsoft/VibeVoice-ASR")
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    "microsoft/VibeVoice-ASR",
    torch_dtype=torch.float16
).to("cuda")

# Read audio (supports up to 60 minutes)
audio_array, sampling_rate = sf.read("meeting.wav")

# Single-pass inference over the full audio
inputs = processor(
    audio_array,
    sampling_rate=16000,
    return_tensors="pt"
).to("cuda", torch.float16)

with torch.no_grad():
    outputs = model.generate(**inputs)

# Structured output with speaker labels and timestamps
transcript = processor.batch_decode(outputs, skip_special_tokens=True)
print(transcript[0])

Realtime TTS model:

git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice
pip install -e .
# Optional: install Flash Attention for speedup
pip install flash-attn --no-build-isolation

from vibevoice import VibeVoiceRealtime

tts = VibeVoiceRealtime.from_pretrained("microsoft/VibeVoice-Realtime-0.5B")

# Streaming synthesis — audio starts playing ~300ms after first input
for audio_chunk in tts.stream("Welcome to VibeVoice. This is a real-time synthesis demo."):
    play_audio(audio_chunk)

How It Compares

Dimension	VibeVoice	ElevenLabs	OpenAI TTS	Whisper (ASR)
Max single-pass length	TTS 90 min / ASR 60 min	~5 min	Heavy limits	30s chunks stitched
Multi-speaker	Up to 4 speakers	Multiple calls required	Not supported	Post-processing only
First-chunk latency (realtime)	~300ms	~75ms	~200ms	N/A
Local deployment	Fully supported	Not supported	Not supported	Supported
Cost	Free, open-source	$5-330/month	$15/1M chars	Free
Hardware requirement	8GB+ VRAM	None (API)	None (API)	4GB+ VRAM
ICLR academic recognition	Oral 2026	None	None	Yes

Deep Dive

Why 7.5 Hz? The Token Math

To understand VibeVoice's core innovation, start with the fundamental tension it resolves: audio data is enormous vs. LLM context windows are finite.

90 minutes of 24kHz audio encoded with a traditional codec at 50 Hz (like Encodec) produces approximately 270,000 tokens. That exceeds the context capacity of any existing LLM. End-to-end long-audio understanding is effectively impossible.

VibeVoice compresses the framerate from 50 Hz to 7.5 Hz. Token count drops to ~40,500 — a compression ratio of roughly 80x relative to Encodec. 90 minutes of audio now fits inside a 64K context window.

90 min audio × 60 s/min × 7.5 frames/s = 40,500 tokens ✓ (fits in 64K context)
90 min audio × 60 s/min × 50 frames/s = 270,000 tokens ✗ (exceeds any LLM window)

The real question: can 7.5 Hz preserve enough acoustic information for high-quality synthesis? That's what the σ-VAE tokenizer architecture is designed to answer.

Dual Tokenizer Architecture (σ-VAE)

VibeVoice uses two parallel continuous speech tokenizers:

Raw audio (24kHz)
    │
    ├── Acoustic Tokenizer
    │   └── σ-VAE architecture
    │   └── Captures timbre, prosody, pitch, voice quality
    │   └── Output: acoustic latent vectors @ 7.5 Hz
    │
    └── Semantic Tokenizer
        └── Same σ-VAE, but trained with ASR surrogate task
        └── Captures linguistic content, word boundaries
        └── Output: semantic latent vectors @ 7.5 Hz

The two tokenizers describe the same audio from different angles: the semantic tokenizer "knows what was said," the acoustic tokenizer "knows how it was said." This disentanglement is what makes the hybrid LLM + diffusion architecture possible downstream.

LLM + Diffusion Head Hybrid Architecture

Text input / audio token input
          │
    ┌─────▼────────────────────────┐
    │   Qwen2.5 LLM backbone       │
    │   - Understands dialogue semantics   │
    │   - Manages speaker identity changes │
    │   - Outputs semantic token sequence  │
    └─────────────┬────────────────┘
                  │ semantic tokens
    ┌─────────────▼────────────────┐
    │   Diffusion Head             │
    │   - Conditioned on semantic tokens   │
    │   - Generates high-fidelity acoustic latents │
    │   - Handles timbre detail, emotional variation │
    └─────────────┬────────────────┘
                  │ acoustic latents
    ┌─────────────▼────────────────┐
    │   Vocoder                    │
    │   - Decodes acoustic latents to waveform │
    │   - Outputs final audio      │
    └──────────────────────────────┘

The key insight from the ICLR 2026 paper: "Hybrid architecture proves necessary: by explicitly disentangling semantic structure from acoustic realization." Pure LLM approaches underperform on acoustic detail. Pure diffusion approaches drift semantically over long sequences. The hybrid gets both.

ASR: End-to-End Long Audio Understanding

VibeVoice-ASR (7B parameters) doesn't slice long audio. The architectural contrast is stark:

Traditional Whisper:
60 min audio → split into 120 × 30s segments → transcribe each → stitch → speaker tracking breaks ×120

VibeVoice-ASR:
60 min audio → compress to ~27,000 tokens → single LLM inference
             → structured output with speaker labels + timestamps

The model sees the entire conversation's global context. If Speaker A at minute 5 says something relevant to minute 55, the model maintains consistent semantic understanding throughout.

Benchmark performance:

System	Medical audio WER	LibriSpeech test-clean WER
VibeVoice-ASR 9B	8.34%	—
Gemini 2.5 Pro	8.15%	—
ElevenLabs Scribe v2	9.72%	—
OpenAI Whisper large-v3	~11%	—
VibeVoice-Realtime 0.5B	—	2.00%

Realtime Streaming: How 300ms First-Chunk Works

VibeVoice-Realtime (0.5B) is optimized separately for low-latency scenarios using incremental text encoding + parallel acoustic generation:

Text stream input:   "Hello" → "Hello, how" → "Hello, how are" → ...
                       ↓             ↓               ↓
Parallel generation: [chunk 1]    [chunk 2]       [chunk 3]
                       ↓
First audio output: ~200-300ms after first input

Specs:
- 8K context window (supports extended conversation history)
- English only (current version)
- Runs on T4 GPU
- Slight instability on very short inputs (≤3 words)

The Frozen Tokenizer: Long-Term Engineering Value

VibeVoice's tokenizer weights are frozen — only the LLM backbone and diffusion head require training.

This means as Qwen, LLaMA, and other base LLMs continue to improve, VibeVoice can swap in a stronger LLM backbone without retraining the tokenizer. The entire framework upgrades automatically as the LLM ecosystem improves. This is one reason researchers see long-term architectural durability in this design.

The Misuse Incident

On September 5, 2025, Microsoft temporarily pulled access to the primary TTS model, citing "use patterns inconsistent with stated intent." Documented misuse included: synthesizing voices of non-consenting individuals, deepfake audio production, and fraudulent voice content.

Current availability (April 2026):

VibeVoice-ASR: Available, integrated into HuggingFace Transformers
VibeVoice-TTS-1.5B: Restricted access; related calls disabled in public codebase
VibeVoice-Realtime-0.5B: Available for download on HuggingFace
Source code: Fully open-source, MIT license

Microsoft stated they are implementing protective mechanisms (watermarking, access auditing) before restoring full TTS access.

Resources

Official

🌟 GitHub: https://github.com/microsoft/VibeVoice
🏠 Project page: https://microsoft.github.io/VibeVoice/
🤗 HuggingFace models: microsoft/vibevoice collection
📄 Technical report: arXiv 2508.19205
📜 ICLR 2026 paper: Expressive Podcast Generation with Next-Token Diffusion

Demos & Tools

🧪 Colab demo (Realtime): vibevoice_realtime_colab.ipynb
🔁 Replicate online trial: replicate.com/microsoft/vibevoice

Summary

Key Takeaways

The tokenizer is the breakthrough: 7.5 Hz framerate tokenization compresses 90 minutes to 40,500 tokens. Without this, none of VibeVoice's long-audio capabilities are possible — everything else follows from this single decision
Hybrid architecture is necessary, not elegant: The ICLR paper's conclusion is unambiguous: pure LLM fails on acoustic detail, pure diffusion fails on semantic coherence over long sequences. The hybrid solves both
Three models for three real scenarios: ASR-7B for enterprise meeting transcription, TTS-1.5B for podcast/audiobook production, Realtime-0.5B for conversational AI — same architecture, different tradeoffs
ASR quality is production-ready: VibeVoice-ASR 9B at 8.34% medical WER — approaching Gemini 2.5 Pro at 8.15%, beating ElevenLabs and Whisper — is a real result in a demanding domain
The misuse incident is a signal: When TTS capability reaches the level where it needs to be pulled from open access over safety concerns, that tells you something about the capability ceiling it's hitting

Who Should Use This

Podcast creators and content producers: Automate multi-speaker podcast generation from scripts
Enterprise IT teams: Local deployment for meeting transcription with data privacy requirements
AI application developers: Building real-time voice assistants and conversational AI products
Speech AI researchers: Studying long-audio processing, multi-speaker synthesis, diffusion head architectures
Local LLM enthusiasts: Looking for a self-hostable, free alternative to ElevenLabs or OpenAI TTS

Where to Start

Start with VibeVoice-ASR — it's fully integrated into HuggingFace Transformers, has the most complete documentation, and was officially released with Transformers in March 2026. If you need real-time TTS, Realtime-0.5B is the currently available version; the official Colab demo runs out of the box.

For those wanting to understand the architecture deeply, the arXiv technical report (2508.19205) and the ICLR 2026 paper both explain in detail why the hybrid architecture is necessary and why continuous latent space outperforms discrete tokens for high-fidelity acoustic generation.

A Question Worth Sitting With

The misuse incident raises a question that the speech AI field hasn't fully answered: at what capability level does open-source speech synthesis become a dual-use risk that requires gating?

Text generation has largely been treated as freely distributable. Images crossed a threshold where safety concerns became prominent. Voice synthesis — because it can impersonate specific, identifiable individuals — may be in a different category. VibeVoice's temporary takedown isn't a story about one bad actor; it's a data point about where the field is heading.

Visit my personal site for more useful knowledge and interesting products

I Spent Thousands of Dollars in Tokens Building an AI-Driven End-to-End Bug Fix Pipeline

WonderLab — Wed, 29 Apr 2026 02:05:26 +0000

Before We Start

Let me lead with the cost: this system burned thousands of dollars in API tokens during development and debugging.

I still think it's worth writing about. Because this isn't a demo — it's an end-to-end pipeline running against real enterprise systems. A bug ticket in Jira goes in. The AI reads the logs, diagnoses the root cause, writes the fix, runs a Code Review, executes a SonarQube scan, runs unit tests, submits to Gerrit, polls CI/CD for results, adds a human reviewer, and writes back a comment to Jira.

AI drives the whole thing. Humans only step in at critical decision gates.

This article is a full retrospective: how the system is designed, what worked, what failed, what engineering problems I ran into, and how I solved them.

Why Build This

There's a category of software engineering work that consumes enormous human capacity every day but follows a highly standardized process: bug fixing.

A typical bug workflow looks like this:

Receive bug ticket → Read logs → Find root cause → Fix code → Self-test
→ Code Review → Static scan → Unit tests → Submit → Wait for CI
→ Add reviewer → Wait for CR → Merge → Update Jira

Every step involves fixed actions, tool interactions, and a certain amount of accumulated experience. This is exactly the kind of work AI agents are built for.

The challenge: this is a 12+ node long-chain pipeline spanning Jira, log systems, code repositories, review standards, Gerrit, and CI/CD infrastructure. A tool failure at any single node can break the entire flow.

I spent a significant amount of time designing and debugging this workflow on the OpenClaw platform, turning a whiteboard design into a system that actually runs.

Architecture: Three Layers

The system has three layers: Skills, Workflow, and Platform.

┌────────────────────────────────────────────────┐
│              OpenClaw Platform                  │
│  (Enterprise AI Coding Assistant, Claude-based) │
└──────────────────┬─────────────────────────────┘
                   │
┌──────────────────▼─────────────────────────────┐
│              Bug E2E Workflow                   │
│  Node sequence, branching logic, retry policy   │
│  Human gates: checkpoints A / B / C             │
└──────────────────┬─────────────────────────────┘
                   │
┌──────────────────▼─────────────────────────────┐
│                  Skills                         │
│  One independently runnable AI skill per node   │
│  jira-communication / rnd-automotive-issue-     │
│  analyzer / write-code / ph-code-review /       │
│  ph-sonar-scan / ph-junit-ut / commit-format /  │
│  gerrit-verify / ...                            │
└────────────────────────────────────────────────┘

Skills are atomic units. Each skill maps to a single capability and has its own SKILL.md that defines context, input/output contracts, and execution steps. When the agent runs a node, it reads the corresponding skill document and follows the spec.

Workflow is the orchestration layer. Defined in OpenClaw: node execution order, conditional branches (e.g., retry paths when Code Review fails), human gate trigger conditions, and cross-session state management.

Platform is OpenClaw — a Claude-based enterprise AI coding assistant that supports multi-agent concurrency, sub-agent invocation, workspace persistence, and workflow orchestration.

The Full Workflow: 12 Nodes

The workflow starts when a bug ticket arrives and ends when Jira is updated. In between:

#	Node	Skill	Status
1	Fetch bug info & logs	`jira-communication`	✅
2	Root cause analysis	`rnd-automotive-issue-analyzer`	✅
3	Fetch source code	`code-fetch`	✅
4	Fix code	`write-android-code`	✅
5	Code Review	`ph-code-review`	✅
6	Static analysis (SonarQube)	`ph-sonar-scan`	✅
7	Generate & run unit tests	`ph-junit-ut`	✅
8	Commit code	`commit-format`	✅
9	Poll CI/CD verify result	`gerrit-verify`	✅
10	Add Gerrit reviewer	—	✅
11	Automated regression tests	—	✅
12	Write back Jira comment	`jira-communication`	✅

Each node is much more than "call an API." Take root cause analysis as an example:

Download the log attachment from Jira (usually a .zip)
Extract it and locate the relevant log files
Invoke rnd-automotive-issue-analyzer — a skill built specifically for diagnosing crashes, black screens, and system stability issues in automotive Android systems
Output: root cause judgment, affected modules, and suggested fix directions

Only then does code modification begin.

Why is "Fetch Source Code" the hardest node?
To automatically map a bug description to the correct repository, you need a maintained "module → repo" mapping table and bug tickets that have accurate module fields filled in at creation time. On top of that, some repos are massive — pulling fresh every time is too slow, but caching creates workspace storage and staleness problems. This node is currently in cross-team evaluation (DevOps + IT + Development + QA).

Real Tests: 6 Scenarios

The design only matters if it holds up in practice. Here are the 6 most representative cases from actual test runs.

Case 1: The Closest to the Happy Path

This was the most satisfying run. The workflow completed end-to-end as designed:

Fetch bug info
  → Download logs → Extract → Analyze root cause
  → Fix code
  → Code Review (passed)
  → SonarQube scan
  → Unit tests
  → Commit to Gerrit
  → Poll verify status (scheduled)
  → Add Gerrit reviewer
  → Add Jira comment

All 12 nodes, fully AI-driven, zero human intervention. The Gerrit MR was submitted successfully; the Jira ticket was automatically updated with the analysis summary and action log.

One minor glitch: the Commit step was supposed to run automatically but triggered a human confirmation prompt. This was Claude Code's Permission system requiring a second confirmation for certain Git operations. Fixed later by adjusting the Hooks configuration.

Case 2: Tool Failure → Wait for Human

The UT step failed due to a JDK version mismatch in the environment.

When the workflow detected the failure, it didn't crash — it triggered the human notification path, paused, and waited. This is exactly the fallback route designed from the start:

Tool failure
  → Can it be auto-retried?
  → No → Push notification → Wait for human → Resume after confirmation

This case validated that the fault tolerance mechanism works.

Case 3: Session Interrupted → Resume in New Session

A system notification interrupted the agent mid-execution. I opened a new session, sent the same task description, and the agent automatically read the previous workflow_state.json file and resumed from the checkpoint — no restart from scratch.

This relies on the workflow persisting a state file after every node completes:

{
  "bug_id": "AE-33995",
  "current_phase": 4,
  "current_step": "4.3",
  "completed_steps": ["0", "1", "2", "3", "4.1", "4.2"],
  "artifacts": {
    "log_path": "...",
    "review_result": "review_r1.json",
    ...
  }
}

Sessions can be interrupted. Tasks don't get lost. For long-running automation workflows, this is non-negotiable.

Case 4: Code Review Fails → Multi-Round Retry Loop

This is the most interesting section of the whole pipeline.

Code Review failed on the first round (score: 57, with 8 mandatory violations). Instead of immediately escalating to a human, the workflow launched up to 3 automatic fix-and-retry rounds:

Round 1: CR → Failed (57 pts, 8 mandatory violations)
  → Launch sub-agent to fix all 8 violations
  → Round 2 CR → Failed (83 pts, 2 violations)
  → Launch Round 3 fix
  → Round 3 CR → Failed (74 pts, 4 violations)
  → Max retries reached → Trigger human gate B

Each round, the agent reads the previous CR result and makes targeted fixes — not a full rewrite. Round 1 fix summary:

Issue ID	Violation	File
B2-01	Raw Thread → CoroutineScope	PerformanceInfoMonitor.kt
C1-01	`binding!!` → safe call	PerformanceFloatingView.kt
C1-02	`windowManager!!` (5x) → `?.let`	PerformanceFloatingView.kt
C5-01	InterruptedException caught separately	PerformanceInfoMonitor.kt
D1-01	Interface renamed → ICloseClickCallback	PerformanceFloatingView.kt
…	…	…

Case 5: Max Retries Exceeded → Human Gate

Continuing from Case 4. After 3 rounds of fixes, Code Review still failed. But this exposed a deeper and more interesting problem:

Two of the 4 "mandatory violations" in Round 3 were classified as "recommended" in Round 2, then upgraded to "mandatory" the following round.

When an LLM acts as a code review judge, its scoring criteria drift between rounds. The same issue gets rated differently depending on how much history has accumulated in the prompt context. The loop can't converge.

This is a fundamental engineering problem, not something a Prompt tweak can fix.

When human gate B fired, the agent presented a three-round comparison report and offered two options:

A: Accept current code and proceed to submission
   (core bug fixed; remaining issues are style, not functional)

B: Human fixes the remaining 4 items, then notifies agent to re-run quality checks

At this point the human's role is judge, not executor. The AI did everything it could; a person makes the call.

Case 6: CI Pipeline Verify Failed → Human Gate

After submitting to Gerrit, the CI pipeline returned failure votes:

Reviewer	Verified	Compile	UT	Code-Check	Smoke-Test
icvsbgci	-1	+1	+1	+1	0
jenkins.dl	0	0	0	0	-1

The agent read the Gerrit vote state, detected pipeline failures, and entered human gate C with three options:

A — Pipeline issue is known/acceptable; proceed with adding reviewers and updating Jira
B — Needs pipeline re-trigger (manually rebase in Gerrit, then agent re-polls)
C — Abandon this submission and re-evaluate

Engineering War Stories: Three Core Problems

Problem 1: Context Explosion on Long Pipelines

This is the biggest technical challenge the system faces.

A single bug-fix run accumulates: Jira reads, log downloads and extractions, parsing large log files, reading multiple source files, Code Review output, Sonar scan reports… all within one agent turn. In one real test run, a single turn executed 117 tool calls — and then, right after the Sonar scan completed as the agent was about to proceed to the next step, the API request was aborted:

seq=115: sonar poll returned  ← scan result received
seq=117: stopReason="aborted", errorMessage="Request was aborted."

The agent knew exactly what to do next. The turn's accumulated context was simply too large and the server rejected the request outright.

Solution: Sub-Agent Architecture.

Refactor each heavyweight phase into an independent sub-agent call. The main agent only orchestrates; sub-agents execute specific tasks and return structured results. After each sub-agent completes, the main agent receives only a summary — not the full execution trace. Each phase's context stays isolated and doesn't accumulate linearly across the whole pipeline.

Problem 2: LLM Judge Score Drift

Already described in Case 5. The core issue: an LLM's judgment is influenced by its accumulated context. As fix rounds increase, the prompt history grows, and the model's evaluation baseline shifts.

No clean solution exists yet. Directions being explored:

Start a fresh session for each Code Review round (clear all history context)
Decouple the scoring logic from the LLM — use AST-based static rule checking for pass/fail decisions, with the LLM only providing human-readable explanations
Add a consistency validation layer on top of the "mandatory/recommended" classification

Problem 3: Cross-Session Task Recovery

An AI session is fundamentally stateless. In enterprise environments, a bug fix might run for 20 minutes or be interrupted for hours while waiting for CI. You can't rely on a single session staying alive.

The solution is externalizing state. After every node completes, the workflow serializes the current state to the filesystem (workflow_state.json), recording: completed nodes, key artifact paths, and the current phase. When a new session starts, it reads this file first and resumes from the checkpoint.

This is essentially simulating a persistent task queue, with the filesystem as the state store.

What Works, What's Still Missing

What's Running

Jira info retrieval + log download / extraction / analysis
Bug root cause analysis for automotive Android systems (via rnd-automotive-issue-analyzer)
Code fixes (Kotlin, Android application layer)
Code Review with custom standards (multi-round retry + human gate)
SonarQube static analysis (requires SonarQube 10.x — 9.9 doesn't work)
Code submission to Gerrit (with standardized commit messages)
CI/CD result polling + vote status interpretation
Automated Gerrit reviewer assignment
Jira comment write-back

What's Still Being Built

Automatic source code matching is the biggest open problem. Inferring the right repository from a bug description requires a maintained "module → repo" mapping and accurate module fields in bug tickets. This needs cross-team coordination and is currently handled by manually specifying the repo path.

Automated regression testing requires the QA team to co-design the smoke test trigger, execution environment, and result write-back — all of which involve pipeline changes.

Was It Worth It?

Thousands of dollars in tokens, for a pipeline that's still being refined. Worth it?

My answer: depends what you're measuring.

From a pure cost standpoint, no — the per-run token cost is still high and needs optimization.

But from another angle:

This is working proof on real enterprise systems, not a toy demo. It's connected to real production toolchains: Jira, Gerrit, SonarQube, Jenkins.
It reveals the actual boundary of AI agents in enterprise engineering: what can be automated, what needs human judgment, and what's an engineering problem rather than an AI capability limitation.
Sub-Agent architecture, externalized state, and human gates — these three engineering patterns were forged through real failures. They're applicable to any team trying to deploy AI agents in enterprise environments.
Most importantly: the path is viable. Many nodes are still being refined, but "a bug ticket goes in, a Gerrit MR comes out" has been demonstrated.

Summary

This isn't something you knock out in a month. Every skill requires deep familiarity with enterprise tooling. Every workflow node debugged reveals another edge case in how AI agents behave in complex real-world environments.

But the direction is right. AI isn't here to replace engineers — it's here to replace the part of engineering that engineers hate doing but that still has to get done.

If you're working on similar problems, I'd love to compare notes.

Questions or thoughts on your own AI automation experience? Drop a comment below.

One Open Source Project a Day (No.50): The TypeScript Wizard Pushed His .claude Directory to GitHub and Hit #1 Worldwide Overnight

WonderLab — Tue, 28 Apr 2026 06:06:26 +0000

Introduction

"My agent skills that I use every day to do real engineering — not vibe coding."
— Matt Pocock, README first line

This is article No.50 in the "One Open Source Project a Day" series. Today's project is skills (GitHub).

Let's start with what makes this repository unusual.

It's not a new framework. It's not a big company's open-source release. It's not even a runnable program. It's just one engineer's .claude working directory — 21 Markdown files, each telling Claude Code how to behave in a specific engineering scenario.

Matt Pocock pushed this directory to GitHub. No promotion. No blog post explaining it. No YouTube demo. No Hacker News submission.

In 24 hours: 22,000 Stars. GitHub Trending #1 globally.

Final count: 30,800+ Stars.

This wasn't random. The same day, free-claude-code pulled 1,701 stars and awesome-codex-skills pulled 517. Three repositories dominated the Trending page with one shared theme: how to configure your AI to work the way you want it to.

That day, GitHub's community voted: we need this.

What You'll Learn

Why "configuring how your AI works" is becoming a serious engineering practice
All 8 core skills broken down: grill-me, tdd, triage-issue, and more
The "vertical slice" and "anti-vibe-coding" philosophy
How to install and adapt these skills to your own workflow

Prerequisites

Have used Claude Code or a similar AI coding assistant
Familiar with basic software engineering concepts (TDD, refactoring, PRDs)

Project Background

What Is It?

skills is the public version of Matt Pocock's personal Claude Code skill library. Each "skill" is a standalone folder with a SKILL.md as the main file, describing how an Agent should work in a specific engineering scenario — goal, steps, constraints, output format.

Installation is frictionless:

npx skills@latest add mattpocock/skills/grill-me
# Copies the SKILL.md into your .claude/ directory

About the Author: Matt Pocock

Matt Pocock's position in the TypeScript community is something like its chief evangelist:

GitHub: 10,300+ followers
Identity: "TypeScript wizard" (GitHub bio), "I teach devs for a living" (X/Twitter)
Notable projects:
- Total TypeScript: Comprehensive production-grade TypeScript course — his core business
- ts-reset (8,400 ⭐): Called "the CSS Reset for TypeScript"
- evalite (1,500 ⭐): LLM application evaluation in TypeScript
- sandcastle (1,000 ⭐): TypeScript sandbox coding agent orchestration
AI Hero Newsletter: ~60,000 subscribers
Background: Former Vercel engineer, former Stately engineer; used to be a vocal coach (yes, seriously)

He isn't primarily someone who builds tools for others — he's first and foremost an engineer who uses AI seriously every day. This repository is a direct output of his actual work process.

Project Stats

⭐ GitHub Stars: 30,800+ (22,000 in first 24 hours)
🍴 Forks: 2,400+
📝 Commits: 34
📄 License: MIT
📦 Install tool: npx skills@latest
📰 Newsletter: AI Hero (60,000 subscribers)

Key Features

Core Philosophy: Deliberately Slow AI Down

Most people's mental model for AI-assisted coding is "generate" — have it write functions, fill boilerplate, produce whole pages, faster the better. The industry calls this Vibe Coding.

Matt's approach is the opposite. Most of what he teaches Claude to do isn't generating code — it's thinking the problem through before writing a single line.

Here's what that looks like:

`grill-me` — His Most Useful Skill

Matt has called this his most useful skill.

What it does: Turn Claude into a relentless technical interviewer who interrogates your design until you run out of answers.

Not the AI that gently suggests two improvements and calls it done. This one hunts — every branch, every edge case, every assumption you haven't made explicit — one question at a time, until you find yourself thinking "damn, I hadn't thought that through."

Key design choices in the skill:

One question at a time (no question barrages — forces systematic thinking)
Gives recommended answers (doesn't just ask, also helps you think)
Actively explores the codebase to answer questions itself, rather than always asking you
Goal: Resolve every decision branch before a single line of code is written

Scenario: You want to add a caching layer to your project
After grill-me:
  "What's your cache granularity — function-level or request-level?"
  "If cache keys collide, what's your invalidation strategy?"
  "Do you have race conditions? Multiple requests missing cache simultaneously?"
  "If the cache service goes down, what's your fallback?"
  "How do your tests mock the cache?"
  ...
After the interrogation, your design is either much tighter,
or you've realized you shouldn't build this at all.

`tdd` — Enforced Red-Green-Refactor

This skill doesn't let Claude write out a full feature at once. It enforces a strict TDD workflow:

Tracer Bullet (build minimal working path first)
    ↓
Increment loop:
  Write one failing test
    ↓
  Write the minimum code to make it pass
    ↓
  Refactor (code cleaner, tests still green)
    ↓
  Next test...

Key constraints from the SKILL.md:

No horizontal slicing: Writing all tests first before any implementation is forbidden (leads to over-engineering)
Tests verify behavior, not implementation: Test through public interfaces, not internals
Ships with 6 reference documents: deep-modules.md, interface-design.md, mocking.md, refactoring.md, tests.md

Slow, deliberate, old-fashioned — and exactly how serious engineers actually write code.

`triage-issue` — Diagnose First, Fix Second

When a bug report comes in, most people (and most AI assistants) react the same way: find the line, change it, open a PR.

Matt's triage-issue skill adds a step before fixing: thorough diagnosis.

Five-stage pipeline:

1. Capture the problem (understand the symptom)
2. Explore and diagnose (comb the codebase, find root cause)
3. Determine the fix (not the first viable fix — the best fix)
4. Design a TDD fix plan (write the test first, then implement)
5. Create a GitHub Issue (diagnosis + fix plan + acceptance criteria)

The output isn't code. It's a GitHub Issue. When you (or another Agent) go to actually fix it, there's already a clear root-cause analysis and a test-driven roadmap waiting.

`design-an-interface` — Force Real Alternatives

This skill implements John Ousterhout's "Design It Twice" principle from A Philosophy of Software Design: for any important decision, generate at least two genuinely different approaches before choosing.

Implementation: Launch multiple parallel sub-Agents, each constrained to a different dimension:

Sub-Agent A: Minimum method count (minimize interface surface)
Sub-Agent B: Maximum flexibility (maximize extensibility)
Sub-Agent C: Optimize common cases (minimize usage friction)

After the three proposals, it evaluates them across four dimensions: simplicity, generalizability, implementation efficiency, and "depth" (how much complexity does the interface hide). Then it helps you decide.

`to-issues` — Vertical Slice Decomposition

Breaking a feature plan into GitHub Issues sounds ordinary. The key is how it slices: vertical slices.

❌ Horizontal slicing (don't do this):
   Issue 1: Database Schema
   Issue 2: API layer
   Issue 3: Frontend UI
   Issue 4: Tests

✅ Vertical slicing (Matt's way):
   Issue 1: Users can create a basic draft (schema + API + UI + tests, all layers)
   Issue 2: Drafts can contain rich text content
   Issue 3: Drafts can be published

Each Issue is classified as:

HITL (Human In The Loop): Requires human decision before proceeding
AFK (Away From Keyboard): Safe to run unattended

`git-guardrails-claude-code` — Stop AI From Deleting Your Work

Intercepts dangerous git commands via Claude Code's PreToolUse hook. Blocked operations:

git push (including --force)
git reset --hard
git clean -f / -fd
git branch -D
git checkout .
git restore .

When Claude tries to run these, it receives: "it lacks authority to access these commands." Configurable at project level or globally.

Full Skill Inventory

Skill	Purpose
`to-prd`	Conversation context → structured PRD, submitted as GitHub Issue
`request-refactor-plan`	Creates detailed refactor plans with small commits, refined through user interview
`improve-codebase-architecture`	Finds architectural "deepening" opportunities using CONTEXT.md and ADR docs
`setup-pre-commit`	Configures Husky + lint-staged + Prettier + type checks
`ubiquitous-language`	Extracts DDD-style shared vocabulary from the current conversation
`write-a-skill`	Creates new skills following the standard structure (a skill that writes skills)
`edit-article`	Improves writing by restructuring sections and clarifying language
`obsidian-vault`	Search, create, and manage notes in an Obsidian vault with wikilinks

Deep Dive

What a Skill File Actually Looks Like

Each skill is a standalone folder with SKILL.md as the main file. The tdd skill as an example:

skills/tdd/
├── SKILL.md           ← Main skill definition
├── deep-modules.md    ← Reference: deep module design principles
├── interface-design.md
├── mocking.md
├── refactoring.md
└── tests.md

The SKILL.md structure:

# TDD

## Goal
Build features one vertical slice at a time using TDD...

## Steps
1. Tracer Bullet: First, write the minimal end-to-end path...
2. Increment: For each behavior to implement...
   a. Write a failing test
   b. Write minimal code to pass the test
   c. Refactor if needed

## Rules
- NO horizontal slicing...
- Tests should verify behavior, not implementation details...

## Resources
@deep-modules.md
@interface-design.md

Plain text, no special syntax, no runtime dependencies. It's configuration as documentation.

Why "Configuring Your AI" Is Becoming Engineering Practice

Three AI-configuration repos dominating Trending on the same day isn't coincidence. It points at something structural:

Same models (GPT-4o, Claude Opus) — available to everyone
Same tools (Cursor, Claude Code) — download and install
Same subscription fees

Why do some people get 2x productivity from Claude Code
while others fight hallucinations and throwaway code all day?

The gap isn't the model. It's the configuration:
  • How you describe the boundaries of a problem
  • Which checkpoints require human sign-off
  • What engineering conventions to follow
  • Which mistakes you tolerate, which you don't

Nobody teaches you this. Nobody sells it.
You grind it out across weeks of daily use.

Matt published what he ground out. The industry called this the "npm moment" for Claude Code Skills — just as npm let the Node.js community share reusable packages, Skills is enabling the Claude Code community to share workflow recipes. JetBrains and other major vendors started publishing official skills packages shortly after this repo went viral.

"Is Your .claude Directory Empty?"

This is the most valuable question this project raises.

If your .claude directory (or .cursorrules, or AGENTS.md) is empty, you're starting from zero every time. Your AI doesn't remember the mistake you made last month, doesn't know your project's architecture conventions, doesn't understand your team's code standards. It's a new hire every single session.

Matt's 21 skills aren't meant to be copied wholesale — he writes TypeScript, builds education products, his workflow probably looks different from yours. But he's provided a living sample of what it looks like when an engineer treats their AI configuration as a serious engineering artifact.

Resources

Official

🌟 GitHub: https://github.com/mattpocock/skills
📦 Install: npx skills@latest
📰 AI Hero Newsletter: https://aihero.dev (60,000 subscribers)
🎓 Total TypeScript: https://totaltypescript.com

ts-reset (8,400 ⭐): https://github.com/total-typescript/ts-reset
evalite (1,500 ⭐): LLM application evaluation tooling
Simon Willison's LLM skills: Another widely-referenced personal Claude skills repo
MCP Protocol: https://modelcontextprotocol.io

Summary

Key Takeaways

Anti-vibe-coding: Most of Matt's skills aren't about generating code faster — they're about thinking through the problem more thoroughly before writing any. That's how senior engineers use AI.
Vertical slice first: tdd, to-issues, and triage-issue all enforce "complete one full path at a time" over "finish all of layer X first" — this is a genuine engineering philosophy, not just a style preference
Skill files are engineering artifacts: Like Dockerfile, .eslintrc, and tsconfig.json, your AI configuration is infrastructure worth maintaining and versioning
The "npm moment" significance: This repo's virality marked a shift — Claude Code Skills is becoming a community-shared ecosystem, not just personal configuration files
The gap is in configuration, not the model: Same Claude Opus, two different engineers — the difference is what they've built around it

Who Should Use This

Claude Code power users: Already using Claude Code and want a stricter, more reliable workflow
TypeScript engineers: Some skills (migrate-to-shoehorn, scaffold-exercises) are TypeScript-ecosystem specific
Engineers building their own skill library: Matt's repo is the best reference sample available; write-a-skill can even generate new skills for you
Engineering leads standardizing team AI practices: git-guardrails, to-issues, triage-issue can directly standardize how your team uses AI

A Question Worth Sitting With

The line in Matt's README deserves to be read more than once: "Agent skills for real engineers, not vibe coding."

Behind that line is a judgment: there are two paths for AI-assisted engineering. One path uses AI to generate more code, faster — Vibe Coding. The other uses AI to make every decision before that code more rigorous — AI as a stricter thinking partner, not a faster typist.

Both paths ship software. But they arrive at very different places.

Visit my personal site for more useful knowledge and interesting products

One Open Source Project a Day (No.49): free-claude-code - Run Claude Code for Free with One Environment Variable

WonderLab — Mon, 27 Apr 2026 06:10:47 +0000

Introduction

"When a tool is too expensive, programmers build a cheaper key."

This is article No.49 in the "One Open Source Project a Day" series. Today's project is free-claude-code (GitHub).

Claude Code is Anthropic's AI coding agent — deeply integrated into the terminal and VS Code, able to autonomously read files, edit code, and run commands. The catch: it requires a real Anthropic API key, and those API calls add up fast.

free-claude-code is built on a deceptively simple insight: Claude Code is just an HTTP client that calls the Anthropic Messages API. If you run a compatible proxy server locally that intercepts those requests and forwards them to any free or cheap backend, Claude Code has no idea the difference. Change one environment variable (ANTHROPIC_BASE_URL), and suddenly your $20/month tool is running on NVIDIA's free GPU credits.

14,300+ Stars, 2,000+ Forks — it surged to the top of GitHub Trending in April and stayed there for four consecutive days. The author, Ali Khokhar (Alishahryar1), was virtually unknown before this project. His other pinned repos have 3 stars each. This is a textbook "one-hit wonder" in the best sense.

What You'll Learn

The core proxy architecture: how ANTHROPIC_BASE_URL redirection works
Multi-backend routing: NVIDIA NIM free tier, OpenRouter free models, local Ollama
API format translation: adapting Anthropic Messages ↔ OpenAI Chat Completions
Thinking Token conversion: mapping <think> tags to Claude's native thinking blocks
Discord/Telegram bot mode: remote-control Claude Code from your phone
Real limitations: what you actually lose when you swap out the model

Prerequisites

Comfortable with terminal operations and environment variables
Basic familiarity with Claude Code
Basic understanding of REST APIs (optional)

Project Background

What Is It?

free-claude-code is a local HTTP proxy server built with FastAPI that emulates the Anthropic API interface. When Claude Code sends API requests, the proxy intercepts them, translates formats, routes to a configured free backend, converts the response back to Anthropic format, and returns it to Claude Code — completely transparently.

The name is blunt and accurate: free-claude-code. It runs Claude Code for free.

The core architecture in one sentence:

Claude Code → sends Anthropic API requests
           → proxy intercepts at localhost:8082
           → translates to OpenAI format
           → forwards to NVIDIA NIM / OpenRouter / Ollama
           → translates response back to Anthropic format
           → returns to Claude Code as if nothing happened

About the Author

Author: Ali Khokhar (GitHub: Alishahryar1)
Location: Sunnyvale, California
Bio: "Writing easily understandable code..."
Background: Individual developer; before this project, essentially no significant GitHub presence. Classic "overnight" open-source success.

Project Stats

⭐ GitHub Stars: 14,300+ (10,700+ in April alone)
🍴 Forks: 2,000+
👥 Contributors: 22
📄 License: MIT
📈 Trend: Topped GitHub Trending (Python + global) April 24-27

Key Features

6 Free Backends

NVIDIA NIM    → Free tier: 40 req/min, models: GLM-4, Llama 3, Mistral
OpenRouter    → 580+ models, many with daily free quotas
DeepSeek      → Ultra-low cost, native Anthropic Messages format support
LM Studio     → Local GUI for running quantized models
llama.cpp     → CPU/GPU inference, maximum control
Ollama        → One-line local model deployment, completely offline

Per-Model-Tier Routing

Claude Code internally uses three model "tiers" (Opus, Sonnet, Haiku) for different task complexity levels. free-claude-code lets you route each tier to a different backend:

# Route heavy reasoning tasks to a large model
MODEL_OPUS=nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1

# Route standard coding to a mid-size model
MODEL_SONNET=nvidia_nim/nvidia/llama-3.3-70b-instruct

# Route simple tasks (status checks, classification) to a small fast model
MODEL_HAIKU=nvidia_nim/meta/llama-3.1-8b-instruct

Thinking Token Conversion

Some open-source models (DeepSeek-R1, QwQ, GLM-Z1) output their reasoning wrapped in <think> tags. The proxy automatically extracts these and maps them to Claude's native thinking content blocks:

# Proxy logic (simplified)
if "<think>" in model_output:
    thinking_content = extract_between_tags("<think>", "</think>", model_output)
    response["content"].insert(0, {
        "type": "thinking",
        "thinking": thinking_content
    })

This means VS Code's Claude Code extension — which collapses and displays thinking blocks — works correctly with open-source reasoning models.

Intelligent Rate Limiting

NVIDIA NIM's free tier caps at 40 requests/minute. Claude Code's aggressive request pattern (frequent context sends, tool calls) hits this quickly. The proxy implements two-layer protection:

Proactive throttling: Before sending a request, predicts whether it would exceed the rate limit and preemptively waits
Reactive backoff: On receiving 429 Too Many Requests, parses retry-after headers or applies exponential backoff
Concurrency control: PROVIDER_MAX_CONCURRENCY env var limits simultaneous in-flight requests

Discord/Telegram Bot Mode

Beyond pure proxying, the project manages Claude Code's full lifecycle in bot mode:

Phone message: "refactor the auth module to use async/await"
      ↓
Telegram/Discord bot receives message
      ↓
Spawns Claude Code subprocess in CLAUDE_WORKSPACE directory
      ↓
Streams real-time output back to your chat window
      ↓
Session persisted for follow-up messages

Quick Start

# 1. Clone the repo
git clone https://github.com/Alishahryar1/free-claude-code.git
cd free-claude-code

# 2. Copy config
cp .env.example .env

# 3. Edit .env — add your free NVIDIA NIM key
# NVIDIA_NIM_API_KEY=nvapi-xxxx
# MODEL_OPUS=nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1
# MODEL_SONNET=nvidia_nim/nvidia/llama-3.3-70b-instruct
# MODEL_HAIKU=nvidia_nim/meta/llama-3.1-8b-instruct

# 4. Start the proxy (requires uv: pip install uv)
uv run uvicorn server:app --host 0.0.0.0 --port 8082

# 5. In another terminal, run Claude Code through the proxy
ANTHROPIC_BASE_URL="http://localhost:8082" claude

That's it. Claude Code is now running on NVIDIA's free GPU infrastructure. No Anthropic API key. No charges.

Deep Dive

Architecture: Transparent Proxy Pattern

Claude Code CLI / VS Code Extension
              │
              │  POST /v1/messages (Anthropic format)
              ▼
┌─────────────────────────────────────┐
│        free-claude-code Proxy       │
│  ┌────────────────────────────────┐ │
│  │    FastAPI Router              │ │
│  │  ┌──────────────────────────┐  │ │
│  │  │ Request parsing &        │  │ │
│  │  │ model tier routing       │  │ │
│  │  └──────────┬───────────────┘  │ │
│  │             │                   │ │
│  │  ┌──────────▼───────────────┐  │ │
│  │  │ Format translation layer │  │ │
│  │  │  Anthropic → OpenAI      │  │ │
│  │  │  (or direct passthrough) │  │ │
│  │  └──────────┬───────────────┘  │ │
│  │             │                   │ │
│  │  ┌──────────▼───────────────┐  │ │
│  │  │ Rate limiting & retry    │  │ │
│  │  └──────────┬───────────────┘  │ │
│  └─────────────┼─────────────────┘ │
└────────────────┼────────────────────┘
                 │
    ┌────────────┼────────────┐
    ▼            ▼            ▼
NVIDIA NIM   OpenRouter    Ollama
(free tier)  (free models) (local)

The Hard Part: API Format Translation

This is the real engineering challenge. Anthropic Messages API and OpenAI Chat Completions API differ significantly:

Anthropic format (what Claude Code sends):

{
  "model": "claude-opus-4-5",
  "max_tokens": 8096,
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Refactor this code"},
        {"type": "tool_result", "tool_use_id": "xxx", "content": "..."}
      ]
    }
  ],
  "tools": [...],
  "thinking": {"type": "enabled", "budget_tokens": 5000}
}

OpenAI format (what NVIDIA NIM / OpenRouter accepts):

{
  "model": "nvidia/llama-3.1-nemotron-ultra-253b-v1",
  "messages": [
    {"role": "user", "content": "Refactor this code"}
  ],
  "tools": [...],
  "stream": true
}

The translation points the proxy handles:

Multimodal content: Flatten Anthropic's content array (text, images, tool results) into OpenAI's string or object format
Tool calls: tool_use blocks ↔ function_call / tool_calls
Streaming: Convert backend SSE stream to Anthropic's event: content_block_delta format
Thinking blocks: Extract <think> tags into separate thinking content blocks

Full Environment Variable Reference

# === Backend selection ===
NVIDIA_NIM_API_KEY=nvapi-xxxxx
OPENROUTER_API_KEY=sk-or-xxxxx
DEEPSEEK_API_KEY=sk-xxxxx
OLLAMA_BASE_URL=http://localhost:11434

# === Model tier routing ===
MODEL_OPUS=nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1
MODEL_SONNET=nvidia_nim/nvidia/llama-3.3-70b-instruct
MODEL_HAIKU=nvidia_nim/meta/llama-3.1-8b-instruct

# === Thinking support ===
ENABLE_SONNET_THINKING=true
ENABLE_OPUS_THINKING=true

# === Rate limiting ===
PROVIDER_RATE_LIMIT=1           # max requests/second
PROVIDER_MAX_CONCURRENCY=5      # max concurrent requests

# === Bot config (optional) ===
MESSAGING_PLATFORM=telegram
TELEGRAM_BOT_TOKEN=xxxxx
ALLOWED_TELEGRAM_USER_ID=123456789

Limitations Worth Knowing

Before setting this up, these caveats matter:

1. Model quality gap is the fundamental trade-off
free-claude-code gives you the Claude Code interface, not Claude models. Open-source models on free tiers lag behind Claude Sonnet/Opus on complex multi-step reasoning, instruction following stability, and tool call reliability. One YouTube reviewer put it clearly: "Simply wrapping Claude's application layer around open LLMs will not produce the same quality output."

2. NVIDIA NIM free tier exhausts quickly
40 req/min sounds like a lot until Claude Code starts sending its full context window on every turn. Rate-limited sessions introduce noticeable pauses. Real coding sessions will hit the ceiling.

3. Tool call compatibility is imperfect
Claude Code depends heavily on structured tool calls (file read/write, bash execution, search). Open-source models vary in their tool call formatting discipline. The proxy includes heuristic parsing as a fallback, but failures happen — especially with smaller models.

4. Claude-specific features unavailable

Computer Use (Anthropic's vision+interaction feature)
True Extended Thinking (deep reasoning mode)
Latest Claude training data and safety alignment

5. Local models need real hardware
Running quality coding models locally (e.g., Qwen2.5-Coder-32B via Ollama) requires 20-24GB+ VRAM. Most consumer GPUs won't run the best models comfortably.

How It Compares

Project	Approach	vs. free-claude-code
openclaw	Alternative Claude Code CLI	Independent implementation; doesn't use the official Claude Code client
Aider	Standalone AI coding CLI	Mature and stable; native multi-model support; different UX entirely
OpenCode	Terminal AI coding tool	Native multi-model design, not a proxy
free-claude-code	API proxy layer	Preserves the complete Claude Code UX; just swaps the backend

The unique value: users already fluent in Claude Code's workflow don't need to learn a new tool. One environment variable change, costs go to zero.

Resources

Official

🌟 GitHub: https://github.com/Alishahryar1/free-claude-code
📋 Issues: https://github.com/Alishahryar1/free-claude-code/issues

Summary

Key Takeaways

The insight is the trick: Claude Code is just an Anthropic API client. ANTHROPIC_BASE_URL redirection is all it takes — the engineering complexity is in the format translation layer, not the core idea
Real technical work: Anthropic ↔ OpenAI format conversion, streaming response handling, tool call adaptation, and Thinking Token mapping — these are non-trivial engineering challenges the project solves well
"Free" has a price: You save on API costs but trade model quality. NVIDIA NIM's free tier rate limits will make sessions feel slow during heavy usage
Community momentum: 14k+ stars, 22 contributors, active issues — the project is iterating fast on the rough edges
Use case clarity: Best for learning/experimentation and offline private deployment; not a substitute for production-grade Claude in complex agentic tasks

Who Should Use This

Students and hobbyists: Want to experience Claude Code's full terminal agent workflow without paying for an API subscription
Model researchers: Want to compare open-source models using Claude Code's interface as a consistent test harness
Enterprise private deployment: Using Ollama for a fully offline, air-gapped AI coding assistant on internal infrastructure
Remote coding enthusiasts: Using the Telegram/Discord bot to control a remote server's Claude Code instance from a phone

A Question Worth Sitting With

free-claude-code's virality is more than just "free stuff is popular." It reflects a specific tension: Anthropic built a genuinely excellent developer tool, then gated it behind a per-token billing model that makes sustained use expensive. The community's immediate response was to route around it.

The question isn't whether this is "ethical" — it's clearly operating in a gray zone. The more interesting question is what it signals: when developers immediately build free alternatives to paid AI tools, it suggests the underlying capability is perceived as infrastructure, not a premium product. Infrastructure wants to be free or at least flat-rate. The market is making that preference clear.

Visit my personal site for more useful knowledge and interesting products

Claude Code + SonarQube Static Analysis: The AI Quality Loop is Finally Closed

WonderLab — Mon, 27 Apr 2026 05:44:14 +0000

Introduction

Sound familiar? You finish writing some code, open a PR, and then CI blows up with a wall of static analysis findings. You spend the next hour tracking down issues that, honestly, you could have caught at the keyboard.

SonarQube is one of the most widely adopted code quality platforms in the industry. It detects bugs, vulnerabilities, code smells, and security hotspots, and tracks coverage and duplication. Now it can be integrated directly into Claude Code — so the same AI session that helps you write code can also scan it, flag problems, and fix them on the spot.

This article is a complete walkthrough: from first install to day-to-day usage. But before we get into the steps, there is one critical gotcha you need to know about first — otherwise you'll burn hours troubleshooting something that has nothing to do with your configuration.

The Big Gotcha: SonarQube Server Version

Key point: sonarqube-cli (and the sonarqube-mcp-server behind it) does not support SonarQube Server 9.x. You must be running 10.x or later.

Our team had SonarQube 9.9 LTS deployed — a rock-solid long-term support release that plenty of organizations still run. We followed the official docs to the letter, set up sonarqube-cli, ran authentication, and kept getting connection errors with no useful message. Spent a good chunk of time ruling out network issues, token problems, and firewall rules.

The root cause turned out to be simple: sonarqube-mcp-server calls the next-generation SonarQube API (/api/v2/ prefix). Those endpoints were introduced in version 10.x. They simply do not exist on a 9.9 instance.

Fix: We spun up a fresh SonarQube Server 10.x instance, pointed the config at it, and everything worked immediately.

Check your version before anything else. Log into the SonarQube console and look in the bottom-right corner or under Administration > System. If you see a 9.x version number, you need to upgrade or deploy a separate 10.x instance before continuing with any of the steps below.

Version Compatibility at a Glance

SonarQube Server Version	Works with sonarqube-cli / MCP?
9.9 LTS and below	❌ Not supported
10.0 – 10.x	✅ Supported
SonarQube Cloud	✅ Supported

Architecture Overview

Understanding the three-layer stack makes troubleshooting much easier.

Claude Code
    │
    ├── sonarqube-agent-plugins    ← Plugin layer: slash commands and Skills
    │       └── /sonar-analyze, /sonar-integrate, etc.
    │
    ├── sonarqube-cli (sonar)      ← CLI layer: lightweight tool for auth and analysis
    │       └── ~/.local/share/sonarqube-cli/bin/sonar
    │
    └── sonarqube-mcp-server       ← MCP layer: containerized server for deep analysis
            └── Runs via Docker/Podman, calls SonarQube Server API

Each layer has a distinct responsibility:

sonarqube-agent-plugins: The official plugin bundle that injects Sonar slash commands and Skills into Claude Code
sonarqube-cli: A lightweight CLI tool that handles authentication and basic analysis — no container required
sonarqube-mcp-server: A Docker/Podman container that acts as the MCP server, powering advanced capabilities like coverage, quality gates, and duplication detection

Prerequisites

Make sure the following are in place before you start:

Node.js 18+: Required by the plugin's SessionStart hook script (scripts/setup.js)
Docker or Podman: The MCP Server runs as a container
- On macOS in corporate environments, Docker Desktop is often disallowed (licensing); use Podman instead (see the macOS section below)
- Linux and Windows can use Docker directly
SonarQube Server 10.x (or SonarQube Cloud): Deployed and reachable over the network
Browser logged into SonarQube: The OAuth authorization step requires a browser session

Installation

Step 1: Install the sonarqube-agent-plugins Plugin

Open Claude Code and run these two slash commands in order:

/plugin marketplace add SonarSource/sonarqube-agent-plugins

/plugin install sonarqube@sonar

Then reload the plugins (or start a fresh Claude Code session):

/reload-plugins

Verify: Type /sonar in Claude Code. If you see a list of Sonar commands, the plugin installed correctly.

Step 2: Run the Integration Wizard

In Claude Code, run:

/sonar-integrate

This launches an interactive wizard that walks you through: installing sonarqube-cli → connecting to SonarQube Server → completing OAuth authorization → registering the MCP Server.

2.1 Install sonarqube-cli

The wizard installs sonarqube-cli automatically in the first step. The CLI lands at:

~/.local/share/sonarqube-cli/bin/sonar

If you get a "command not found" error when running sonar later, add it to your PATH manually:

# Add to ~/.zshrc or ~/.bashrc
echo 'export PATH="$HOME/.local/share/sonarqube-cli/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

2.2 Connect to SonarQube Server

The wizard prompts you to choose a connection method. Select option 4: "Type something" and enter your SonarQube Server URL directly:

http://your-sonarqube-server:9000/

Reminder: this URL must point to a SonarQube 10.x or later instance. Pointing at a 9.9 server here will cause auth and scan failures downstream.

2.3 Authenticate

Once the wizard recognizes the server address, it tells you the next step. Run the auth command in the Claude Code terminal or a separate terminal window:

sonar auth login -s http://your-sonarqube-server:9000/

This automatically opens a browser and navigates to the SonarQube authorization page. Click Allow connection to complete the flow.

Order matters: make sure your browser is already logged into SonarQube before running this command. If you're not logged in, the redirect will take you to the login screen first and the flow gets messier.

Once you've clicked Allow in the browser, return to Claude Code and tell it "Authorization complete." Claude Code will run a connection health check, and if everything passes, moves to the next step.

2.4 Choose Integration Scope

The wizard asks where to apply the SonarQube integration:

Current project: Takes effect only in the current working directory. Config is written to the project-level .claude/ directory. Recommended for shared codebases.
Global: Applies to all projects. Config goes into ~/.claude/.

After you choose, Claude Code automatically registers the MCP Server in the appropriate config file.

Once this is done, exit Claude Code completely.

Corporate Network: Replacing the Docker Image Mirror

In corporate environments, Docker Hub (registry-1.docker.io) is often blocked. You'll need to point the sonarqube-mcp-server image at an internal mirror.

Edit the MCP Config

The MCP config lives in ~/.claude.json (global) or .claude/claude.json (project-level). Find the sonarqube entry under mcpServers and update the image reference.

Example (using a JFrog Artifactory proxy):

// Before
"image": "sonarsource/sonarqube-mcp-server:latest"

// After — internal mirror proxy
"image": "jfrog.yourcompany.com/external-docker-public-virtual/sonarsource/sonarqube-mcp-server:latest"

Mapping rule: prepend your internal registry host to the original image name: <registry>/<original-image>:<tag>. Check with your infrastructure team for the exact proxy URL and path prefix.

macOS: Using Podman Instead of Docker

Many corporate macOS environments prohibit Docker Desktop due to licensing requirements. Podman is a fully open-source, Docker-compatible alternative.

Install Podman

Download the macOS installer (.pkg) from podman.io and double-click to install.

Add Podman to your PATH:

echo 'export PATH="/opt/podman/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

Verify:

which podman
# /opt/podman/bin/podman

podman --version
# podman version 5.x.x

Initialize the Podman Machine

On macOS, Podman needs a lightweight VM to run containers (similar to Docker Desktop's VM layer):

# First-time setup (downloads ~500 MB base image — takes a while)
podman machine init

# Start the VM
podman machine start

# Check status
podman machine list
# NAME                     VM TYPE  CREATED  LAST UP            CPUS  MEMORY  DISK SIZE
# podman-machine-default*  applehv  ...      Currently running  5     2GiB    100GiB

After every Mac restart, Podman Machine does not start automatically. You need to run podman machine start manually, or configure a launchd service to start it on login.

Start Up and Verify

With all configuration done, quit Claude Code fully and relaunch it so the MCP config takes effect.

On the first launch, Claude Code will pull the sonarqube-mcp-server container image. This will be slow the first time — give it a minute or two. Then run:

/mcp

When sonarqube shows as connected, the integration is live.

Optional: sonar-project.properties

Create a sonar-project.properties file in your project root to pre-declare project metadata. This lets the analysis commands auto-detect the project without needing you to pass the project key manually each time:

sonar.projectKey=my-project
sonar.projectName=My Project
sonar.projectVersion=1.0
sonar.sources=src
sonar.sourceEncoding=UTF-8

Command Reference

CLI Commands (no MCP required)

Command	Description
`/sonar-integrate`	Re-run setup: re-authenticate, re-register MCP, reinstall hooks
`/sonar-list-projects [keyword]`	List accessible SonarQube projects
`/sonar-list-issues [project] [--severity CRITICAL]`	Search and filter project issues
`/sonar-fix-issue <rule> <file>[:<line>]`	Fix a specific rule violation in a file

MCP Commands (requires connected MCP Server)

Command	Description
`/sonar-analyze [file path]`	Analyze a single file and display issues
`/sonar-quality-gate [project] [--branch]`	Check Quality Gate status
`/sonar-coverage [project] [--max N] [--file]`	View code coverage
`/sonar-duplication [project] [--pr N] [--file]`	View code duplication
`/sonar-dependency-risks [project] [--pr N]`	View dependency risks (requires Advanced Security)

Example: Scan a Single File

/sonar-analyze ./src/main/java/com/example/UserService.java

Claude Code calls the MCP Server, runs the analysis, and returns a structured list of bugs, vulnerabilities, and code smells with fix suggestions. You can immediately ask Claude to act on the results:

Fix all CRITICAL issues found in the last scan

Troubleshooting

Authentication Fails

# Re-authenticate (overwrites the old token)
sonar auth login -s http://your-sonarqube-server:9000/

# Check auth status
sonar auth status

Run this any time you deploy a new SonarQube Server or switch instances.

`sonar` Command Not Found

echo 'export PATH="$HOME/.local/share/sonarqube-cli/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

MCP Server Fails to Start

Verify the container runtime is working: docker info or podman info
Double-check the image path in ~/.claude.json (especially for corporate mirror setups)
On macOS, confirm Podman Machine is running: podman machine start

404 / API Errors When Connecting to SonarQube (the most common failure)

If you see 404s or API errors during auth or scanning, the server version is almost certainly the culprit:

# Check the server version via API
curl http://your-sonarqube-server:9000/api/server/version

If the response is 9.9.x, you need to upgrade to 10.x. There is no workaround — the new API endpoints simply don't exist on 9.x.

Summary

Here's what we covered:

Architecture: Three layers — agent-plugins, sonarqube-cli, and mcp-server — each with a distinct role
The version trap: SonarQube Server must be 10.x or later; 9.9 LTS is not supported
Full installation: Plugin marketplace → integration wizard → auth → MCP registration
Corporate network setup: Internal mirror proxy for Docker images + Podman as a Docker Desktop replacement on macOS
Daily workflow: File scanning, quality gates, coverage, and fix commands

With this setup in place, Claude Code isn't just helping you write code faster — it's also helping you write code that passes quality gates before it ever hits CI. That's what AI-assisted development should actually look like.

References

SonarSource/sonarqube-agent-plugins — Official Claude Code plugin repository
SonarQube MCP Server — MCP Server implementation
Podman Installation Guide

Questions or issues? Drop a comment below.