DEV Community: Yu-Chen, Lin

How I Improved RAG Accuracy from 73% to 100% - A Chunking Strategy Comparison

Yu-Chen, Lin — Tue, 30 Dec 2025 16:04:38 +0000

Introduction

When building RAG (Retrieval-Augmented Generation) systems, you often hit a wall where accuracy just won't improve no matter what you try.

In this article, I'll share how I improved answer accuracy from 73.3% to 100% in a RAG system for internal company policy documents. Among the various chunking strategies tested, surprisingly, the simplest solution turned out to be the most effective.

I also discovered that adding Re-ranking, which I expected to improve accuracy, actually made it worse. I'll explain why this happened.

Project Architecture

Tech Stack

Layer	Technology
Frontend	Next.js 16 + React 19 + TypeScript
Backend	FastAPI + Python 3.12
Embedding Model	intfloat/multilingual-e5-large
Vector DB	Chroma
LLM	Google Gemini 2.0 Flash
Deployment	Vercel (Frontend) + Hugging Face Spaces (Backend)

System Architecture

User Query
    │
    ▼
┌─────────────┐
│  Next.js    │  Frontend (SSE streaming support)
└─────────────┘
    │
    ▼
┌─────────────┐
│  FastAPI    │  Backend
└─────────────┘
    │
    ├─── Embedding ─── multilingual-e5-large
    │
    ├─── Vector Search ─── Chroma DB
    │
    └─── LLM ─── Gemini 2.0 Flash
                    │
                    ▼
            Streaming Response

User queries are first vectorized by the embedding model, then similar chunks are retrieved from Chroma DB. These chunks are passed as context to the LLM, which generates the answer.

Common RAG Failure Patterns

When RAG accuracy is low, the problem is usually not the LLM's capability, but the retrieval step. Here are the specific patterns that caused issues in this project.

1. Implicit Reference Problem

This occurs when terminology in documents doesn't match the terms users actually use.

Example:

Document: "Employees defined in Article 2-2 shall have a maximum allowance of 20,000 yen per month"
User query: "What's the commuting allowance limit for part-time workers?"

The fact that "employees defined in Article 2-2" refers to "part-time workers" isn't clear unless you read the entire document. Vector search prioritizes chunks containing "part-time workers," missing this critical exception clause.

2. Multi-hop Reasoning

This happens when answering requires information from multiple sources.

Example:

User query: "What's the difference between full-time and part-time employee wedding bonuses?"

To answer this, you need both the full-time bonus (50,000 yen) and the part-time bonus (10,000 yen). If these are in different parts of the document, only one may be retrieved, making it impossible to calculate the difference.

3. Buried Exception Clauses

Japanese company regulations typically follow this structure:

Articles 1-10: General rules (for all employees, high keyword density)
     ↓
Articles 11+: Special provisions (for specific employee types, lower keyword density)
     ↓
Supplementary Provisions: Additional rules (exception cases, lowest keyword density)

Vector search is influenced by keyword frequency, so general rules are retrieved while exception clauses get buried.

4. Negative/Exclusion Queries

Queries with negations like "cannot" or "not eligible for" are challenging for vector search.

Example:

"What benefits are part-time workers not eligible for?"

Vector search cannot directly handle logical operations (NOT, AND, OR), making it difficult to retrieve appropriate information for such queries.

Designing a Dataset That Intentionally Fails

To validate RAG improvements, I created a dataset that intentionally includes the problems described above.

Design Principles

Mimicked actual Japanese company regulation structure
- General rules → Special provisions → Supplementary provisions pattern
- 6 documents total (commuting allowance, leave, expenses, remote work, conduct, benefits)
Multiple terms for the same concept (alias problem)
| User's term | Document terms |
|-------------|----------------|
| Part-timer | Short-term employee, Person defined in Article 2-2 |
| Part-time | Part-time employee, Person working 4+ days per week |
Exception clauses intentionally placed far from general rules

Concrete Example: Commuting Allowance Policy

# Commuting Allowance Policy

## Article 3 (Payment of Commuting Allowance)
The maximum commuting allowance shall be 50,000 yen per month.
← General rule (keyword "commuting allowance" appears frequently)

...(about 50 lines later)...

## Article 12 (Special Provisions for Payment)
For persons defined in Article 2-2, the maximum shall be 20,000 yen per month,
and commuter passes shall not be provided; payment shall be prorated based on actual work days.
← Exception clause (no "part-timer" keyword)

When a user asks "What's the commuting allowance limit for part-timers?":

Vector search retrieves chunks around Article 3 containing "commuting allowance"
The exception in Article 12 (actual answer: 20,000 yen) is not retrieved
LLM gives the wrong answer: "50,000 yen"

Evaluation Queries

I created 15 evaluation queries, each with "required keywords" and "prohibited keywords."

{
  "question": "What is the maximum commuting allowance for part-timers?",
  "required_keywords": ["20,000 yen"],
  "prohibited_keywords": ["50,000 yen"]
}

If "50,000 yen" (full-time limit) appears in the answer, it's incorrect. If "20,000 yen" (part-time limit) appears, it's correct. This enables automated evaluation.

Chunking Strategies Tested

1. Standard Chunking (Baseline)

The most basic chunking strategy.

# Configuration
chunk_size = 1000  # characters
overlap = 200      # characters

Result: 73.3% (11/15 correct)

With 1000-character chunks, general rules and exception clauses ended up in separate chunks, causing failures on exception-related questions.

2. Large Chunking

A simple improvement by increasing chunk size.

# Configuration
chunk_size = 2000  # characters
overlap = 500      # characters

Result: 100% (15/15 correct)

By using 2000-character chunks, general rules and exception clauses more often ended up in the same chunk, achieving perfect accuracy.

3. Parent-Child Chunking

A strategy that separates small chunks for search (children) from large chunks for LLM context (parents).

# Configuration
child_chunk_size = 400   # for search
parent_chunk_size = 2000 # for LLM context

How it works:

Perform vector search with small child chunks (400 chars)
Pass the parent chunk (2000 chars) associated with the hit to the LLM

Result: 93.3% (14/15 correct)

This enabled more detailed and comprehensive answers, but failed on one comparison query ("difference between full-time and part-time"). This was because the two amounts were in different parent chunks.

4. Hypothetical Questions

A strategy that pre-generates expected questions for each chunk using an LLM.

# During index creation
for chunk in chunks:
    questions = llm.generate(f"Generate 3 questions a user might ask about this content:\n{chunk}")
    # Vectorize and index the generated questions

Example:

Original chunk: "For persons defined in Article 2-2, the maximum shall be 20,000 yen per month"
Generated question: "What is the maximum commuting allowance for part-timers?"

Result: 93.3% (14/15 correct)

This was effective for solving the alias problem ("persons defined in Article 2-2" ≈ "part-timers"), but the downside is increased LLM API costs.

5. Re-ranking (Surprising Result)

A strategy that re-ranks initial search results using a Cross-Encoder.

# Configuration
initial_k = 10  # retrieve 10 in initial search
final_k = 4     # narrow to top 4 with Cross-Encoder
model = "cross-encoder/ms-marco-MiniLM-L-6-v2"

Result: 60.0% (9/15 correct) ← Worse than Standard alone

This was unexpected. Adding Re-ranking actually decreased accuracy.

Analysis of Results

Results Summary

Strategy	Accuracy	Implementation Complexity
Standard (1000/200)	73.3%	Low
Large (2000/500)	100%	Low
Parent-Child	93.3%	Medium
Hypothetical Questions	93.3%	High
Standard + Reranking	60.0%	Medium

Why Large Chunk Was the Winner

In this dataset, exception clauses were located within 300-500 characters of general rules. With 2000-character chunks, both could often be included in the same chunk.

This is a "simple solution wins" result, but there are caveats:

This result is dataset-dependent
If exception clauses are more than 1000 characters away, Large Chunk will also fail
Chunks that are too large may reduce search precision

Why Re-ranking Failed

Re-ranking decreased accuracy because of the difference between Precision and Recall.

Re-ranking's role:
  Select the most relevant 4 from 10 retrieved results
  → A tool for improving Precision

The actual problem:
  Exception clauses weren't in the 10 results to begin with
  → A Recall problem

Conclusion:
  Information not retrieved in the initial search
  cannot be rescued by Re-ranking

Re-ranking is a tool for "removing noise," not for "finding missed information." Since the core problem was retrieval misses (insufficient Recall), Re-ranking had no effect.

In fact, relevant chunks that barely made it into the lower ranks of the initial search were likely eliminated by Re-ranking, causing accuracy to decrease.

Future Improvements

Improvements I Want to Implement

1. Header-based Chunking

Instead of fixed-length chunks, split by Markdown headings (##, ###).

## Article 3 (Payment of Commuting Allowance)
...content...

## Article 4 (Application Method)  ← Split here
...content...

This creates chunks that preserve semantic coherence.

2. Query Expansion

Convert user queries to match document terminology before searching.

User: "What's the commuting allowance for part-timers?"
     ↓ Expand with LLM
Expanded: "part-timer short-term employee Article 2-2 commuting allowance transportation"

3. Increase Retrieval Count

Simply increasing k raises the probability of including exception clauses.

# Before
k = 4

# After
k = 10

However, increasing k too much lengthens the context and increases LLM processing costs.

Data Quality Improvements

There are limits to improving chunking strategies. A more fundamental solution is to preprocess the documents themselves.

Before: commuting_allowance_policy.md (for all employees, exceptions scattered)

After:
  - commuting_allowance_policy_fulltime.md
  - commuting_allowance_policy_parttime.md
  - commuting_allowance_policy_temporary.md

By splitting documents by employee type and rewriting from each perspective, we can fundamentally eliminate the exception clause problem. This approach achieved 93.3% accuracy.

Things I Want to Test

Semantic Chunking: Automatically detecting meaning boundaries for chunking
GraphRAG: RAG using knowledge graphs
Fine-tuned Embedding Models: Domain-specific embedding models

Conclusion

This article presented validation results for RAG accuracy improvement using real data.

Key Findings

The simplest solution was the most effective
- Large Chunk (2000 chars) achieved 100%
- More effective than complex Parent-Child or Hypothetical Questions
Re-ranking is not a silver bullet
- Has no effect on Recall (retrieval coverage) problems
- Can actually decrease accuracy in some cases
Data quality matters most
- There are limits to chunking strategy improvements
- Document preprocessing can be a fundamental solution

Key Takeaway for RAG Accuracy Improvement

"The key to RAG accuracy improvement is not LLM optimization, but data quality and retrieval accuracy."

First, analyze what kinds of retrieval failures are occurring with your dataset. Then, start with the simplest solutions.

References

Technical Stack Details

Embedding Model: intfloat/multilingual-e5-large
Re-ranking: cross-encoder/ms-marco-MiniLM-L-6-v2
Vector DB: Chroma
LLM: Gemini 2.0 Flash

Stop Reinventing the Wheel: forge-npm-pkg CLI Tool for Best-Practice npm Package Development

Yu-Chen, Lin — Sun, 07 Dec 2025 12:41:11 +0000

Introduction

Every time you create a new npm package, do you find yourself doing this?

// Copy-pasting the same config again...
const tsconfig = {
  compilerOptions: {
    target: "ES2022",
    module: "NodeNext",
    strict: true,
    // Grabbed this from my last project, but is it still best practice?
  }
};

package.json exports configuration is way too complex

ESM? CommonJS? Dual? Which one should I use?

Writing GitHub Actions CI/CD from scratch every time

Setting up Dependabot manually again

That template from 6 months ago is already outdated...

To solve this "reinventing the wheel every time" problem, I created forge-npm-pkg - a CLI tool that generates npm packages with always up-to-date best practices.

npx forge-npm-pkg my-awesome-package

One command gives you an npm package based on the latest best practices.

📦 NPM: https://www.npmjs.com/package/forge-npm-pkg

5 Problems forge-npm-pkg Solves

Problem	Traditional Approach	forge-npm-pkg Solution
Dependency versions	Hardcoded in templates → outdated in 6 months	Dynamically fetched from npm registry
Node.js LTS version	Manually update engines field	Auto-detected from Node.js official API
package.json exports	Wrestling with documentation	Perfectly generated for ESM/CommonJS/Dual
CI/CD configuration	Hand-written every time, tribal knowledge	Fully equipped GitHub Actions generated
Quality inconsistency	Different settings per project	Standardized best practices

How Dynamic Version Fetching Works

Let's look at the problem with traditional template tools.

❌ Traditional Templates

// template/package.json
const dependencies = {
  "typescript": "5.3.0",  // ← Version fixed at creation time
  "vitest": "1.0.0",      // ← Outdated in 6 months
  "tsup": "8.0.0",        // ← Becomes stale without maintenance
}

✅ forge-npm-pkg's Approach

async function fetchLatestVersion(packageName: string): Promise<string> {
  const response = await fetch(
    `https://registry.npmjs.org/${packageName}/latest`
  );
  const data = await response.json();
  return data.version;
}

async function fetchLatestVersions(packages: string[]): Promise<Record<string, string>> {
  const results = await Promise.all(
    packages.map(async (pkg) => [pkg, await fetchLatestVersion(pkg)])
  );
  return Object.fromEntries(results);
}

💡 By dynamically fetching the latest versions from npm registry, no template maintenance is required. You always start with the latest dependencies, no matter when you use the tool.

Additionally, it warns about versions released within the last 30 days:

⚠ typescript@5.7.0 was released only 15 days ago.
  Consider using 5.6.x for stability.

The design balances cutting-edge features with stability.

Automatic Node.js LTS Detection

Managing the Node.js engines field and CI/CD matrix manually is tedious, right?

❌ Traditional Method

{
  "engines": { "node": ">=18.0.0" }
}

↑ You won't notice when Node.js 18 reaches EOL...

✅ forge-npm-pkg's Method

async function fetchNodeLTSVersions(): Promise<string[]> {
  const response = await fetch('https://nodejs.org/dist/index.json');
  const releases = await response.json();

  return releases
    .filter((r: any) => r.lts)  // Extract only LTS versions
    .map((r: any) => r.version.replace('v', '').split('.')[0])
    .slice(0, 2);  // Latest 2 LTS versions (e.g., [22, 20])
}

It fetches current LTS versions from the official Node.js API and automatically configures the engines field and GitHub Actions matrix.

Perfect package.json exports Configuration

Are you confident your package.json exports field is correct?

forge-npm-pkg generates perfect exports configuration for all three module formats: ESM, CommonJS, and Dual.

ESM (Modern)

{
  "type": "module",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/index.js"
    }
  }
}

Dual (ESM + CommonJS)

{
  "type": "module",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/index.mjs",
      "require": "./dist/index.cjs"
    }
  }
}

It also auto-generates a validation script using @arethetypeswrong/cli:

npm run check:exports  # Validates exports configuration

📌 @arethetypeswrong/cli is a tool that checks whether your package's exports configuration correctly resolves TypeScript types. It catches ESM/CJS configuration mistakes early.

Fully Equipped CI/CD

Let's look at the generated GitHub Actions workflows.

ci.yml - Automated Testing on Pull Requests

jobs:
  test:
    strategy:
      matrix:
        node-version: [20.x, 22.x]  # ← Auto-detected LTS
    steps:
      - run: npm test
      - run: npm run typecheck
      - run: npm run lint
      - run: npm run build

publish.yml - Auto-publish on Tag Push

on:
  push:
    tags: ['v*']  # Triggers on v1.0.0 format tags

jobs:
  publish:
    steps:
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

dependabot-auto-merge.yml - Auto-merge Dependencies

# patch/minor → auto-merge
# major → manual review required

💡 GitHub Actions versions (like actions/checkout@v4) are also fetched from the GitHub API to ensure the latest versions.

async function fetchActionVersion(action: string): Promise<string> {
  const [owner, repo] = action.split('/');
  const response = await fetch(
    `https://api.github.com/repos/${owner}/${repo}/releases/latest`
  );
  const data = await response.json();
  return data.tag_name;  // e.g., "v4"
}

Interactive Configuration

◆ What language do you want to use?
│ ○ TypeScript (Recommended)
│ ○ JavaScript

◆ What module format do you want to use?
│ ○ ESM (Modern)
│ ○ CommonJS (Legacy)
│ ○ Dual (ESM + CommonJS)

◆ What test runner do you want to use?
│ ○ Vitest (Recommended)
│ ○ Jest
│ ○ None

◆ Enable ESLint + Prettier?
│ ● Yes / ○ No

◆ Initialize git repository?
│ ● Yes / ○ No

◆ Setup GitHub Actions CI/CD?
│ ● Yes / ○ No

Flexibly customize based on your project requirements.

Generated Project Structure

my-awesome-package/
├── src/
│   ├── index.ts
│   └── index.test.ts
├── .github/
│   ├── workflows/
│   │   ├── ci.yml
│   │   ├── publish.yml
│   │   └── dependabot-auto-merge.yml
│   └── dependabot.yml
├── package.json          # Perfect exports configuration
├── tsconfig.json         # Strict mode enabled
├── tsup.config.ts        # Build configuration
├── vitest.config.ts      # Test configuration
├── eslint.config.js      # ESLint flat config
├── .prettierrc
├── .gitignore
└── README.md

All files work together in harmony. No need to worry about consistency issues from copy-pasting individual configurations.

Usage

No Installation Required (npx)

npx forge-npm-pkg my-package-name

Global Installation

npm install -g forge-npm-pkg
forge-npm-pkg my-package-name

Create GitHub Repository Simultaneously

# If gh CLI is installed
forge-npm-pkg my-package-name --github

Design Philosophy: The Power of Standardization

I have experience standardizing development processes for teams at work:

Task management with Epic/Story/Task structure

PR-based development workflow

Automatic versioning with semantic-release

What I learned from this experience:

When you "systematize" good practices, the entire team's quality improves

forge-npm-pkg applies the same concept to "npm package development."

Define the "correct configuration" once, and anyone can create best-in-class packages anytime

This isn't just about individual productivity—it leads to quality improvement across teams and organizations.

Potential for Broader Application

This approach can be applied to other areas:

Domain	What Can Be Standardized
RAG Projects	Vector DB settings, chunking strategies, prompt templates
Next.js Projects	Auth configuration, API structure, deployment settings
AWS CDK	Security settings, tagging rules, monitoring

"Repeating the same thing every time" → "Templatize to ensure standards"

This pattern can be leveraged across various development domains.

Conclusion

forge-npm-pkg is a tool that makes "npm package development best practices" accessible to everyone.

✅ 5 Problems Solved

Problem	Solution
Writing the same config every time	Generate everything with 1 command
Outdated dependency versions	Dynamic fetching from npm registry
Complex package.json exports	Perfect auto-generated configuration
Tribal knowledge in CI/CD	Fully equipped GitHub Actions
Quality inconsistency	Standardized best practices

✅ Technical Highlights

Dynamic version fetching from npm registry API

Auto-detection of LTS from official Node.js API

Actions version fetching from GitHub API

Warnings for too-new versions

If you find yourself "repeating the same setup every time you start npm package development," give it a try.

npx forge-npm-pkg my-awesome-package

📦 NPM: https://www.npmjs.com/package/forge-npm-pkg
📂 GitHub: https://github.com/your-username/forge-npm-pkg

Feedback and contributions are welcome!

How I Built a RAG App from Scratch to Production in 5 Days

Yu-Chen, Lin — Tue, 02 Dec 2025 12:46:57 +0000

Introduction

"I hear about RAG a lot lately, but how do you actually build one?"

This 5-day development log started from that very question. I will share my learnings from having zero knowledge to completing a RAG application running in a production environment.

Completed App: https://github.com/oharu121/rag-demo

What I Built

I built a RAG application that allows users to ask questions about internal documentation using natural language.

Key Features:

Streaming responses like ChatGPT
Document upload and management
Citation display for answers (with filenames and line numbers)
Multi-turn conversation

Tech Stack:

Layer	Technology	Reason for Selection
Frontend	Next.js 16 / React 19	App Router + RSC support
Backend	FastAPI	Asynchronous processing & Type safety
Embedding Model	multilingual-e5-large	High accuracy for Japanese text
Vector DB	Chroma	Low learning curve & OSS
LLM	Gemini 2.0 Flash	Free tier available
Deployment	Vercel + HF Spaces	Production-ready for free

Day 1: Understanding the Basics of RAG

What is RAG?

RAG (Retrieval-Augmented Generation) = Retrieval + Generation

User Question
    ↓
[Retrieval] Fetch relevant documents from Vector DB
    ↓
[Generation] Ask LLM: "Please answer using these materials"
    ↓
Answer with evidence

Why is RAG Necessary?

Problems with standalone LLMs:

Knowledge Cutoff: They don't know information past their training date.
No Access to Private Data: The LLM has never seen your company's documents.
Hallucinations: They confidently tell lies.

RAG solves these issues by "reinforcing with search."

The Importance of Chunking

The first thing I stumbled upon was chunking (text splitting).

The problem with chunk_overlap = 0:

"Mr. Tanaka lives in Tokyo. He is | an engineer."
                          ↑ If split here
                          It becomes unclear who "He" refers to.

Lesson: A chunk_overlap of 10-20% of the chunk_size is recommended to prevent context fragmentation.

How Embeddings Work

I wondered, "Why can't we just use simple character code conversion?"

# Naive approach - Cannot capture meaning
"cat" -> [99, 97, 116]
"dog" -> [100, 111, 103]
# "cat" and "feline" would end up in completely unrelated positions!

# Embedding Model - Captures meaning
"cat"      -> [0.82, 0.15, -0.34, ...]
"feline"   -> [0.79, 0.18, -0.31, ...]  # Close!
"airplane" -> [-0.45, 0.67, 0.12, ...]  # Far

Embedding models are trained so that "words used in similar contexts result in similar vectors."

Day 2-3: Backend Implementation

Implementing Streaming API with FastAPI

To achieve the "text flowing in" experience like ChatGPT, I adopted Server-Sent Events (SSE).

@router.post("/chat")
async def chat(request: ChatRequest):
    async def generate():
        # 1. Search related documents
        docs = vector_store.similarity_search(request.message)

        # 2. Send source info first
        yield f"event: sources\ndata: {json.dumps(sources)}\n\n"

        # 3. Stream from LLM token by token
        async for token in llm.stream(prompt):
            yield f"event: token\ndata: {json.dumps({'token': token})}\n\n"

        yield f"event: done\ndata: {{}}\n\n"

    return StreamingResponse(generate(), media_type="text/event-stream")

The FastAPI 404 Issue in Production

When I added a new endpoint, it kept returning a 404 error.

GET /api/documents/abc123/content
→ {"detail": "Not Found"}  # FastAPI's default 404

Since it was FastAPI's default 404 and not my custom error (HTTPException(404, "Document not found")), it meant the route wasn't matching at all.

Debugging Method:

# 1. Add a debug endpoint
@router.get("/debug/routes")
async def debug_routes():
    return {"routes": [...]}

# 2. Print route list at startup
for route in app.routes:
    print(f"{route.methods} {route.path}")

It turned out the code just hadn't been reflected during deployment.

Lesson: When debugging in production, first verify "is the code actually deployed?"

Day 4: Frontend Implementation

Receiving SSE Streaming

export async function* streamChat(message: string, history: Message[]) {
  const response = await fetch(`${API_URL}/chat`, {
    method: "POST",
    body: JSON.stringify({ message, history }),
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });

    // Parse SSE events
    const lines = buffer.split("\n");
    for (const line of lines) {
      if (line.startsWith("event: ")) {
        // Event processing...
        yield { type: eventType, data };
      }
    }
  }
}

The React Stale Closure Problem

This was the bug where I learned the most.

Symptom: The AI answers based on old messages.

1. "Tell me about the vacation policy" → Answers correctly
2. "Tell me about the branch office" → Somehow returns the answer about the vacation policy

Cause: Stale closure in useCallback.

// Buggy code
const sendMessage = useCallback(async (content: string) => {
  // This `messages` is the value from when useCallback was created!
  const history = messages.map(m => ({ role: m.role, content: m.content }));

  await streamChat(content, history); // Sends old history
}, [messages]); // Even with the dependency array, timing issues occur

Solution: Get the current state using setState functional updates.

// Fixed code
const sendMessage = useCallback(async (content: string) => {
  let capturedHistory: Message[] = [];

  // Get current state via functional update
  setMessages((prev) => {
    capturedHistory = prev.map(m => ({
      role: m.role,
      content: m.content
    }));
    return [...prev, userMessage, assistantMessage];
  });

  // capturedHistory is guaranteed to be latest
  await streamChat(content, capturedHistory);
}, []); // Dependency array can be empty

Lesson: If you need the latest state within an async callback:

Use useRef to synchronize state
Utilize setState functional updates

Day 5: UX Improvements

Document Preview Feature

I added a preview feature so users can check "which documents are being targeted for search."

// DocumentChipsBar - List of documents always displayed
<DocumentChipsBar
  documents={allDocuments}
  onPreview={(doc) => setPreviewDoc(doc)}
/>

// Modal display on click
<DocumentPreviewModal
  doc={previewDoc}
  onClose={() => setPreviewDoc(null)}
/>

Onboarding Flow

For first-time visitors, I displayed step-by-step hints:

Tooltip pointing to the "Manage Documents" button.
"Click to preview" hint pointing to the document chips.

// Tooltip position calculation
useEffect(() => {
  if (targetRef.current) {
    const rect = targetRef.current.getBoundingClientRect();
    setPosition({
      top: rect.bottom + 12,
      right: window.innerWidth - rect.right,
    });
  }
}, [targetRef]);

Project Structure

simple-rag-app/
├── frontend/              # Next.js Frontend
│   ├── app/
│   │   ├── components/    # UI Components
│   │   ├── page.tsx
│   │   └── layout.tsx
│   ├── lib/               # API, Constants, Type definitions
│   ├── hooks/             # Custom Hooks (useChat, useDocuments, etc.)
│   └── package.json
│
├── backend/               # Python Backend
│   ├── app/
│   │   ├── routers/       # API Endpoints (chat, documents)
│   │   ├── services/      # Business Logic (RAG, Document Management)
│   │   ├── models/        # Pydantic Schemas
│   │   └── utils/         # Rate limiting, Error handling
│   ├── pyproject.toml
│   └── Dockerfile
│
└── README.md

Why Separate Frontend and Backend?

I considered a monolithic structure (doing everything with Next.js API Routes), but separated them for the following reasons:

Leveraging the Python Ecosystem: AI/ML libraries like LangChain, Chroma, and sentence-transformers are overwhelmingly richer in Python.
Asynchronous Processing: FastAPI's async/await works very well with streaming responses.
Deployment Flexibility: I can choose the optimal platform for both frontend and backend.

Cost Efficiency: Strategy for Free Production

For personal development or learning purposes, keeping costs to zero is crucial. I achieved a completely free production environment with the following configuration:

Deployment Selection

Component	Deployment	Free Tier	Reason for Selection
Frontend	Vercel	100GB bandwidth/mo	Creators of Next.js. Zero-config deployment.
Backend	HF Spaces	2vCPU, 16GB RAM	Docker support. Easy installation of ML libraries.
LLM	Gemini API	15 RPM, 1M tokens/day	Most generous free tier.
Vector DB	Chroma (Inside HF)	-	Persisted within HF Spaces.

Why Vercel + Hugging Face Spaces?

Vercel (Frontend):

Since they make Next.js, it has perfect compatibility with the latest features like App Router and RSC.
Automatic deployment just by pushing to GitHub.
Fast delivery via Edge Network.
Commercial use allowed on the free tier.

Hugging Face Spaces (Backend):

Docker support allows handling complex dependencies.
Pre-installed environment for ML/AI libraries (PyTorch, sentence-transformers, etc.).
16GB RAM allows embedding models to run comfortably.
Safe environment variable management via Secrets.
There is a cold start issue, but it's acceptable for demo use.

Cost Comparison: LLM Selection

In RAG apps, LLM API fees tend to be the biggest cost.

Model	Input (1M tokens)	Output (1M tokens)	Feature
Gemini Flash	$0.075	$0.30	Free tier available
GPT-4o mini	$0.15	$0.60	Good balance
Claude Haiku	$0.25	$1.25	Good at Japanese
GPT-4o	$2.50	$10.00	High precision

Result: Selected Gemini Flash (Free Tier) for learning and demo purposes.

Summary of Learnings

Technical Takeaways

Chunking: Overlap is mandatory. 10-20% of chunk_size.
Vector DB: Chroma is sufficient for learning. Consider Pinecone, etc., for production.
Streaming: Implemented with SSE. Contributes significantly to UX.
React Closures: In async processing, use useRef or functional setState.

Architectural Takeaways

[Vercel]          [HF Spaces]        [Chroma]
Next.js    <--->    FastAPI    <--->  Vector DB
Frontend            Backend           Persistence
   ↑                   ↑
   └──── SSE ──────────┘

Separating frontend and backend allows for:

Independent scaling
Deployment to the optimal platform for each
Maximizing the use of free tiers

Process Takeaways

Debug Endpoints: Prepare verification endpoints like /debug/routes in production.
Incremental Logging: Essential for isolating issues.
Deployment Verification: First check "Is the code actually deployed?"

Conclusion

In 5 days, I was able to experience everything from the basics of RAG to production deployment.

Repository: https://github.com/oharu121/rag-demo

What made the biggest impression on me was:

The Power of RAG: Realizing "answers with evidence," which is impossible with an LLM alone.
Stale Closures: Be very careful with React async processing.
Free Production: Achievable with Vercel + HF Spaces + Gemini Free Tier.

RAG is a technology that makes "LLMs usable for actual work." I hope this demo app serves as a useful reference for your learning.

References

Stop Struggling with Axios! My First NPM Package "axios-fluent" Solves 3 Major Pain Points

Yu-Chen, Lin — Mon, 24 Nov 2025 02:54:21 +0000

Introduction

Do you experience this every time you write HTTP requests?

// Configuration is too complex...
const response = await axios.post('https://api.example.com/users', data, {
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  timeout: 5000,
  httpsAgent: new https.Agent({
    rejectUnauthorized: false  // Writing this for self-signed certificates every time...
  })
});
const user = response.data;  // Why do I need to write .data every single time?

// Error handling is a nightmare
try {
  await axios.get('/api/data');
} catch (error) {
  // error.response?.data?.message? error.response?.status? error.config?.url?
  // Always checking docs to find where things are...
  console.log('Status:', error.response?.status);
  console.log('Message:', error.response?.data?.message);
  console.log('URL:', error.config?.url);
}

To solve these developer frustrations, I created axios-fluent. This is my first published npm package, built from real pain points I experienced in production projects—every "annoying thing about Axios" packed into one practical library.

npm install axios-fluent

https://www.npmjs.com/package/axios-fluent

🎯 3 Problems axios-fluent Solves

1. Complex Request Building → Clean with Builder Style

Before (Plain Axios)

const config = {
  baseURL: 'https://api.example.com',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  timeout: 10000
};

const instance = axios.create(config);
const response = await instance.get('/users/1');
const user = response.data;

After (axios-fluent)

const client = Axon.new()
  .baseUrl('https://api.example.com')
  .bearer(token)
  .json()
  .timeout(10000);

const user = await client.get('/users/1').data();

💡 Technical Deep Dive: Immutable Builder Pattern

Each method in axios-fluent returns a new instance by design.

public bearer(token: string): Axon {
  const newConfig = {
    ...this.config,
    headers: { ...this.config.headers, Authorization: `Bearer ${token}` },
  };
  return new Axon(newConfig, this.instance);  // Returns new instance
}

This enables:

Natural method chaining
No mutation of original instance (no side effects)
Follows functional programming principles

2. Axios Errors Too Verbose → AxonError with Only Essential Info

Before (Plain Axios)

try {
  await axios.get('/api/users');
} catch (error) {
  // Information scattered across different properties...
  const status = error.response?.status;
  const message = error.response?.data?.message;
  const url = error.config?.url;
  const method = error.config?.method;

  console.log(`${method} ${url} failed with ${status}: ${message}`);
  // Writing this every time is exhausting
}

After (axios-fluent)

try {
  await client.get('/api/users');
} catch (error) {
  if (error instanceof AxonError) {
    console.log(error.toString());
    // ↓ Automatically formatted output
    // AxonError: Request failed with status code 404
    //   Request: GET https://api.example.com/api/users
    //   Status: 404 Not Found
    //   Response: {"error":"User not found"}
  }
}

💡 Technical Deep Dive: Only 5 Essential Properties

export class AxonError extends Error {
  public readonly status?: number;         // HTTP status code
  public readonly statusText?: string;     // Status text
  public readonly url?: string;            // Request URL
  public readonly method?: string;         // HTTP method
  public readonly responseData?: any;      // Response body

  public toString(): string {
    const parts = [`AxonError: ${this.message}`];
    if (this.method && this.url) {
      parts.push(`  Request: ${this.method} ${this.url}`);
    }
    if (this.status) {
      parts.push(`  Status: ${this.status} ${this.statusText || ''}`);
    }
    if (this.responseData) {
      parts.push(`  Response: ${JSON.stringify(this.responseData)}`);
    }
    return parts.join('\n');
  }
}

In v2.0.0, I removed unnecessary properties (headers, originalError) and focused on only the information you actually need for debugging.

3. Self-Signed Certificate Error Nightmare → allowInsecure Has Your Back

When using self-signed HTTPS certificates in development, you end up writing this every time:

Before (Plain Axios)

const https = require('https');

const instance = axios.create({
  httpsAgent: new https.Agent({
    rejectUnauthorized: false  // Copy-pasting this for every dev environment...
  })
});

After (axios-fluent)

// Convenient factory for development
const client = Axon.dev();

// Or explicitly specify
const client = Axon.new({ allowInsecure: true });

💡 Technical Deep Dive: Secure by Default, Easy for Development

public static new(options: Options = { allowInsecure: false }) {
  const config: AxiosRequestConfig = {};

  if (options?.allowInsecure) {
    config.httpsAgent = new https.Agent({
      rejectUnauthorized: false,  // Only disabled when explicitly opted-in
    });
  }

  return new Axon(config);
}

public static dev() {
  return Axon.new({ allowInsecure: true });  // Development shortcut
}

Security-First Design:

Default is certificate validation enabled (safe for production)
allowInsecure requires explicit opt-in
Axon.dev() is a convenience tool for development (purpose clear from name)

📦 Practical Usage

Building an API Client Wrapper

import { Axon } from 'axios-fluent';

class ApiClient {
  private client: Axon;

  constructor(baseUrl: string, token?: string) {
    this.client = Axon.new()
      .baseUrl(baseUrl)
      .json()
      .timeout(10000);

    if (token) {
      this.client = this.client.bearer(token);
    }
  }

  async getUser(id: number): Promise<User> {
    return await this.client.get<User>(`/users/${id}`).data();
  }

  async createUser(data: CreateUserDto): Promise<User> {
    return await this.client.post<User>('/users', data).data();
  }

  async uploadAvatar(userId: number, file: File): Promise<void> {
    const formData = new FormData();
    formData.append('avatar', file);

    await this.client
      .multipart()
      .post(`/users/${userId}/avatar`, formData);
  }
}

export const api = new ApiClient(
  process.env.API_URL || 'https://api.example.com',
  localStorage.getItem('authToken') || undefined
);

Response Convenience Methods Reduce Boilerplate

// Before: Writing .data every time
const response = await axios.get<User>('/users/1');
const user = response.data;
const status = response.status;

// After: Get exactly what you want directly
const user = await client.get<User>('/users/1').data();
const status = await client.get('/users/1').status();
const isOk = await client.post('/users', data).ok();  // Check if 2xx

💡 Technical Deep Dive: PromiseLike for Backward Compatibility

export class AxonResponse<T = any> implements PromiseLike<AxiosResponse<T>> {
  async data(): Promise<T> {
    const response = await this.responsePromise;
    return response.data;
  }

  async status(): Promise<number> {
    const response = await this.responsePromise;
    return response.status;
  }

  async ok(): Promise<boolean> {
    const response = await this.responsePromise;
    return response.status >= 200 && response.status < 300;
  }

  // Promise interface for backward compatibility
  then<TResult1 = AxiosResponse<T>, TResult2 = never>(
    onfulfilled?: ((value: AxiosResponse<T>) => TResult1 | PromiseLike<TResult1>) | null,
    onrejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | null
  ): PromiseLike<TResult1 | TResult2> {
    return this.responsePromise.then(onfulfilled, onrejected);
  }
}

This design adds new features without breaking existing code.

Error Handling Best Practices

import { AxonError } from 'axios-fluent';

async function fetchUserWithRetry(id: number, maxRetries = 3): Promise<User> {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await client.get<User>(`/users/${id}`).data();
    } catch (error) {
      if (error instanceof AxonError) {
        // Branch logic based on status code
        if (error.status === 404) {
          throw new Error(`User ${id} not found`);
        }

        if (error.status && error.status >= 500) {
          // Retry server errors
          if (attempt === maxRetries) {
            throw new Error(`Server error after ${maxRetries} attempts: ${error.toString()}`);
          }
          await sleep(1000 * attempt);  // Exponential backoff
          continue;
        }

        // Don't retry 4xx errors
        throw error;
      }

      // Non-AxonError (network errors, etc.)
      throw error;
    }
  }

  throw new Error('Unexpected error in retry logic');
}

Implementing Authentication Flow

class AuthService {
  private client: Axon;
  private refreshToken?: string;

  constructor(baseUrl: string) {
    this.client = Axon.new().baseUrl(baseUrl).json();
  }

  async login(email: string, password: string): Promise<void> {
    const response = await this.client
      .post<AuthResponse>('/auth/login', { email, password })
      .data();

    this.refreshToken = response.refreshToken;
    this.client = this.client.bearer(response.accessToken);
  }

  async refreshAccessToken(): Promise<void> {
    if (!this.refreshToken) {
      throw new Error('No refresh token available');
    }

    const response = await this.client
      .post<AuthResponse>('/auth/refresh', { refreshToken: this.refreshToken })
      .data();

    this.refreshToken = response.refreshToken;
    this.client = this.client.bearer(response.accessToken);
  }

  async fetchProtectedResource<T>(url: string): Promise<T> {
    try {
      return await this.client.get<T>(url).data();
    } catch (error) {
      if (error instanceof AxonError && error.status === 401) {
        // Token expired → Refresh and retry
        await this.refreshAccessToken();
        return await this.client.get<T>(url).data();
      }
      throw error;
    }
  }
}

🚀 Why You Need axios-fluent

1. Faster Development

Builder pattern makes request configuration intuitive
Response convenience methods reduce boilerplate code
Simplified error handling

2. Improved Code Readability

// Before: Deep nesting
const response = await axios.post(
  'https://api.example.com/users',
  data,
  {
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${token}`
    },
    timeout: 5000
  }
);

// After: Flat and readable
const user = await Axon.new()
  .baseUrl('https://api.example.com')
  .bearer(token)
  .json()
  .timeout(5000)
  .post('/users', data)
  .data();

3. Type Safety Maintained

interface User {
  id: number;
  name: string;
  email: string;
}

// Full TypeScript type inference
const user: User = await client.get<User>('/users/1').data();
//    ^? Inferred as User type

4. Production-Ready

Secure by default (HTTPS certificate validation ON)
Development convenience maintained (Axon.dev())
Systematic error handling
100% backward compatibility (doesn't break existing code)

📊 Performance & Bundle Size

axios-fluent is an Axios wrapper, so performance overhead is nearly zero.

Metric	Value
Package Size	~17KB (minified)
Dependencies	axios (peer), form-data
TypeScript Support	✅ Full support
Node.js Support	v20+

Summary

axios-fluent is my first published npm package, designed to solve pain points I felt in real development.

✅ 3 Major Improvements

Builder pattern makes request construction easy
AxonError makes error handling intuitive
allowInsecure frees you from development certificate errors

✅ Practical-Focused Design

Secure by default, convenient for development
100% backward compatibility (doesn't break existing code)
Maintains TypeScript type safety
Provides 6 practical code examples

If you've ever thought "Axios configuration is complex," "error handling is tedious," or "self-signed certificates always error out," give it a try.

npm install axios-fluent

📦 NPM: https://www.npmjs.com/package/axios-fluent
📂 GitHub: https://github.com/oharu121/axios-fluent

Feedback and contributions are welcome!

axios #npm

Building an AI Development Environment with Claude Code Claude Router Open Router

Yu-Chen, Lin — Sun, 17 Aug 2025 09:37:19 +0000

Introduction

Claude Code is a CLI & GUI tool specialized for developers, based on the AI assistant “Claude” provided by Anthropic.

It allows you to perform code completion, design reviews, and prompt optimization in your local environment, enhancing development support.

This article will cover:

How to install and launch Claude Code

How to configure Claude Code via Claude Router

Practical use cases (Prompt design with Lyra → AI implementation support)

Troubleshooting

Useful commands to improve development efficiency

anthropics / claude-code

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

Claude Code

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows -- all through natural language commands. Use it in your terminal, IDE, or tag @claude on Github.

Learn more in the official documentation.

Get started

Note

Installation via npm is deprecated. Use one of the recommended methods below.

For more installation options, uninstall steps, and troubleshooting, see the setup documentation.

Install Claude Code:

MacOS/Linux (Recommended):

curl -fsSL https://claude.ai/install.sh | bash

Homebrew (MacOS/Linux):

brew install --cask claude-code

Windows (Recommended):

irm https://claude.ai/install.ps1 | iex

WinGet (Windows):

winget install Anthropic.ClaudeCode

NPM (Deprecated):

npm install -g @anthropic-ai/claude-code

Navigate to your project directory and run claude.

Plugins

This repository includes several Claude Code plugins that extend functionality with custom commands and agents. See the plugins…

View on GitHub

musistudio / claude-code-router

Use Claude Code as the foundation for coding infrastructure, allowing you to decide how to interact with the model while enjoying updates from Anthropic.

This project is sponsored by Z.ai, supporting us with their GLM CODING PLAN.

GLM CODING PLAN is a subscription service designed for AI coding, starting at just $10/month. It provides access to their flagship GLM-4.7 & （GLM-5 Only Available for Pro Users）model across 10+ popular AI coding tools (Claude Code, Cline, Roo Code, etc.), offering developers top-tier, fast, and stable coding experiences.

Get 10% OFF GLM CODING PLAN：https://z.ai/subscribe?ic=8JVLJQFSKB

Progressive Disclosure of Agent Tools from the Perspective of CLI Tool Style

A powerful tool to route Claude Code requests to different models and customize any request.

✨ Features

Model Routing: Route requests to different models based on your needs (e.g., background tasks, thinking, long context).
Multi-Provider Support: Supports various model providers like OpenRouter, DeepSeek, Ollama, Gemini, Volcengine, and SiliconFlow.
Request/Response Transformation: Customize requests and responses for different providers using transformers.
Dynamic Model Switching: Switch models on-the-fly…

View on GitHub

Using Claude Code Alone

1. Install claude code

npm install -g @anthropic-ai/claude-code

2. Launch Claude Code

Run the following command in the command prompt:

claude

3. Initial Setup

Theme Selection (Light / Dark)

Account Linking (Claude subscription or Anthropic Console account)

Using Claude Code via Claude Router

1. Install Claude Router

npm install -g @musistudio/claude-code-router

2. Create a Config File

macOS / Linux: ~/.claude-code-router/config.json

Windows: C:\Users\<username>\config.json

{
  "Providers": [
    {
      "name": "openrouter",
      "api_base_url": "https://openrouter.ai/api/v1/chat/completions",
      "api_key": "sk-xxx",
      "models": [
        "anthropic/claude-sonnet-4"
      ],
      "transformer": {
        "use": ["openrouter"]
      }
    }
  ],
  "Router": {
    "default": "openrouter,anthropic/claude-sonnet-4"
  },
  "LOG": true,
  "HOST": "127.0.0.1"
}

Paste the API token generated in Open Router.

https://openrouter.ai/

3. Start Claude Router

Run ccr code

4. Troubleshooting

Service not running, starting service...

💡 Solution

Run ccr start

Error starting server: Error: listen EACCES: permission denied 127.0.0.1:3456

💡 Solution

Add PORT to the config file:

{
  "Providers": [
    {
      "name": "openrouter",
      "api_base_url": "https://openrouter.ai/api/v1/chat/completions",
      "api_key": "sk-xxx",
      "models": [
        "anthropic/claude-sonnet-4"
      ],
      "transformer": {
        "use": ["openrouter"]
      }
    }
  ],
  "Router": {
    "default": "openrouter,anthropic/claude-sonnet-4"
  },
  "LOG": true,
  "HOST": "127.0.0.1",
+ "PORT": 23456
}

After modifying the config, run ccr strop and then ccr code to apply the changes.

Example: Using as AI Design Assistance

Here we’ll show an example of combining Lyra Prompt Optimization with Claude Code to request system design.

1. Start Claude Router → Begin Project

Create a project folder (ai-guess-game)

Open command prompt in the project folder

Run ccr code

2. Create Instruction with Lyra

Input: “Build a drawing game with Next.js, and use Gemini API for AI to guess the drawing.”

Input Example

DETAIL using Claude —

## Project Overview

Create an online "Pictionary" game where players can draw on a canvas, and an AI system is responsible for guessing what the drawing represents.

## Technology Stack

* Framework: Next.js
* Use Gemini's API for AI, with API calls made from the backend
* Do not use any SDKs; call the Gemini API endpoint directly

Lyra generates an optimized prompt and passes it to Claude Code

Result

**Your Optimized Prompt:**

You're an expert AI product architect and full-stack developer.

I'm building an online "Pictionary"-style web app using **Next.js** where players draw on a canvas and an AI guesses what the drawing represents.

**Your mission:**
Help me design the core architecture and implementation flow for this app. I will use **Gemini's API directly from the backend (no SDKs)** to send user drawings to the AI.

Please provide:

1. **System Architecture**  
   - How to structure the frontend and backend in Next.js  
   - Where and how to process the drawing data for API calls  
   - How to securely call Gemini API directly without an SDK

2. **AI Integration Plan**  
   - How to convert the user's canvas drawing into an appropriate format (image or strokes)  
   - Best format to send to Gemini for accurate interpretation  
   - API request/response structure examples

3. **Gameplay Flow**  
   - Steps from player drawing to AI making a guess  
   - Whether to guess after each stroke or on demand (with pros/cons)  
   - Optional: Add top-N guess logic with confidence scores

4. **Security &amp; Performance Tips**  
   - How to protect Gemini API key  
   - How to minimize latency for real-time interaction

Do not suggest using Gemini SDKs. All API calls should be handled manually.

Start by outlining the architecture, then walk through each step of the AI interaction and gameplay logic.

https://qiita.com/oharu121/items/cfe9d8b3850846a64d14

3. Claude Code Proposes the Design

Architecture structure

API integration flow

Gameplay steps

Security & performance measures

4. Code Generation and Execution Result

Enter implement with the above plan following the design

Implementation code is generated, and the drawn “pig” was correctly recognized as “pig head”

This time the top-up was 4 USD, but only 0.87 USD was used

5. Confirming the Results

https://github.com/oharu121/ai-guess-game

I drew a pig, and it correctly recognized it as “pig head”!

How to Use Claude Code

Claude Code isn’t just for chat—it also provides the following prompt design support features:

1. Handy Commands in Claude Code

Command	Description
`/init`	Summarizes project info into `CLAUDE.md`
`/compact`	Reduces token consumption via context compaction
`/clear`	Resets conversation history
`alt + m`	Mode switching (Manual approval / Auto approval / Plan mode)

2. VS Code Integration

Install the Claude Code extension in VS Code

Run /ide in Claude Code and select Visual Studio Code

Edit diffs are visualized, improving code review efficiency

Conclusion

Now you’re ready to run Claude Code locally and use it in combination with Claude Router to support your development. Next, try it out in an actual development project for design reviews, code generation, and improvement suggestions.