DEV Community: Daichi

Building a RAG-Based Subsidy Matching System from Scratch with Python

Daichi — Sat, 28 Mar 2026 10:50:41 +0000

What I Built

A RAG (Retrieval-Augmented Generation) system that helps Japanese small businesses find government subsidies. Users describe their business situation, and the system retrieves relevant subsidies from a vector database, then generates a detailed answer using Claude API.

Why RAG Instead of Just Using an LLM?

LLMs alone have three problems for this use case:

Hallucination — They confidently make up subsidy details that don't exist
Stale data — Subsidy information updates frequently; LLM training data can't keep up
No sources — Users need to verify the information, but LLMs don't cite where it came from

RAG solves all three by retrieving real data first, then passing it to the LLM as context. The LLM generates answers grounded in actual documents, not its training data.

Architecture

User Query
    │
    ▼
┌─────────────┐     ┌──────────────┐
│  Retriever  │────▶│  Generator   │
│ (embed +    │     │ (Claude API) │
│  search)    │     └──────┬───────┘
└──────┬──────┘            │
       │                   ▼
┌──────▼──────┐      AI Answer
│  ChromaDB   │      with sources
│ (Vector DB) │
└─────────────┘

The data ingestion pipeline:

JSON Data → Sentence Chunker → Embedding Model → ChromaDB

Tech Stack and Why

Layer	Choice	Why
Embedding	multilingual-e5-large	Best multilingual performance for free. Runs locally — no data leaves the machine
Vector DB	ChromaDB	Zero config. `pip install` and it works with local persistence
LLM	Claude API	Strong long-context handling and Japanese language quality
Frontend	Streamlit	Full UI in ~40 lines of Python

Why Not LangChain/LlamaIndex?

I intentionally built the pipeline from scratch first. Frameworks abstract away the internals, which is great for production but bad for learning. By writing each step manually — chunking, embedding, storing, retrieving, prompting — I understood exactly what each piece does and why it matters.

Key Implementation Details

1. Sentence-Boundary Chunking

The most impactful decision was how to split documents into chunks.

Naive approach (fixed-size split):

...eligible IT tools include accounting software, ord  ← cut mid-sentence
er management software, payment software...           ← next chunk

My approach (split at Japanese sentence boundaries 。):

sentences = text.split("。")
# Greedily pack sentences into chunks up to CHUNK_SIZE
# Overlap the tail for context continuity

This preserves semantic meaning in each chunk, which directly improves retrieval accuracy.

2. Embedding with Prefix

multilingual-e5 requires specific prefixes to distinguish between stored passages and search queries:

# When indexing documents
prefixed = [f"passage: {text}" for text in texts]

# When searching
query_embedding = model.encode(f"query: {user_query}")

Missing this prefix is a common mistake that silently degrades search quality.

3. Prompt Design for Grounded Answers

The generation prompt explicitly constrains the LLM:

prompt = """You are a Japanese subsidy advisor.
Answer based on the reference information below.
If information is insufficient, state that explicitly.
For critical details like deadlines and amounts, be precise."""

This reduces hallucination by telling the model to stick to the provided context.

Evaluation: Measuring What Matters

Building a RAG that "seems to work" isn't enough. I created a test suite with 10 queries, each with an expected correct subsidy:

Query	Expected Result	Rank
"Want to introduce accounting software"	Digital/AI Subsidy	1st
"Want to install new manufacturing equipment"	Manufacturing Subsidy	2nd
"Small shop wanting to attract customers via flyers"	Small Business Sustainability Subsidy	1st
"Restaurant wanting to start EC business"	New Business Advancement Subsidy	3rd
"Want to convert part-timers to full-time"	Career Up Grant	1st
...	...	...

Results:

Metric	Score
Hit Rate @5	100% (10/10)
MRR @5	0.817

Hit Rate @5: The correct answer appeared in the top 5 results for every query
MRR (Mean Reciprocal Rank): On average, the correct answer ranked between 1st and 2nd place

Why These Metrics?

Hit Rate tells you "can the system find the right answer at all?"
MRR tells you "how quickly does it find it?" (1st place = 1.0, 2nd = 0.5, 3rd = 0.33)

Both are standard IR (Information Retrieval) metrics. Having these numbers lets you objectively compare different chunking strategies, embedding models, or retrieval parameters.

What I Learned

1. Chunking strategy matters more than the embedding model

Switching from fixed-size to sentence-boundary chunking had a bigger impact on retrieval quality than I expected. The embedding model can only work with what it's given — if chunks are semantically broken, even the best model can't fix that.

2. Evaluation should come before optimization

Building the evaluation script early gave me a baseline to measure improvements against. Without it, I would have been tuning blindly.

3. Start simple, verify, then add complexity

The entire system is ~200 lines of Python (excluding tests). No frameworks, no complex abstractions. This made debugging straightforward — when something went wrong, there were very few places to look.

What's Next

Reranker: Add a cross-encoder reranker to improve ranking quality (especially for Q4 and Q9 which ranked 3rd)
Hybrid search: Combine vector similarity with keyword matching (BM25) for better recall
More data: Scale from 15 to 100+ subsidies with automated collection
Production frontend: Migrate from Streamlit to Next.js + FastAPI for a proper web application

If you're building a RAG system for the first time, I'd recommend the same approach: build from scratch first, measure with real metrics, then decide if you need a framework. The understanding you gain is worth far more than the time saved by a framework.

When to Use pdfme vs Jupyter for Custom Report Generation

Daichi — Sun, 22 Mar 2026 10:28:28 +0000

In many enterprise systems, “custom report generation” sounds simple — until it isn’t.

Clients don’t just want static PDFs. They want flexibility, speed, and sometimes deep data analysis.
The challenge is choosing the right tool for the right use case.

In this article, I’ll break down two powerful approaches:

pdfme → Best for layout-driven, self-service reports
Jupyter (Python) → Best for logic-heavy, data-driven reports

🧩 1. Use Case: pdfme (Layout Customization & Self-Service)

Typical Client Request

“Can you move the logo slightly to the right?”
“I want the remarks section to be larger.”

Solution

With pdfme, clients can:

Drag & drop elements (logo, text, tables)
Adjust layout visually
Instantly reflect changes in the output PDF

👉 No engineering involvement required.

✅ When pdfme is the Right Choice

Layout changes are the primary requirement
You want clients to self-manage templates
Reports are standardized formats:
- Quotations
- Invoices
- Delivery notes
Speed of iteration is critical

💡 Why It Works

pdfme separates:

Template (layout)
Data (JSON input)

This makes it extremely flexible for non-engineers.

🚀 Key Benefit

“Empower the client to fix their own reports — instantly.”

📊 2. Use Case: Jupyter (Data Processing & Advanced Logic)

Typical Client Request

“Generate monthly sales reports, and extract top 10 companies into a separate sheet.”

Solution

With Jupyter Notebook + Pandas:

Aggregate and transform data
Apply complex business logic
Generate charts (matplotlib / seaborn)
Export to PDF or Excel

👉 Fully customizable, but requires engineering support.

✅ When Jupyter is the Right Choice

Data processing is complex
Each report has different business logic
Reports include:
- Aggregations
- Rankings
- Forecasts
Visualization is required (charts, graphs)

💡 Why It Works

Jupyter allows you to combine:

Code
Data
Visualization
Documentation

All in one reproducible environment.

🚀 Key Benefit

“Turn raw data into insights — not just documents.”

⚖️ Side-by-Side Comparison

Aspect	pdfme	Jupyter
Primary Focus	Layout	Data Processing
User	End-user (Client)	Internal (Engineers)
Flexibility	High (UI-based)	Very High (Code-based)
Maintenance	Low	Medium–High
Best For	Invoices, templates	Analytics reports

🧠 Decision Framework

Ask this simple question:

“Is the complexity in the layout, or in the data?”

If layout-heavy → Use pdfme
If logic-heavy → Use Jupyter

🔥 Real-World Insight

In many projects, the best solution is not choosing one.

👉 Combine both:

Use pdfme for standardized documents
Use Jupyter for analytical reports

This separation reduces:

Engineering workload
Client dependency
Maintenance complexity

✍️ Final Thoughts

Custom report generation is not just a technical problem — it's a product design decision.

Choosing the wrong tool leads to:

Endless small requests
Engineering bottlenecks
Frustrated clients

Choosing the right one enables:

Scalable, self-service reporting + powerful analytics

If you're building a reporting system, start with this question:

“Who should control the report — the client, or the engineer?”

That answer will guide your architecture.

Building an AI-Powered CRM Chat Assistant with n8n and OpenRouter

Daichi — Tue, 10 Mar 2026 10:50:59 +0000

Build a No-Code AI CRM Assistant in One Day (n8n + Claude + OpenRouter)

What if your team could query a CRM just by asking a question?

Instead of navigating dashboards or exporting CSV files, imagine typing:

“Show me deals closing this month.”

…and getting the answer instantly.

In this article, I'll show you how I built a no-code AI CRM assistant in a single day that allows employees to query a CRM using natural language.

The entire system runs without writing a single line of application code.

Tech Stack

n8n — self-hosted workflow automation
OpenRouter — AI model API gateway
CRM — CRM with REST API
Docker — container runtime
Claude Sonnet 4.6 (via OpenRouter) — LLM powering the agent

Architecture

[Employee] → n8n Chat UI → [AI Agent (Claude via OpenRouter)]
                                ├── Tool: Search API
                                └── Tool: Search API
                                      ↓
                         　　　    [CRM API]
                                      ↓
                        [AI generates response in Japanese]
                                      ↓
                          [Employee receives answer]

The key idea is to let the AI agent decide which CRM API to call based on the user's natural language query.

Step 1 — Run n8n with Docker

Spinning up n8n locally takes only one command:

docker run -d \
  --name n8n \
  --restart always \
  -p 5678:5678 \
  -e WEBHOOK_URL=http://YOUR_IP:5678/ \
  -e N8N_EDITOR_BASE_URL=http://YOUR_IP:5678/ \
  -v n8n_data:/home/node/.n8n \
  n8nio/n8n

Key Learnings

Authentication has changed

Older guides mention N8N_BASIC_AUTH_*, but these variables are now deprecated.

n8n now uses email-based account setup through the web UI on first launch.

LAN access requires correct environment variables

Without setting:

WEBHOOK_URL
N8N_EDITOR_BASE_URL

the chat UI may attempt to call localhost, causing CORS errors for other users on the network.

Step 2 — Building the AI Agent

Instead of a simple pipeline like:

Chat → HTTP Request → AI

I built a tool-enabled AI Agent.

This allows the LLM to choose the correct CRM endpoint autonomously.

Workflow structure:

Chat Trigger → AI Agent
                ├── Tool: search
                └── Tool: search

Each tool calls a specific CRM endpoint

Dynamic query example:

{{ $json.query }}

Step 3 — The System Prompt (The Most Important Part)

The system prompt determines whether the agent works reliably.

Step 4 — Sharing on LAN

Find your IP:

ipconfig getifaddr en0

Share the chat URL:

http://YOUR_IP:5678/webhook/xxxxx/chat

Enable Basic Auth for security.

Challenges & Lessons Learned

Token Limit Explosion

Querying the CRM without filters returned the entire dataset.

Result: 8M+ tokens

Solution:

enforce search conditions
block empty queries

Cost

Component	Cost
n8n (self-hosted)	$0
OpenRouter (Claude Sonnet)	~$26/month
Infrastructure	$0

Supports roughly 10 users × 5 queries/day.

Conclusion

Using n8n + OpenRouter, I built a working AI CRM assistant in less than a day.

The biggest takeaway:

System prompt design is critical.

With the right guardrails, you can turn any API-driven system into a natural-language interface.

The Death of SaaS: Is It Possible to Replace Software Interfaces with AI Agents?

Daichi — Sun, 08 Mar 2026 11:10:01 +0000

For more than a decade, SaaS (Software as a Service) has dominated the software industry.

Companies built dashboards.
Users logged in.
People clicked buttons to get work done.

But something fundamental is changing.

With the rise of AI agents, the way we interact with software may be entering a completely new phase.

Some people are now asking a provocative question:

Is SaaS dying?

Maybe not completely — but the way SaaS works is about to change dramatically.

The Traditional SaaS Model

Classic SaaS follows a simple pattern:

User
 ↓
UI (dashboard)
 ↓
Software
 ↓
Business outcome

Think about tools like CRM systems, analytics dashboards, or ticket management platforms.

A human logs in, navigates the UI, and performs tasks manually.

This model has worked incredibly well because:

software moved to the cloud
companies paid subscriptions
pricing scaled with the number of users

The more employees a company had, the more SaaS licenses it needed.

This is why per-seat pricing became the standard.

AI Agents Change the Equation

AI agents introduce a completely different model.

Instead of humans operating software, AI can operate software directly.

Human
 ↓
AI Agent
 ↓
APIs
 ↓
Software systems

An AI agent can:

retrieve data
analyze information
trigger workflows
update records
communicate across multiple systems

All without opening a dashboard.

The “user” of the software is no longer necessarily a human.

The Interface Is Becoming Less Important

Historically, SaaS companies invested heavily in:

beautiful dashboards
intuitive UX
complex workflow interfaces

But AI agents don't care about dashboards.

They interact with systems through:

APIs
structured data
automation frameworks

In other words:

The interface was built for humans — not for machines.

As AI agents take over operational tasks, the value of the UI layer may decrease.

The Problem With Per-Seat Pricing

Traditional SaaS pricing assumes something important:

Every user is a human.

But AI agents break this assumption.

Imagine a company where AI agents perform the work of multiple employees:

generating reports
managing tickets
analyzing data
updating CRM records

Does it make sense to charge per seat anymore?

Probably not.

Future pricing models might shift toward:

task-based pricing
API usage
outcome-based pricing

This would fundamentally reshape the SaaS business model.

From Software Tools to Autonomous Systems

Another major shift is happening.

Traditional SaaS gives you tools.

AI-driven systems deliver outcomes.

Instead of logging into five different tools, you might simply tell an AI:

"Analyze our sales pipeline and generate a weekly report."

The AI could automatically:

retrieve CRM data
analyze the pipeline
generate insights
produce a presentation
send it to the team

No dashboard required.

SaaS Isn't Dead — It's Becoming Infrastructure

Despite the dramatic headline, SaaS probably isn't disappearing.

Instead, it may evolve into something different.

SaaS products may increasingly become:

systems of record
data infrastructure
API platforms for AI agents

The new architecture could look like this:

Human
 ↓
AI Agent
 ↓
APIs
 ↓
SaaS infrastructure

In this model, SaaS becomes the backend layer of intelligent automation.

What This Means for Builders

If you're building software today, the key question might no longer be:

How do we build a better dashboard?

Instead, it might be:

How do we build software that AI agents can operate?

This means focusing on things like:

robust APIs
machine-readable workflows
automation primitives
agent-friendly architectures

The most valuable products in the future may not have the best UI.

They may have the best interfaces for AI.

The Real Question

So is SaaS dying?

Maybe not.

But the human interface layer of SaaS might be fading, replaced by AI-driven automation.

The future might look less like this:

Human → SaaS

and more like this:

Human → AI → Software

If that's true, the winners of the next decade won't just build great software.

They'll build software that AI can drive.

I'm curious what other engineers think.

Are we witnessing the evolution of SaaS, or the beginning of its replacement?