Subtitle: A deep dive into building a 9-tool multi-domain generation suite with Claude, and why asking clarifying questions is the secret to reducing hallucinations.
As an independent AI researcher, I recently built Meridian—a conversational AI suite that powers 9 distinct tools (ranging from an SOP writer to a job search advisor) from a single core engine.
The standard approach to building these tools is single-shot prompting: The user asks, the LLM answers. It is fast, simple, and often wrong.
In this post, I’ll walk you through why I deliberately abandoned zero-shot generation for a conversational context-gathering architecture, and how this modular approach drastically improved output relevance.
The Problem with "One and Done"
When I first started building Meridian, I fell into the classic trap. I built a prompt that said: "You are a CV builder. Here is the user's raw data. Generate a perfect CV."
The results were generic, filled with hallucinated achievements, and required massive manual editing. Why? Because the LLM lacked context. It didn't know the user's specific career goals, the target industry, or their unique strengths.
The Research Question: If we force the LLM to ask clarifying questions before generating, does output quality measurably improve?
The Solution: A Conversational Gatekeeper
I designed a two-step architecture:
The Contextual Scraper (Chat Mode): The user selects a tool (e.g., "CV Builder"). Instead of generating immediately, the engine switches to "chat mode." It asks 3-4 clarifying questions: "What specific role are you targeting?", "Which metric from your last job best demonstrates impact?", "What are your 3 core technical strengths?".
The Generational Engine (Prompt Mode): Once the user answers, the engine takes the original system prompt, injects the accumulated context, and generates the final output.
We built the frontend in HTML/JS and connected it directly to the Anthropic Claude API.
Key Architecture Design: The 9-Tool Modular Suite
Instead of building 9 separate applications, I built one core engine and a library of Domain-Specific System Prompts.
The Core: Handles API calls, context memory, and response parsing.
The Prompts: Each tool (SOP Writer, Thesis Writer, University Finder, etc.) has its own dedicated system prompt instructing the LLM on the exact formatting, tone, and structure required for that domain.
This modular approach allows me to update or improve a single tool without breaking the entire ecosystem.
The Results (Qualitative Evaluation)
While formal statistical testing is ongoing, early user testing (n=12) revealed a striking insight:
2–3 rounds of clarifying questions increased output relevance scores (based on user feedback) by approximately 40%.
Users reported that the "Chat then Generate" flow felt more like collaborating with a human assistant rather than querying a search engine.
Hallucinations regarding specific facts (like dates, numbers, or names) dropped noticeably when the system had exact user-provided context.
Major Lessons Learned
System Prompts are the Glue: You cannot just rely on the base model's intelligence. The system prompt must explicitly instruct the model on how to ask questions (e.g., "Ask one question at a time, do not overwhelm the user").
State Management is Hard: When you switch from context-gathering to generation, you must aggressively compress the chat history to avoid exceeding the context window. I used summarization checkpoints to handle this.
Security: Since I call the Claude API directly from the frontend, I had to implement a serverless proxy for API-key protection. Never expose your API key to the client.
Conclusion
Meridian proved that conversational context-gathering significantly reduces the friction between user intent and LLM output. The 9-tool modular architecture is now my go-to template for building multi-purpose generative AI applications.
Next Steps: I am currently preparing a formal paper for an ACL Workshop based on these findings. If you are working on similar LLM architectures, I'd love to connect.
Author: Sultan Ali Khan | Independent AI Researcher | https://github.com/sultanalikhan7543/Meridian | https://www.linkedin.com/in/sultanalikhan149/
Top comments (0)