Building ScopedAgent: A Production Template for Knowledge-Scoped AI Agents

How to build production-ready customer support AI agents in Next.js 14 & LangChain.js that never hallucinate off-topic code, respect strict knowledge boundaries, and log out-of-scope user attempts across OpenAI, Anthropic, Gemini, Ollama, LM Studio, and custom endpoints.

---

The Problem With Generic RAG Starter Kits

Standard Retrieval-Augmented Generation (RAG) starter kits often suffer from a major flaw: unbounded scope.

When a user asks a support bot for your SaaS product to "write a Python script to scrape a website" or "explain binary search trees", traditional RAG systems still query the vector database and try their best to answer—often hallucinating or generating arbitrary code that has nothing to do with your company.

ScopedAgent addresses this problem by introducing a Pre-Retrieval Scope Guard Layer before vector search or document ingestion occurs.

---

Core Architecture & Flow

Every prompt follows this strict workflow:

---

Key Technical Implementation

1. Scope Guard Classifier (lib/agent/scope-guard.ts)

Before touching ChromaDB or document stores, the query is pre-classified into allowed vs. refused categories (writing_code, generic_technical, off_topic, jailbreak_attempt).

const ClassificationSchema = z.object({
  category: z.enum([
    "in_scope",
    "writing_code",
    "generic_technical",
    "competitor_analysis",
    "personal_advice",
    "off_topic",
    "jailbreak_attempt",
  ]),
  confidence: z.number().min(0).max(1),
  reasoning: z.string(),
  sanitized_query: z.string(),
});

export async function classifyQuery(
  query: string,
  context: string,
  config: Config,
  llm: BaseChatModel
): Promise<Classification> {
  const chain = SCOPE_GUARD_PROMPT.pipe(
    (llm as any).withStructuredOutput(ClassificationSchema)
  );
  return chain.invoke({
    company: config.agent.company,
    allowed_topics: config.scope.allowed_topics.map((t) => `- ${t}`).join("\n"),
    query,
    context: context || "None",
  });
}

2. Multi-Provider AI Inference Engine (lib/providers/index.ts)

Developers and end users can pick their preferred AI model directly in the UI header or configuration:

• OpenAI: gpt-4o, gpt-4o-mini
• Anthropic: claude-3-5-sonnet
• Google Gemini API: gemini-1.5-flash
• Local AI: Ollama (http://localhost:11434), LM Studio (http://localhost:1234), and custom OpenAI-compatible endpoints.

3. Single-File Developer Configuration (scopedagent.config.ts)

Developers only edit one configuration file to define their agent's scope rules, company identity, and custom refusal responses:

export default defineConfig({
  agent: {
    name: "Aria",
    company: "Northpeak Software",
    role: "Customer Support Assistant",
  },
  scope: {
    allowed_topics: [
      "product features and how-to questions",
      "pricing and plan comparisons",
      "billing and refund policy",
    ],
    refused_categories: [
      "writing_code",
      "generic_technical",
      "off_topic",
      "jailbreak_attempt",
    ],
    confidence_threshold: 0.72,
    fallback_message: "I don't have enough info on that. Contact [e-mail].",
  },
});

---

Admin Analytics Dashboard (/admin)

Every out-of-scope attempt is persisted to SQLite (lib/db/queries.ts), giving developers real-time metrics on:

• Top refused categories (Code Gen vs. Competitor Analysis vs. Off-topic).
• Raw prompt logs and classification confidence scores.
• Document re-indexing controls to update knowledge on the fly.

---

Try It Out & Contribute

• Repository: https://github.com/harishkotra/scopedagent