Chat with Docs Using OpenAI and a Serverless RAG Tool

Documentation can be hard to dig through. Let’s fix that by turning any LLM into a doc-savvy AI agent with memory, all in a few steps.

Using BaseAI, we’ll create an AI agent locally that can pull answers from docs and respond to user queries.

Prerequisites

• Signup on Langbase
• Basic knowledge of BaseAI
• OpenAI LLM key

Quick Setup

Here's how it's done:

1. Create a Project Folder:

Start a new project, install dev dependencies, and add dotenv to manage environment variables. Also, add the OpenAI key in .env file.

2. Set Up an AI Agent Pipe:

Run this command to create a serverless AI agent:

   npx baseai@latest pipe

Enter a name (e.g., pipe-with-memory) and a description. Here’s a basic config:

   import { PipeI } from '@baseai/core';

   const pipeWithMemory = (): PipeI => ({
       apiKey: process.env.LANGBASE_API_KEY!,
       name: 'pipe-with-memory',
       description: 'Pipe attached to a memory',
       model: 'openai:gpt-4o-mini',
       stream: true,
       store: true,
       max_tokens: 1000,
       temperature: 0.7,
       memory: [],
       tools: []
   });

   export default pipeWithMemory;

3. Create and Add Memory:

Run this command in terminal to create a memory:

   npx baseai@latest memory

Name it chat-with-docs, then drop markdown docs into baseai/memory/chat-with-docs/documents.

4. Embed the Memory:

To make the docs searchable generate memory embeddings by running this command:

   npx baseai@latest embed -m chat-with-docs

5. Connect Memory to the Agent

Import the memory to the agent config and update the code:

import { PipeI } from '@baseai/core';
import chatWithDocsMemory from '../memory/chat-with-docs';

const pipePipeWithMemory = (): PipeI => ({
    // Replace with your API key https://langbase.com/docs/api-reference/api-keys
    apiKey: process.env.LANGBASE_API_KEY!,
    name: 'pipe-with-memory',
    description: 'Pipe attached to a memory',
    status: 'private',
    model: 'openai:gpt-4o-mini',
    stream: true,
    json: false,
    store: true,
    moderate: true,
    top_p: 1,
    max_tokens: 1000,
    temperature: 0.7,
    presence_penalty: 1,
    frequency_penalty: 1,
    stop: [],
    tool_choice: 'auto',
    parallel_tool_calls: true,
    messages: [{ role: 'system', content: `You are a helpful AI assistant.` }],
    variables: [],
    memory: [chatWithDocsMemory()],
    tools: []
});

export default pipePipeWithMemory;

6. Running the Agent in CLI

Create a CLI with index.ts file to interact with your AI agent. Add the following code in the index.ts file you create in the project directory:

import 'dotenv/config';
import { Pipe } from '@baseai/core';
import inquirer from 'inquirer';
import ora from 'ora';
import chalk from 'chalk';
import pipePipeWithMemory from './baseai/pipes/pipe-with-memory';


const pipe = new Pipe(pipePipeWithMemory());

async function main() {

    const initialSpinner = ora('Conversation with document...').start();
    try {
        const { completion: chatWithDocsMemory} = await pipe.run({
            messages: [{ role: 'user', content: 'Hello' }],
        });
        initialSpinner.stop();
        console.log(chalk.cyan('Report Generator Agent response...'));
        console.log(chatWithDocsMemory);
    } catch (error) {
        initialSpinner.stop();
        console.error(chalk.red('Error processing initial request:'), error);
    }

    while (true) {
        const { userMsg } = await inquirer.prompt([
            {
                type: 'input',
                name: 'userMsg',
                message: chalk.blue('Enter your query (or type "exit" to quit):'),
            },
        ]);

        if (userMsg.toLowerCase() === 'exit') {
            console.log(chalk.green('Goodbye!'));
            break;
        }

        const spinner = ora('Processing your request...').start();

        try {
            const { completion: reportAgentResponse } = await pipe.run({
                messages: [{ role: 'user', content: userMsg }],
            });

            spinner.stop();
            console.log(chalk.cyan('Agent:'));
            console.log(reportAgentResponse);
        } catch (error) {
            spinner.stop();
            console.error(chalk.red('Error processing your request:'), error);
        }
    }
}

main();

This index.ts file sets up a simple CLI for chatting with an AI agent trained on your documentation. It begins by loading essential packages: environment variables with dotenv, the Pipe class from BaseAI, and libraries for user interaction and styling—inquirer for prompts, ora for spinners, and chalk for color-coding responses.

The file then initializes a Pipe instance, pipePipeWithMemory, configured with AI memory, enabling the agent to access and respond with information from your documents. In the main() function, an initial test message ("Hello") is sent to the agent, displaying a spinner while awaiting a response.

Once received, the agent's response is logged to the console. Following this, a continuous loop prompts the user to enter queries, which are processed by the agent via pipe.run(). Responses appear in the console, and the loop allows ongoing interaction until the user types “exit.”

Any errors in processing are logged as well. This setup provides a simple, interactive way to chat with an AI agent about your documentation directly from the terminal.

7. Start the Server and Test

Start the BaseAI’s dev server by this command:

   npx baseai@latest dev

Run the CLI (index.ts) file, run this command:

   npx tsx index.ts

You’re all set! Now you can ask questions, and your agent will pull answers straight from your docs, all with zero cloud costs and the results can be viewed in your terminal.

Resources

• BaseAI Docs
• BaseAI Learn Platform