RAG - Designing the CLI interface

This is the second article of a series in which I am building an open source RAG CLI tool. In the previous article we defined the goal of the tool and analyzed the steps to achieve it.

In this article, we will design the CLI interface and start writing some code.

Hold on, do I really need to read the previous article to be able to follow with this one?

Ideally yes, but here is the summary we achieved in the previous article:

We are building a CLI tool that stores documentations of different frameworks/libraries and allows to do semantic search and extract the relevant parts from them.

Requirements:

• Git: will be used to clone the documentations repository
• Ollama: will be used to generate embeddings

Assumptions / Limitations:

• We are assuming that the documentations are available as markdown files on a Git repository

Database:
We choose to go with SQLite for now and add support for other databases in the future.

Designing the CLI interface

Brainstorming

Let's call our tool rag, the first commands that come to mind are:

• rag setup: setup the requirements of the tool (git, ollama, the embeddings model, ...).
• rag add: add documentations to the tool.
• rag get: retrieve the relevant parts from documentations.

if I understand correctly, the setup command should be run right after the installation of the tool. What happens if the user doesn't run it and start by doing a rag add or rag get directly?

Good catch, my first idea would be to show an error message asking the user to run the setup command first. But now that I am thinking about it, if we can detect when some requirements are missing, then we can automatically install them, let the user know via a message (maybe on stderr to avoid polluting the output), and then continue with the requested command.

Pros: The commands just work out of the box, they automatically add missing requirements when needed. No need for a setup command.
Cons: All commands have to check requirements at startup, which can slow down the tool.

I think the resilience we gain by automatically handling requirements is worth the performance hit. A slower command is better than a command that doesn't work.

I agree with you on removing the setup command and checking requirements at the start of each command. But I would ask for user's permissions before installing any new software on their system.

You are right, we should ask for permissions before installing any new software.

Ok, now for the add command, what would be the arguments?

I was thinking of something like:

rag add <name> <repo_url> [--subdir <subdir>] [--branch <branch>]

• <name>: the name of the framework/library
• <repo_url>: the URL of the repository
• --subdir <subdir>: a subdirectory of the repository where the documentation markdown files are stored
• --branch <branch>: the branch of the repository

Looking good, but how do we handle different versions of the same framework/library?

Ah, good point. I feel like adding a --version flag would make the command complicated, especially because we should also point to the correct branch/tag to fetch the docs of the correct version.
A simpler way is to let the user handle that by adding the version to the name itself, something like

rag add tailwindcss:v2 https://github.com/tailwindlabs/tailwindcss --branch v2 --subdir src/pages/docs
rag add tailwindcss:v3.4 https://github.com/tailwindlabs/tailwindcss --branch v3.3 --subdir src/pages

I feel that this will be easier to handle while giving the user the ability to have docs of different versions of the same framework/library.

I am not convinced that this is the best approach, but let's go with it. Now what if the user already has a documentation in the database and try to add it again? will it overwrite the existing one?

Good question, I think we should show an error message saying that a documentation with the same name already exists.

Ok, but how would the user update a documentation that already exists?

We can add a new command to do that

rag update <name> [--repo_url <repo_url>] [--subdir <subdir>] [--branch <branch>]

This would update the existing documentation repo_url, subdir or branch if provided, then fetch and update the docs.

And we can also add a rag remove command to remove a documentation from the database when needed.

So far we have the following commands:

rag add <name> <repo_url> [--subdir <subdir>] [--branch <branch>]
rag update <name> [--repo_url <repo_url>] [--subdir <subdir>] [--branch <branch>]
rag remove <name>
rag get ...

Since the three commands add, update and remove handle docs, we can group them under a docs command:

rag docs add <name> <repo_url> [--subdir <subdir>] [--branch <branch>]
rag docs update <name> [--repo_url <repo_url>] [--subdir <subdir>] [--branch <branch>]
rag docs remove <name>

Good, now what about the get command? what are the arguments?

This is the main command of our tool, let's start with something like:

rag get <prompt> [--count <count>]

• prompt: the prompt for which we want to retrieve the relevant parts
• --count <count>: the number of relevant parts to retrieve

This would print the relevant parts to stdout.

What if the prompt contains multiple lines? wouldn't it make sense to read the prompt from stdin?

Hmm, yeah. Let's support both; if the argument exists we use it, otherwise we read the prompt from stdin.

I think it will also be useful to to have a json output option, so that we can call the command from within some code and easily parse the output.

Good idea, let's add a --json flag. With this flag, the output would be as follows:

[
  {
    "meta": {
      "collection": "tailwindcss:v3.4",
      "filename": "getting-started.md",
      "similarity": 0.8,
      ...
    },
    "content": "markdown content of the relevant part ..."
  },
  ...
]

Specifications

The rag tool will have the following subcommands:

# add new documentations
rag docs add <name> <repo_url> [--subdir <subdir>] [--branch <branch>]

# update existing documentations
rag docs update <name> [--repo_url <repo_url>] [--subdir <subdir>] [--branch <branch>]

# remove existing documentations
rag docs remove <name>

# get relevant parts from documentations
rag get <prompt> [--count <count>] [--json]

All commands should check for requirements at startup and ask for permissions before installing any new software on the user's system.

Implementing the CLI interface

Choosing the programming language

I think it's time to choose the language in which we will implement this tool, so that we can start to write some code. My constraints are:

1. I want to be able to ship the tool as a single executable file.
2. I want implement a working version as soon as possible.
3. The tool is not computationally intensive, and I don't need to handle many threads.

Given these constraints, I will go with Typescript and Bun for this project.

But you said you need to compile your code into a single binary, can Typescritpt do that?

Yes, Bun can produce a binary of your code: https://bun.sh/docs/bundler/executables

I thought Python was the preferred language for AI stuff, and it seems to fit your constraints.

Yes, Python would be a good option. I just chose Typescript because I am more familiar with it.

Creating the project

Let's start by creating an empty bun project:

bun init

After writing the project name rag and the entry file src/main.ts, I have the following directory structure:

node_modules/
src/
  main.ts
.gitignore
bun.lockb
package.json
tsconfig.json

I like to use Prettier to format my code, so

bun add -D prettier

and create a .prettierrc file:

{
  "printWidth": 155,
  "semi": false,
  "tabWidth": 2,
  "singleQuote": true,
  "trailingComma": "all",
  "bracketSpacing": true
}

Implementing the CLI interface

I chose to use the commander package to implement the CLI interface. After installing it, I wrote the following code in src/main.ts:

import { program } from 'commander'

program.name('rag').version('0.0.1').description('Simple RAG system for developers')

const docs = program.command('docs').description('Manage documentations')

docs
  .command('add')
  .description('Add new documentation')
  .argument('<name>', 'Name of the documentation')
  .argument('<repo_url>', 'URL of the Git repository')
  .option('--subdir <subdir>', 'Subdirectory within the repository', '.')
  .option('--branch <branch>', 'Branch to use', 'main')
  .action((name, repo_url, options) => {
    console.log(`Adding documentation ${name} from ${repo_url} with options ${JSON.stringify(options)}`)
  })

docs
  .command('update')
  .description('Update existing documentation')
  .argument('<name>', 'Name of the documentation')
  .option('--repo_url <repo_url>', 'New repository URL')
  .option('--subdir <subdir>', 'New subdirectory within the repository')
  .option('--branch <branch>', 'New branch to use')
  .action((name, options) => {
    console.log(`Updating documentation ${name} with options ${JSON.stringify(options)}`)
  })

docs
  .command('remove')
  .description('Remove existing documentation')
  .argument('<name>', 'Name of the documentation')
  .action((name) => {
    console.log(`Removing documentation ${name}`)
  })

program
  .command('get')
  .description('Get relevant parts from documentations')
  .argument('<prompt>', 'Search prompt')
  .option('--count <count>', 'Number of relevant parts to retrieve', '5')
  .option('--json', 'Output results in JSON format')
  .action((prompt, options) => {
    console.log(`Getting relevant parts from documentations for prompt ${prompt} with options ${JSON.stringify(options)}`)
  })

program.parseAsync().catch((err) => {
  console.error(err)
  process.exit(1)
})

Now running bun src/main.ts shows the following output:

Usage: rag [options] [command]

Simple RAG system for developers

Options:
  -V, --version           output the version number
  -h, --help              display help for command

Commands:
  docs                    Manage documentations
  get [options] <prompt>  Get relevant parts from documentations
  help [command]          display help for command

And doing bun src/main.ts docs add tailwindcss https://github.com/tailwindlabs/tailwindcss shows the following output:

Adding documentation tailwindcss from https://github.com/tailwindlabs/tailwindcss with options {"branch":"main", "subdir":"."}

So I guess the CLI interface is working as expected.

No, you said that the prompt argument should be read from stdin if it doesn't exist. But it seems to be required right now.

You are right, if I do

bun src/main.ts get

I get the following error:

Missing required argument 'prompt'

To make the argument optional, we should use [prompt] instead of <prompt> in the argument definition. And now we can check when the argument is missing, and read it from stdin.

program
  .command('get')
  .argument('[prompt]', 'Search prompt')
  // ...
  .action(async (prompt, options) => {
    prompt = prompt || await Bun.stdin.text()
    console.log(`Getting relevant parts from documentations for prompt "${prompt}" with options ${JSON.stringify(options)}`)
  })

Which gives the following:

❯ bun src/main.ts get "arg prompt"
Getting relevant parts from documentations for prompt "arg prompt" with options {"count":"5"}

❯ echo "stdin prompt" | bun src/main.ts get             
Getting relevant parts from documentations for prompt "stdin prompt
" with options {"count":"5"}

Summary

In this article,

• we designed the CLI interface and chose to have the commands add, update, remove and get.
• we chose to implement the tool using Typescript and Bun.
• we implemented the CLI interface using the commander package.

What's next

The next steps are:

• Add the database
• Implement the commands logic
• Add tests
• Create CI/CD pipeline

Feel free to comment if you have any suggestions or feedback. See you in the next article!