Developing AI-Friendly CLIs

I publish a variety of CLIs as OSS (aqua, pinact, tfcmt, ghalint, ghir, etc).
These days AI handles CLIs more and more often, but unless an OSS project is fairly well known, AI doesn't have the knowledge it needs to answer questions about it or to troubleshoot it.
Coding agents therefore fall back on Web Fetch, but fetching blindly isn't efficient, and Web Fetch has the additional problem that the content gets summarized.

Now that using coding agents has become the norm, when you develop a CLI it matters a great deal how you give AI knowledge about that CLI.

ghtkn is an OSS project that creates short-lived (8 hours) GitHub App user access tokens for secure local development.
It has features designed to make it AI-friendly.
For example, coding agents such as Claude and Codex can reach ghtkn's documentation without using Web Fetch.

This article introduces that work.

1. Include messages for agents in the CLI's help and logs
2. Organizing the documentation
3. Add subcommands that list documents and print their contents
4. Add pointers to the docs command in README.md, --help, --version, error messages, and more
5. Call the docs command from an Agent Skill

Note that this article is based on ghtkn v0.3.5, but nothing here is specific to ghtkn — it applies to CLI development in general.

1. Include messages for agents in the CLI's help and logs

ghtkn includes messages like the following.

1. Pointers to the docs command

   If you are a coding agent, run 'ghtkn docs list' to list the documentation and
   ghtkn docs show <doc> to read it before answering questions about ghtkn or
   troubleshooting its errors.

   -

   Run ghtkn docs list to see documentation and ghtkn docs show <name> to read it; this may help resolve the error.

2. Guidance to hand commands that require human interaction back to the user

   The Device Flow is interactive and can't be completed by a background or non-interactive process.
   If you are a coding agent, do NOT run ghtkn get yourself because it would fail the same way;
   instead, ask the user to run ghtkn auth in their own interactive terminal to authenticate

3. Warnings about leaking secrets

   The output is a secret. Do not print, echo, log, or include it in a chat message,
   a commit, or any other output, and do not run 'ghtkn get' (including -f json) just
   to display or inspect the token. If you are a coding agent, this applies to your
   responses too: a leaked token can be used until it is revoked. Consume it without
   showing it: assign it to an environment variable and pass that to the tool, e.g.
   'GH_TOKEN=$(ghtkn get) gh issue list'. Better still, avoid handling the raw token
   at all - for git, use the credential helper ('ghtkn git-credential'), which lets
   git fetch the token automatically; for gh, use a wrapper that sets GH_TOKEN.

I've also made the logs more detailed in various other places.
Agents will actually read logs that a human would skip right past (at least sometimes), so emitting detailed logs has become more important than it used to be.

2. Organizing the documentation

Keep the documentation in the same repository as the CLI's code, and split it by topic.
Write the documents in Markdown, with a description in the YAML frontmatter.
Write that description for agents, in the same way you would write an Agent Skill's description.
Rather than maintaining the human-facing documentation and the skill separately, unify them so they're easier to maintain.
That means writing the body so that both humans and agents find it easy to read.

README.md # Cover the details of each topic under docs/*.md and link to them
docs/
  install.md
  ...

3. Add subcommands that list documents and print their contents

Provide subcommands that output the documentation, so that agents can reach it on their own.
No skill installation is required, so every user benefits.
By embedding the documents into the tool at build time, the tool's version and the documentation's version can never drift apart.
The flip side is that you have to release a new version to ship a documentation update, but having the versions line up should cause fewer problems overall.

Provide a docs list command that lists the documents and a docs show command that prints the body of a given document.
The docs list command also emits a small but useful help message:

Run ghtkn docs show {name} to see the details of each document.

For a document's name, take its /-separated file path and strip the common docs/ prefix and the .md extension.

$ ghtkn docs list
{
  "results": [
    {
      "name": "agent-deployment",
      "description": "Run the ghtkn agent as a long-lived process. Use to set it up as a systemd user service, to start it from a container entrypoint, or to run it on the host so that containers use it as a client, which is what refresh tokens need."
    },

    ...(omitted)

    {
      "name": "troubleshooting",
      "description": "Diagnose ghtkn problems and known limitations. Use when ghtkn or the gh wrapper misbehaves, a token is expired (401), the device flow code is not shown, or hitting Packages API / cross-user repo limits."
    }
  ],
  "help": "Run `ghtkn docs show {name}` to see the details of each document."
}

Now an agent can run something like ghtkn docs show troubleshooting to read the documentation.

ghtkn doesn't have that many documents yet, so I haven't implemented a search feature.
A search can come up empty and waste tokens, and there's also the question of how to implement search in a CLI in the first place, so unless you have a lot of documents, a list command and a show command should be enough.

Implementation detail: Go has embed

I write nearly all of my CLIs in Go, and ghtkn is no exception.
In Go you can embed files into the tool with the embed package, and you can select the target files with a glob such as //go:embed *.md.

Refs: code1, code2

Directory layout:

go.mod
docs/
  doc.go # Embeds the documents into the tool with embed
  backend.md
  ...

4. Add pointers to the docs command in README.md, --help, --version, error messages, and more

This overlaps with section 1, but I add pointers to the docs command in many places.
Adding a command that outputs documentation is pointless if the agent never notices it.

An agent will sometimes skip the help entirely, run only --version, and then go straight to running commands, so I added a pointer to the --version output as well.
It may feel noisy to a human, but as long as the message isn't too long it should be acceptable.

$ ghtkn -v
ghtkn version 0.3.5
Aug  1 11:24:33.623 INF If you are a coding agent, run `ghtkn docs list` to list the documentation and `ghtkn docs show <name>` to read it before answering questions about ghtkn or troubleshooting its errors. program=ghtkn version=0.3.5

5. Call the docs command from an Agent Skill

Create an Agent Skill and have the skill call the docs command (example).

Skills have a few drawbacks compared with the docs command, so not shipping a skill at all is a legitimate option.

1. Many users won't install a skill for every tool
   1. So even if you publish a skill, it may never get used
2. Third-party skills carry security risks, so some organizations ban them outright or require a security review
3. You have to keep updating the skill's version to match the CLI's version (as described below, having the skill call the docs command mitigates this)

That said, if you build it the way described below, maintenance isn't much of a burden and it doubles as a way to promote the tool to users, so there's little to lose by shipping one.

1. Create only one skill per CLI (don't split it up by topic)
2. Introduce the docs command inside the skill and encourage the agent to use it

Rather than shipping the documents themselves as a skill, have the skill run the docs command.
This keeps the skill itself very simple and keeps its update frequency low, and it makes version mismatches between the skill and the CLI less likely.
If you split the skill up by topic, then when the topic structure changes or a topic is renamed or removed, stale skills stick around even after an update; consolidating into a single skill avoids that problem entirely.

Verifying whether agents use the docs command autonomously

Adding the command and the pointers to it doesn't guarantee that agents will actually use them on their own, so let's verify it.
If the verification shows that the pointers aren't enough, add more.
In my case, watching how the agent behaved is what led me to add a pointer (an info log) to --version.

1. Uninstall the agent skill
2. Move aside any local clone of the code to prevent agents from reading it
3. Start a coding agent in an unrelated directory and ask it some questions

I deliberately avoid explicitly forbidding local file access or Web Fetch, and check whether the agent still prefers the docs command.

First I asked whether it knew about ghtkn. It didn't seem to know much and used Web Fetch.
Its query did contain keywords like GitHub App and access token, so it appeared to know something, but I can't really tell whether that came from my environment or whether it genuinely knew.
At the very least, it did not run ghtkn at this point.

Next I asked about the ghtkn agent. It explored as follows: it started with Web Fetch, but noticed ghtkn docs partway through and was able to answer correctly.

1. Web Fetch against a made-up URL, which 404s and comes up empty
2. which ghtkn to confirm that ghtkn is installed
3. Notices ghtkn docs while reading the help from ghtkn --help
4. ghtkn docs list to list the documents
5. ghtkn docs show backend to read the details

I followed up by asking what to do about a leaked access token. This time it skipped Web Fetch entirely, ran ghtkn docs show revoke-tokens and ghtkn revoke --help, and answered correctly.

So the agent noticed the docs command on its own, ran it, and read the documentation.
That said, agents behave probabilistically, so naturally it doesn't always go this well.

Next I installed the skill and asked the same questions in a new session. This time it skipped Web Fetch from the very beginning, ran ghtkn docs, and read the documentation.

Conclusion

That's a tour of the techniques I use to develop AI-friendly CLIs.
I develop plenty of CLIs besides ghtkn, so I'd like to make those AI-friendly as well where it makes sense.
This article doesn't cover GitHub Actions such as tfaction and securefix-action, but I'd like to make GitHub Actions AI-friendly too.
Being AI-friendly is convenient for users, of course, and it would be great if it also reduced the support burden on maintainers.