A coding agent CLI can say it's signed in and still not work: the account has no plan for it, or it's out of usage until the limit resets.
So in Styx, the app I'm building, 'Ready' means more than 'signed in'. During setup it runs the vendor's own installer and sign-in, then sends one tiny message and waits for the answer. If the answer comes back, the plan really works. If it says 'needs a plan' or 'out of usage', the card says that in one sentence, with the fix.
Two things I learned doing it.
Never write your own login flow for someone else's CLI. Run their installer and their sign-in, and keep only where the binary lives and which account it reports. Passwords and tokens stay with the vendor.
Some CLIs have no cheap check. Goose has no offline identity probe, and its provider check spends a model turn. Amp has none either. For those, the row says 'not verified' and still lets you start a task, because a green tick I can't back up is worse than an honest grey one.
The tradeoff is that the tiny message costs a few tokens on a real plan, so I only send it during setup and when you ask.
How do you check that an agent CLI works before you hand it a real task?
Top comments (0)