DEV Community: keterslayter

Caliber is building in the open: here is our roadmap and we want your input

keterslayter — Fri, 20 Mar 2026 16:12:29 +0000

If you work with AI coding tools like Claude Code, Cursor, or Codex, you know the pain: every new project means hunting down configs, re-adding MCP servers, copying prompts, and hoping nothing breaks. Caliber fixes that with one command.

Caliber continuously syncs your AI setups, generates optimized agent skills, MCPs and config files, and keeps everything in one place so your whole team starts fast and stays consistent.

We are building this in the open and we want the community to shape what comes next. Here is what is on our roadmap right now.

1. MCP server discovery, sync, and installation

Right now, MCP server configs live in scattered files across your machine. Caliber will scan your existing tool configs (Cursor, Claude, Codex), store them centrally, and automatically install them for every developer joining a project. No more "it works on my machine" for AI setups. Secrets stay out of git, transport support covers both HTTP and stdio servers, and the whole thing runs in under 30 seconds on a fresh machine.

2. SkillBench: a community skill marketplace built into the CLI

Think npm, but for AI agent skills. With SkillBench you will be able to run caliber skills search to find community-built skills by framework or use case, install them with one command, and publish your own. Skills are ranked by stack compatibility and community usage, and the registry follows the OpenSkills format so they work across tools. A developer should be able to find and install a relevant skill in under 10 seconds.

3. Show instant value: tokens saved, time saved, errors prevented

One of the hardest parts of adopting any new tool is justifying it to your team or yourself. After you run caliber init or caliber score, Caliber will immediately show you the tangible impact: estimated tokens saved per session, developer time recovered from pre-configured commands, and errors caught from wrong file paths or hallucinated configs. No setup required to see the ROI.

What do you think is missing?

This is the part where we genuinely want your help. We are building Caliber for developers who use AI tools every day and we know there are pain points we have not thought of yet.

Drop a comment below with what you would add to this roadmap, what you wish existed in the AI tooling space, or just tell us if any of these features would actually change your workflow. You can also browse the full public roadmap and comment directly on the issues on GitHub: https://github.com/caliber-ai-org/ai-setup

If you want to follow along, contribute, or just lurk, join the conversation on r/aisetups on Reddit. All feedback shapes what gets built next.

Your AI agent is lying to you — and your CLAUDE.md is why

keterslayter — Mon, 16 Mar 2026 18:33:47 +0000

TL;DR — Your AI coding agent's quality is capped by the quality of its context. Most devs have stale, generic, or missing CLAUDE.md / .cursorrules files. Caliber fixes this in one command — scans your repo, generates tailored configs, recommends MCPs, and gives you a 0-100 setup score.
---
## The problem nobody talks about
Everyone's debating which AI coding tool is best — Claude Code vs Cursor vs Codex. Meanwhile, the real reason most developers aren't getting great results is upstream of all of them:
Their project context is wrong.
Your CLAUDE.md was written in 20 minutes when you first set up the project. Your .cursorrules was copy-pasted from a Reddit thread. Neither has been touched since.
And your AI agent is making decisions based on that stale, inaccurate information every single session.
## What bad context actually looks like
Here are the most common failure modes I've seen:
🚨 Stale architecture — Your CLAUDE.md says you use REST APIs but you migrated to GraphQL 3 months ago. The agent keeps generating REST patterns.
🚨 Contradictory rules — Old rules say "use CommonJS", newer ones say "use ESM". The agent picks one arbitrarily.
🚨 No MCP coverage — You're running PostgreSQL and there's a great Postgres MCP that would let your agent query your schema directly. You've never heard of it.
🚨 Config drift — You refactor every week. Your AI config was updated once, on day one.
🚨 Team inconsistency — One dev has MCPs set up, another doesn't. Rules differ across machines. There's no source of truth.
## Meet Caliber
Caliber is a CLI tool that solves this with one command. It scans your project and:
- ✅ Generates a tailored CLAUDE.md with your actual stack, architecture, and commands
- ✅ Creates .cursorrules / .cursor/rules/*.mdc matching your dependencies
- ✅ Recommends MCPs you should install based on what you're running
- ✅ Deletes stale rules that contradict your current code
- ✅ Scores your setup 0–100 across 6 dimensions
And it can run continuously — so configs stay fresh as your code evolves.
## See it in action
Here's what caliber init looks like on a real Next.js + TypeScript project:

bash $ caliber init Scanning project structure... Detected: TypeScript, React, Next.js, Tailwind CSS Detected: 847 files, 12 dependencies with AI relevance Config files: + create CLAUDE.md project context + create .cursorrules cursor rules ~ modify .cursor/rules/testing.mdc outdated patterns - delete .claude/rules/old-api.md stale, contradicts code Skills: + create .claude/skills/deploy.md deploy flow + create .cursor/skills/review.md code review MCP Recommendations: • @modelcontextprotocol/server-postgres (detected: pg in package.json) • @modelcontextprotocol/server-github (detected: .git, gh-cli) • @upstash/context7-mcp (detected: React 18+) 

Notice it's not just adding files — it also deletes stale ones and modifies outdated patterns.
## The setup score: caliber score
This is my favorite feature. Run caliber score on any project and get a deterministic 0–100 grade — no LLM needed, works offline.

bash $ caliber score Score: 87/100 Grade: A Existence 23/25 • Config files present, cross-platform parity Quality 22/25 • Commands documented, no bloat, no vague rules Coverage 18/20 • Dependencies & services reflected in configs Accuracy 13/15 • Documented commands & paths actually exist Freshness 8/10 • Config recency, no leaked secrets Bonus 3/5 • Hooks, AGENTS.md, OpenSkills format 

This gives you an objective baseline before onboarding a new dev, switching AI tools, or auditing your setup.
## The full CLI — 4 commands, that's it
| Command | What it does |
|---|---|
| caliber init | Scan repo, generate/update all config files |
| caliber score | Rate your setup 0–100 (offline, no LLM) |
| caliber recommend | Discover MCPs and skills for your stack |
| caliber config | Set provider, API key, model |
Works with: Claude Code, Cursor, OpenAI Codex
No API key needed with Claude Code or Cursor — uses your existing subscription.
## Why this matters for teams
This isn't just a solo-dev tool. The consistency problem is worse at the team level:
> "One dev has MCPs configured, another doesn't. Cursor rules differ across machines. Nobody knows which CLAUDE.md is the canonical one."
With Caliber, you commit your configs to git like any other file. Every developer who runs caliber init gets the same baseline — and the same AI agent experience. New team members are set up in 30 seconds.
## Get started in 30 seconds

bash npm install -g @rely-ai/caliber caliber init 

That's it. No API key needed if you're on Claude Code or Cursor.
🔗 GitHub: https://github.com/rely-ai-org/caliber
📦 npm: https://www.npmjs.com/package/@rely-ai/caliber
💬 Discord: https://discord.gg/XUNaJEsw
MIT licensed. Open source. Your code never leaves your machine.
---
Built at Rely AI. If this saved you time, a ⭐ on GitHub goes a long way — and PRs are very welcome.

Why your AI coding agent is working with bad context/skills and how to make AI setup tailored for your codebase.

keterslayter — Mon, 16 Mar 2026 13:44:32 +0000

npx @rely-ai/caliber init
That's it. On first run, Caliber walks you through provider setup interactively.

Or install globally:

npm install -g @rely-ai/caliber
caliber init
Already have an API key? Skip the interactive setup:

export ANTHROPIC_API_KEY=sk-ant-...
npx @rely-ai/caliber init
⚙️ How It Works
caliber init
│
├─ 1. 🔌 Connect Choose your LLM provider — Claude Code seat, Cursor seat,
│ or an API key (Anthropic, OpenAI, Vertex AI)
│
├─ 2. 🔍 Discover Analyze languages, frameworks, dependencies, file structure,
│ and existing agent configs via LLM
│
├─ 3. 🛠️ Generate Create CLAUDE.md + skills in parallel (core docs via smart
│ model, skills via fast model). Auto-polishes for max score
│
├─ 4. 👀 Review See a diff of proposed changes — accept, refine via
│ chat, or decline. All changes are backed up automatically
│
└─ 5. 🧩 Skills Search community skill registries and install relevant
skills for your tech stack
Caliber works on any codebase — TypeScript, Python, Go, Rust, Terraform, Java, Ruby, and more. Language and framework detection is fully LLM-driven, not hardcoded.

What makes it different

Private & local — runs on your machine with your own API key. Your code never leaves your environment.
Always up-to-date — run it again after any refactor. It catches the drift.
Multi-platform — one command sets up Claude Code, Cursor, and Codex together.
Fully open source — MIT licensed. Audit every line, fork it, contribute back.

Try it

npm install -g @rely-ai/caliber
caliber init
caliber score

That's the entire CLI. Three commands.

GitHub: github.com/rely-ai-org/caliber
npm: @rely-ai/caliber
Site: caliber-ai.up.railway.app
Discord: discord.gg/XUNaJEsw