DEV Community: Saif Ali

Configuration Management in Markdown

Saif Ali — Sat, 31 Oct 2026 16:53:21 +0000

Introducing Korta: Configuration Management in Markdown

Managing servers should not require a complicated stack just to install packages, create users, or keep services running.

Those tasks are familiar to anyone who operates infrastructure. Yet the knowledge behind them often ends up scattered across shell scripts, internal wikis, configuration files, and someone’s memory. Over time, it becomes harder to answer a simple question: What is supposed to be running on this server?

Korta is our attempt to make that answer easier to write, review, and apply.

Korta is a command-line tool for configuration management in Markdown. You describe the desired state of your servers in a role, preview what would change, and apply it over SSH.

# Packages

- nginx, curl

# Users

- deploy added to apps group, shell /bin/bash

# Directories

- /opt/app owned by deploy:apps, mode 0750

# Services

- nginx started, enabled

Then run:

korta plan role.md -i hosts.md
korta apply role.md -i hosts.md

The plan checks your hosts and shows the changes Korta would make. The apply command plans first, asks for confirmation, and then carries out the changes.

There is no agent to install on each server. Korta connects over SSH and uses root or passwordless sudo.

Configuration you can read

A role is a Markdown document organized around the things you want to manage. You can describe users, groups, packages, directories, files, services, kernel settings, firewall rules, updates, and custom commands.

Your hosts live in a Markdown inventory:

# Production

identity: ~/.ssh/prod_ed25519
- web1 deploy@web1.internal env=prod

# Staging

identity: ~/.ssh/staging_ed25519
- web2 deploy@web2.internal env=staging

You can use labels to select a subset:

korta apply role.md -i hosts.md --limit env=staging

Markdown gives teams a familiar place to keep configuration alongside the context that explains it. A role can live in Git, be reviewed in a pull request, and be read by the next person responsible for the system.

Who Korta is for

Korta is for people who manage Linux servers and want a straightforward way to describe and maintain their configuration.

That includes developers running a few virtual machines, small teams maintaining their own infrastructure, platform engineers who want reviewable roles, and consultants managing servers across multiple environments.

It is especially useful when you want to keep control of your servers and use the SSH access you already have. The core CLI can run directly from your machine without a central manager.

Korta supports Ubuntu, Debian, RHEL, Rocky Linux, AlmaLinux, and CentOS Stream. Hosts need SSH and appropriate privileges; they do not need a Korta agent.

Why now

More teams are reexamining how they run software. Some want more control over infrastructure costs. Others want to run applications and databases on their own servers, or simply make existing systems easier to understand and maintain.

At the same time, teams are increasingly using AI to help write operational code. That makes review more important: generated configuration still needs to be understandable, validated, and approved by the people responsible for the machines.

Korta starts with a workflow built around those needs:

Write the desired state.
See what would change.
Review and approve it.
Apply it to the selected hosts.

The goal is to make routine server work easier to share and repeat, while keeping the operator in control.

Why open source

Server configuration is foundational. The tool that interprets it should be something people can inspect, understand, and improve.

We want Korta’s core to be open source so developers and operators can see how roles are parsed, how plans are produced, and what runs over SSH. We also want teams to be able to use the CLI in their own environments and contribute fixes, ideas, and support for real-world use cases.

Open source is also a practical way to build a better infrastructure tool. Servers, operating systems, and operational requirements vary widely. The people using Korta will encounter cases we have not, and the project will be stronger with their feedback and contributions.

The open-source core includes the korta CLI, Markdown role engine, inventory parser, SSH client, and operating-system detection.

Korta Enterprise builds on that core for teams that need a shared manager. It adds a web console and API, approval and audit workflows, an encrypted vault, an AI role assistant, and deployment features for databases and applications. The same CLI and role files work with the Enterprise manager.

Where to start

Write a small role for a staging server, add the host to an inventory, and run korta plan. Review the output, then apply it when you are ready.

The core project is at github.com/nexusrun/korta (https://github.com/nexusrun/korta). We’re looking forward to sharing it and hearing what you build with it.

Databases on NEXUS AI: Managed, Portable, and Ready for AI-Built Apps

Saif Ali — Mon, 26 Oct 2026 23:21:48 +0000

Databases on NEXUS AI: Managed, Portable, and Ready for AI-Built Apps

Author: NEXUS AI Team

Category: Databases · Platform · AI Deployments

Reviewed: October 26, 2026

CLI syntax verified: NEXUS AI CLI 4.3.2

Reading time: 13 minutes

Most AI-built apps fail at the same point. The interface works, the demo impresses, and then someone asks where the data lives, who backs it up, and what happens when the AI agent runs a migration at 2 a.m.

Generating code got fast. Running the database underneath it did not get any easier.

NEXUS AI gives every app a database path that matches where it is in its life: a database that starts with the app, a standalone database you share between apps, the same database in your own AWS, Azure, or Google Cloud account, and a guarded way to connect databases you already run. One dashboard, one CLI, and one MCP server manage all of them.

This article explains what each option is, where to use it, how it looks from a management point of view, how portable your data stays, and why the design fits AI deployments in particular.

What NEXUS AI databases are

NEXUS AI databases are the data layer of the NEXUS AI deployment platform. There are four ways to use them:

Option	What it is	Engines	Manage with
App database service	A database that runs next to one app, created with the deploy	PostgreSQL, MySQL, MongoDB, Redis	`nexus db`
Managed database on NEXUS AI	A standalone database you create once and attach to any app on NEXUS AI	PostgreSQL, Redis	`nexus managed-db --local`
Managed cloud database	A standalone database on Amazon RDS, Google Cloud SQL, or Azure Database, in the platform account or yours	PostgreSQL, MySQL	`nexus managed-db --provider`
Connected external database	A database you already run elsewhere, connected for schema inspection, guarded queries, and AI fix proposals	PostgreSQL, MySQL	Dashboard and MCP

All four sit behind the same organization, the same role-based permissions, and the same tools.

Where to use each option

App database service: for prototypes and single apps

Add a database when you deploy, and NEXUS AI runs it next to your app and injects the connection settings:

nexus deploy source --repo https://github.com/your-org/helpdesk --services postgres,redis

Use it when one app owns its data: a prototype, an internal tool, a side project, or the first version of a product. It is the fastest path from "generated code" to "working app with real data".

Each service gets backups you control:

nexus db services helpdesk                               # find the service ID
nexus db backup <service-id>                             # back up now
nexus db backup-schedule <service-id> --enable --retention 14   # daily, keep 14 days

Managed database on NEXUS AI: for data shared by several apps

When two apps need the same data (an API and a worker, or an app and its admin panel), the database should not belong to either one. Create it on its own:

nexus managed-db create orders-db --local --engine postgres
nexus managed-db attach orders-db --deployment orders-api
nexus managed-db attach orders-db --deployment orders-worker

Each app receives DATABASE_URL (or REDIS_URL for Redis) on its next deploy. Use --env-prefix to give a second database a different name, such as ANALYTICS_URL. Deleting an app no longer puts the data at risk, because the database has its own lifecycle.

Managed databases on NEXUS AI listen on the primary host's private Docker network, so they attach to apps deployed on NEXUS AI on that host.

Managed cloud database: for production and compliance

When the data needs a cloud provider's durability, encryption, and controls, create the database on Amazon RDS, Google Cloud SQL, or Azure Database for PostgreSQL and MySQL:

nexus managed-db create billing-db --provider AWS_RDS --engine postgres \
  --engine-version 17.10 --instance-class db.t3.micro --allocated-gb 20 --region us-east-1

Or let the deploy create it for you. On a cloud provider, --create-db provisions a managed database paired with the app:

nexus deploy source --repo https://github.com/your-org/billing --provider aws_app_runner --create-db postgres

Amazon RDS instances are created with storage encryption and seven days of automated backups. On AWS you can choose --network-mode vpc to keep the database private, reachable only from App Runner apps in the same account.

If you have added your own cloud credentials in Deployment Providers, new cloud databases are created in your AWS account or Azure subscription, not the platform's. See Bring Your Own Cloud Account.

Connected external database: for data you already have

Not every database should move. Connect an existing PostgreSQL or MySQL database in the dashboard, then inspect its schema, preview queries, and get AI-proposed fixes from your app's error logs, all with guardrails described below.

Quick decision guide

Your situation	Start with
Prototype or demo with real data	App database service
One app, small team, needs backups	App database service with a backup schedule
Several apps share the same data	Managed database on NEXUS AI
Production workload, customer data, uptime commitments	Managed cloud database
Compliance requires data in your own account	Managed cloud database with your own cloud provider
Data already lives somewhere else	Connected external database

The management point of view

For the person responsible for the platform, databases are usually where tools sprawl: one console per cloud, scripts for backups, and a wiki page about who has the passwords. NEXUS AI keeps them in one place.

One inventory. Every database in the organization, whichever option and provider, appears in the dashboard and in nexus managed-db list and nexus db services.

Permissions by role. Reading data and changing it are separate permissions. A teammate with read access can run SELECT queries; write and schema-changing queries require the managed_databases.manage permission. Assign roles per person, and use scoped access tokens for CI.

Backups with a floor. Scheduled backups keep at least 7 days of history, and retention can be raised but never lowered below that. Expired automatic backups are pruned on schedule; backups you upload are never deleted automatically.

Recovery that does not overwrite. Managed database restores always create a new database from a snapshot:

nexus managed-db snapshot orders-db --notes "before v2 migration"
nexus managed-db restore orders-db --snapshot <snapshot-id> --new-name orders-db-check

You compare the restored copy with the original before you switch anything.

Safe sharing of backups. Instead of handing out credentials, create a signed download link that expires in 30 to 3,600 seconds:

nexus db backup-download <service-id> <backup-id> --share --ttl 900

Plan limits you can predict.

Plan	Standalone managed databases
Free	App database services only
Starter	2
Pro	10
Enterprise	Unlimited

Portability: your data is never trapped

A database platform is only as good as your ability to leave it. NEXUS AI is built on that assumption.

Standard engines, no proprietary layer. PostgreSQL, MySQL, MongoDB, and Redis run as themselves. Your app connects with its normal driver and a normal connection URL. No SDK, no special query language.

Backups in native formats. PostgreSQL backups use pg_dump, MySQL uses mysqldump, and MongoDB uses mongodump. Download any backup and restore it with standard tools on any server:

nexus db backup-download <service-id> <backup-id> --out ./prod.dump
pg_restore -d "$TARGET_DATABASE_URL" --no-owner ./prod.dump

Move data between services. Copy production into staging without leaving the platform, as long as both use the same engine:

nexus db restore-to <staging-service-id> <backup-id> --yes

Or bring data in from outside with nexus db backup-upload.

Connection settings, not code changes. Apps read their database from environment variables. Detach one database and attach another, redeploy, and the app follows. Moving from a database on NEXUS AI to Amazon RDS is a data copy and an attach, not a rewrite.

Your cloud account, your resources. Cloud databases created with your own provider credentials live in your account. If you stop using NEXUS AI, they keep running where they are.

Flexibility: one workflow, many shapes

Four engines for different jobs: relational data in PostgreSQL or MySQL, documents in MongoDB, caches and queues in Redis.
Four homes for standalone databases: NEXUS AI, Amazon RDS, Google Cloud SQL, and Azure Database.
Public or private networking on AWS: public endpoints restricted to allowed address ranges, or VPC mode with no public endpoint.
Several databases per app, each under its own environment variable prefix.
Three ways in: the dashboard for people, the CLI for scripts and CI, and the MCP server for AI assistants.

Who it is for

Founders and solo builders who generated an app with AI and need real data, backups, and a URL today, without learning a cloud console.

Product and agency teams shipping many small apps, who need each one to have a database that is backed up, separated, and easy to hand over.

Platform and DevOps leads who want one inventory, consistent backups, and role-based access across every database their teams create.

Regulated and enterprise teams who need the data in their own cloud account, encrypted, with their own audit trail, while developers keep a simple workflow.

Why NEXUS AI is a strong fit for AI deployments

AI changes who touches the database. An agent in Claude, Cursor, or Codex can now create a schema, write a migration, and run it, often faster than a person can review it. That is useful, and it needs guardrails that do not depend on the agent behaving perfectly.

AI assistants get real database tools. The NEXUS AI MCP server exposes database operations directly to AI assistants: listing services, taking and restoring backups, creating and attaching managed databases, taking snapshots, inspecting schemas, and running queries. An agent can deploy an app with a database and back it up before a migration in the same conversation.

Every query is classified before it runs. Queries are sorted into reads, data changes, schema changes, and blocked operations. Statements such as DROP DATABASE, role changes, extension installs, and server file access are refused outright.

Preview before execute. On connected databases, a preview runs read queries with EXPLAIN and does not apply changes, so the agent (and you) can see what a statement would do first. Execution runs with a statement timeout and returns at most 1,000 rows by default.

Permissions apply to agents too. An assistant acts with the permissions of the person or token it uses. Give an agent a read-only role and it cannot change data, whatever it is asked to do.

Fixes are proposals, not surprises. NEXUS AI can read an app's error logs and propose a schema fix, such as a missing column or index. The fix is stored as a proposal and is applied only when someone applies it.

Backups are one command away from any agent. The habit that saves teams most often is simple: snapshot before a migration. With MCP, asking the agent to "take a snapshot first" is all it takes, and restores never overwrite the original.

The database follows the app's lifecycle. AI-built apps change quickly: prototype on Monday, shared with a team on Wednesday, in production the next month. The path from app database service to managed database to your own cloud account uses the same commands and the same connection variables at every step.

What to know before you start

Managed databases on NEXUS AI support PostgreSQL and Redis; use a cloud provider for managed MySQL.
Managed databases on NEXUS AI attach only to apps running on the NEXUS AI primary host.
Private (VPC) cloud databases must be in the same cloud account as the app that uses them.
App database service restores replace the current data in place. Take a fresh backup first if you may want the current state back.
restore-to requires the source and target to use the same engine.

Frequently asked questions

What databases does NEXUS AI support?

PostgreSQL, MySQL, MongoDB, and Redis as app database services. PostgreSQL and Redis as managed databases on NEXUS AI. PostgreSQL and MySQL as managed cloud databases on Amazon RDS, Google Cloud SQL, and Azure Database.

Can I export my data and leave?

Yes. Backups use the engines' native dump formats, so you can download them and restore them anywhere with standard tools. Cloud databases created with your own credentials are already in your account.

Is it safe to let an AI agent run queries?

NEXUS AI classifies every query, blocks dangerous statements, separates read and write permissions, and offers a preview mode. Give agents the least access they need, and ask them to snapshot before migrations.

What is the difference between `nexus db` and `nexus managed-db`?

nexus db manages databases that run alongside one app, created with --services. nexus managed-db manages standalone databases that you create once and attach to any number of apps.

Can I move from a database on NEXUS AI to Amazon RDS later?

Yes. Create the RDS database, copy the data with a backup and restore, attach the new database to the app, and redeploy. The app keeps reading DATABASE_URL.

How long are backups kept?

Scheduled backups are kept for 7 days by default, and you can raise retention per service. Backups you upload are kept until you delete them.

Start with the database your app needs today

Deploy your next app with --services postgres, turn on a backup schedule, and connect your AI assistant to NEXUS AI through MCP. When the data outgrows one app, create a managed database and attach it. When it needs a cloud provider or your own account, move it there with the same tools.

Virtual Models: One Model Name, Any Model Behind It

Saif Ali — Mon, 26 Oct 2026 18:50:55 +0000

Virtual Models: One Model Name, Any Model Behind It

A new model ships almost every week. One is cheaper, one is faster, one finally handles your hardest coding task. Taking advantage of it should be a five-minute decision. In most stacks it is a small project: change an environment variable in three services, update a provider SDK, add a new API key, redeploy, and hope nothing else breaks.

Virtual models remove that work. This article explains what a virtual model is, when to use one, why it fits the way AI agents and apps are built today, and how to route different jobs to different models using the routing strategies in ZRouter, an open source, OpenAI-compatible AI gateway.

What is a virtual model?

A virtual model is a model name you define, plus a rule for what that name means.

Your application sends the name in the normal model field:

{ "model": "assistant", "messages": [{ "role": "user", "content": "Hi" }] }

The gateway reads assistant, looks up its rule, and decides which real provider model serves the request. The rule can be as simple as "assistant means anthropic/claude-sonnet-4-6" or as rich as "send easy prompts to a cheap model, hard prompts to a strong one, and fail over to a second provider if the first is down".

Your app never learns which model answered unless you want it to. It only knows the name.

In ZRouter, a virtual model comes in two forms:

Redirect (alias). A name that points at one or more real models. This is what most of this article is about.
Access policy. A rule with no target that restricts who can use an existing model, by user path, or disables it.

You create them in the dashboard (Models → New virtual model), in config.yaml, through the VIRTUAL_MODELS environment variable, or through the admin API. When prepaid billing is enabled, customers can also create private virtual models in their own dashboard.

The problem it solves: model choice is wired into your code

Without a gateway, the model is usually decided in three places at once:

The endpoint. Each provider has its own base URL.
The credentials. Each provider has its own API key, stored in each app's environment.
The model ID. A string like gpt-4o-2024-08-06 hardcoded or set in an environment variable.

Changing models means changing one or all of these, in every service that calls the model. For a single chatbot that is annoying. For a company with ten apps and twenty agents it is a migration.

With a gateway and virtual models, all three are fixed:

One endpoint: https://zrouter.si/v1 (or your self-hosted gateway), for every provider.
One key: your gateway key. Provider keys live in the gateway, not in each app.
One stable model name: assistant, coder, summarizer, whatever you choose.

from openai import OpenAI

client = OpenAI(base_url="https://zrouter.si/v1", api_key=ZROUTER_API_KEY)

reply = client.chat.completions.create(
    model="coder",
    messages=[{"role": "user", "content": "Refactor this function..."}],
)

Switching coder from one model to another is an edit in the gateway. No environment variable changes, no new endpoint, no redeploy, no restart of the agent. The next request simply goes to the new model.

Why virtual models fit today's AI agents and apps

Models change faster than release cycles

Frontier models are updated, renamed, repriced, and deprecated on a monthly rhythm. If every model change needs a code release, you either fall behind or spend your team's time on plumbing. A virtual model turns "upgrade the model" into a configuration change you can make and undo in seconds.

Agents call models from many places

A modern agent is not one model call. It plans, writes code, calls tools, summarizes results, checks its own work, and often runs sub-agents. Each step has a different ideal model. Hardcoding a model ID in each step spreads model decisions across your codebase. Naming each step by its job (planner, coder, reviewer, summarizer) keeps the agent code about the job, and moves the model decision to the gateway.

Many tools only let you set one model

Coding agents and AI apps such as Claude Code, Codex, Cursor, OpenCode, OpenWebUI, LibreChat, and n8n let you set a base URL and a model name, and nothing more. A virtual model lets that single name carry routing logic, failover, cost control, and guardrails the tool itself knows nothing about.

Reliability needs more than one provider

Providers have outages and rate limits. A virtual model with several targets keeps serving as long as one target is available, without the app writing retry logic for each provider.

Cost needs to follow the task

Sending every request to the strongest model is expensive. Sending every request to the cheapest model hurts quality. Routing by task, time of day, or request difficulty lets you pay for capability only where it matters.

Teams need governance in one place

Who can use which model, which guardrails apply, and how much they can spend are platform decisions. Putting them on the virtual model means every app that uses the name gets the same policy.

When to use a virtual model

Use one when any of these is true:

You expect to change or upgrade the model behind a feature.
More than one app or agent step shares a model decision.
You want failover to a second model or provider.
You want to balance load or cost across several models.
You want different models for different jobs behind a clear name.
You want to A/B test a model without touching the app.
You need to limit access to a model for certain users or teams.
You want guardrails to apply to everything that calls a model name.

When should you skip it? If one script calls one model and you have no routing, failover, or access needs, call the provider model ID directly. ZRouter accepts both, so you can add a virtual model later without changing anything else.

Route by job: name models after what they do

The most useful habit with virtual models is naming them after the job, not the model. Here is a typical agent setup:

Virtual model	Job	Good fit
`planner`	Break a task into steps, reason about trade-offs	Strongest reasoning model
`coder`	Write and edit code	Best coding model, with failover
`fast`	Classify, extract, route, short answers	Small, cheap, low-latency model
`summarizer`	Condense logs, documents, tool output	Cheap model with long context
`reviewer`	Check output, judge quality	A different model family from `coder`
`auto`	General chat where difficulty varies	Difficulty-based routing

The agent code calls planner for planning and fast for classification. When a better coding model ships, you repoint coder and every agent improves at once. When summarizer is costing too much, you swap it for a cheaper model and watch quality, all without a deploy.

Each of these names can use a different routing strategy. That is where the real power is.

Routing strategies, and what each is best for

A ZRouter redirect can have several targets and a strategy that decides which target serves each request. Every strategy shares three behaviors:

Unavailable targets are skipped. An unknown model or a provider that is down does not break the name.
The other targets form a failover chain. If the chosen target fails with a retryable error, ZRouter tries the remaining available targets in order. You can turn this off per virtual model.
Session keeping is on by default. Requests from one detected client session (for example one Claude Code or Codex session) stay on the same target, so provider prompt caches stay warm and behavior stays consistent mid-conversation. The failover strategy ignores this setting because it always uses the first available target anyway.

1. Single target (alias)

What it does: One name, one model.

Best for: Stable names for apps, hiding provider IDs, and making future upgrades a one-line change. Start here for every job, and add targets when you need them.

virtual_models:
  - source: summarizer
    target: openai/gpt-4o-mini

2. Round-robin (default)

What it does: Rotates requests across targets. Weights bias the share: a target with weight 2 gets twice the traffic of a target with weight 1.

Best for:

Spreading load across providers to stay under rate limits.
Gradual migrations and canaries: send 10% of traffic to a new model with weights 9 and 1, then shift.
Comparing models on real traffic.

  - source: coder
    strategy: round_robin
    targets:
      - { model: anthropic/claude-sonnet-4-6, weight: 9 }
      - { model: openai/gpt-5, weight: 1 }

3. Lowest cost

What it does: Sends every request to the cheapest currently available target, using catalog input plus output prices. Targets without pricing are skipped while a priced target exists.

Best for: High-volume jobs where several models are good enough and price is the deciding factor: classification, extraction, tagging, routine summaries. If the cheapest provider goes down, the next cheapest takes over automatically.

  - source: fast
    strategy: cost
    targets:
      - { model: openai/gpt-4o-mini }
      - { model: groq/llama-3.3-70b }
      - { model: anthropic/claude-haiku-4-5 }

4. Failover

What it does: Always uses the first available target in the list. Lower entries only serve when everything above them is unavailable or fails.

Best for: Production features where you have a clear preferred model and want backups only for emergencies. Customer-facing chat, critical agent steps, and anything where consistency matters more than spreading load.

  - source: planner
    strategy: failover
    targets:
      - { model: anthropic/claude-opus-4-6 }
      - { model: openai/gpt-5 }
      - { model: anthropic/claude-sonnet-4-6 }

5. Schedule (time of day)

What it does: Uses one set of targets during a daily peak window and another set off-peak, in a timezone you choose.

Best for:

Paying for speed when users are waiting (business hours) and saving money when they are not.
Overnight batch jobs, evals, and background agents that can run on a cheaper model.
Moving traffic away from a provider that is congested at certain hours.

  - source: assistant
    strategy: schedule
    targets:
      - model: openai/gpt-4o
      - model: groq/llama-3.3-70b
    strategy_config:
      timezone: America/New_York
      peak_start: "08:00"
      peak_end: "18:00"
      peak_targets: [openai/gpt-4o, groq/llama-3.3-70b]
      off_peak_targets: [groq/llama-3.3-70b]

Set the timezone to where your users are, not where the server runs. Windows that wrap midnight work too.

6. Super Intelligence Router (request difficulty)

What it does: Scores every request from 0 to 100 using signals in the request itself: prompt length, code, reasoning words, math, tools, images, conversation length, and requested reasoning effort. The score picks a tier: simple (below 25), standard (25 to 54), or complex (55 and above). The request goes to the first available target in that tier. Scoring adds well under a millisecond.

Best for: General-purpose names like auto where users and agents send a mix of easy and hard work. Greetings and rewrites go to a cheap model, debugging sessions with tools go to your strongest model. A hard request is never downgraded while a strong model is up: if its tier is empty or down, ZRouter tries stronger tiers first.

  - source: auto
    strategy: super_intelligence
    targets:
      - model: groq/llama-3.3-70b
      - model: openai/gpt-4o-mini
      - model: anthropic/claude-sonnet-4-5
    strategy_config:
      simple_targets: [groq/llama-3.3-70b]
      standard_targets: [openai/gpt-4o-mini]
      complex_targets: [anthropic/claude-sonnet-4-5]

Every response explains the decision, which makes tuning easy:

X-ZRouter-Route: super_intelligence; tier=complex; score=63; reasons=code,tools,reasoning-cues

7. Adaptive (ZRouter Pro)

What it does: Picks the target with the best observed mix of latency, error rate, and price, backs off from targets that are rate limited or failing, and keeps a session on the provider whose prompt cache is warm.

Best for: High-traffic production names where provider performance changes during the day and you want the gateway to react on its own. On an open source build this strategy falls back to round-robin.

8. Plugin strategies

What it does: Hands the choice to a routing plugin. ZRouter ships one, cheapest_healthy, which prefers the cheapest (or fastest) target whose recent error rate is acceptable. You can also write your own.

Best for: Cost-sensitive jobs that also need to avoid flaky providers, and teams with custom routing rules (data residency, customer tiers, internal model servers). If the plugin is missing, errors, or takes longer than 250 ms, the redirect falls back to weighted round-robin, so a broken plugin does not take the name down.

  - source: smart-router
    strategy: plugin
    strategy_plugin: cheapest_healthy
    strategy_config:
      prefer: cheapest     # or: fastest
      max_error_rate: 0.2
    targets:
      - openai/gpt-4o-mini
      - anthropic/claude-haiku-4-5

Choosing a strategy

If the job needs...	Use
A stable name and easy upgrades	Single target
Load spread, canaries, gradual migration	Round-robin with weights
The lowest price among good-enough models	Lowest cost
One preferred model with emergency backups	Failover
Speed by day, savings by night	Schedule
One name for mixed easy and hard work	Super Intelligence Router
Automatic tuning on live performance	Adaptive (Pro)
Cheapest model that is also healthy, or custom rules	Plugin (`cheapest_healthy` or your own)

Combine strategies by chaining virtual models

A target can be another virtual model. This lets you build routing in layers instead of one complicated rule.

virtual_models:
  # A cheap pool, balanced across two providers.
  - source: cheap
    targets:
      - { model: openai/gpt-4o-mini }
      - { model: groq/llama-3.3-70b }

  # A premium model with a backup.
  - source: premium
    strategy: failover
    targets:
      - { model: anthropic/claude-sonnet-4-6 }
      - { model: openai/gpt-5 }

  # Difficulty routing on top of both pools.
  - source: auto
    strategy: super_intelligence
    targets:
      - { model: cheap }
      - { model: premium }
    strategy_config:
      simple_targets: [cheap]
      complex_targets: [premium]

Now auto sends easy work to a balanced cheap pool and hard work to a premium model with failover. Change cheap once, and every name that uses it follows. Chains are checked for cycles and can be up to 8 levels deep. Session keeping applies at every level, so a session stays on one concrete model.

More things a virtual model can carry

Routing is the main use, but the same name can hold other policies:

Shadowing. Give a virtual model the same name as a real model, such as anthropic/claude-opus-4-6, and point it somewhere else. Apps that hardcode that ID get the new model without a code change. This is useful for emergency swaps and for testing whether a cheaper model handles an existing flow.
Access control. Limit a name to certain user paths (teams or customer accounts), or disable it.
Guardrails. Attach up to 16 guardrails that run on every request to that name, such as PII redaction or blocked terms. Every app and agent that uses the name is covered.
Promotions. Publish a model for a limited time with per-user and total request and token caps, for launches and free trials.
Slowdown. Add artificial latency to test how your app behaves with a slower model before you switch.
Model list control. Set KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT=true so /v1/models lists only your curated names instead of every provider model.

A practical rollout plan

Name your jobs. List the places your apps and agents call a model and give each job a name: assistant, coder, fast, summarizer.
Start with single targets. Create each name pointing at the model you use today. Point your apps at the gateway URL and switch the model field to the new names. Behavior is unchanged.
Add failover to critical names. Add a second provider to anything customer-facing.
Optimize cost where quality allows. Move high-volume jobs to lowest cost, or put a general auto name on the Super Intelligence Router.
Experiment safely. Use weighted round-robin to try new models on a small share of traffic, and compare usage, latency, and quality in the dashboard.
Keep the names, change the models. From now on, model upgrades are edits in the gateway.

Key takeaways

A virtual model is a stable name that the gateway resolves to a real model at request time.
Your app keeps one endpoint, one key, and one model name. Switching models never means updating environment variables or reconfiguring endpoints.
Name virtual models after jobs, then give each job the routing strategy that fits it.
Round-robin spreads load, lowest cost saves money, failover protects critical paths, schedule follows the clock, the Super Intelligence Router follows request difficulty, adaptive follows live performance, and plugins cover everything else.
Chain virtual models to combine strategies, and attach guardrails and access rules so every caller gets the same policy.

ZRouter is open source under the MIT license and speaks the OpenAI API. Point any existing client at https://zrouter.si/v1, create your first virtual model, and your next model upgrade will be a configuration change instead of a release.

Where to Put Your AI Cache: Client, Gateway, or Provider

Saif Ali — Mon, 26 Oct 2026 18:17:27 +0000

Where to Put Your AI Cache: Client, Gateway, or Provider

There are three places: inside your application, at an AI gateway between your apps and the providers, and at the provider itself. Each has a role. This post compares them and then walks through a complete setup using ZRouter, an open source AI gateway that runs every caching layer in one place.

Option 1: Cache in the application

You add a cache library or a Redis call around your LLM client code.

Good for:

A single app with a narrow use case, such as caching one expensive summary per document.
Business-level caching, where you know something the model call does not. For example, "this product description is unchanged, so reuse yesterday's summary."

Weak at:

Duplication. Every service, language, and team writes its own cache with its own bugs.
Correctness details. Key design, streaming replay, request coalescing, tenant isolation, and invalidation on policy changes are easy to get subtly wrong.
Provider prompt caching. Each provider has its own directives. Your app code ends up full of provider-specific fields that break when you switch models.
Visibility. Hit rates are scattered across services.

Option 2: Cache at the provider

Provider prompt caching runs on the provider's servers. You mostly need to structure prompts well and, for some providers, send a directive.

Good for:

Long, stable prefixes. It is the only layer that can reuse the model's own processing.
Keeping every answer fresh.

Weak at:

It never skips the call. You still pay for output and for the new part of the input.
Every provider works differently: automatic caching, cache keys, breakpoints, or explicit cached-content objects with their own lifetimes.
It depends on routing. Spreading one conversation across providers, models, or API keys can leave the cache cold.

Option 3: Cache at the gateway

An AI gateway is a proxy that sits between your applications and model providers. All LLM traffic passes through it, which makes it the natural home for shared concerns: routing, failover, budgets, guardrails, logging, and caching.

Good for:

One cache for every app. Any client that speaks the OpenAI API benefits without code changes.
Correct keys. The gateway knows the authenticated tenant, the resolved provider and model, and which policies ran, so it can key on all of them.
Provider prompt caching without provider code. The gateway knows the final provider after routing and can add the right directive for it.
Routing that respects caches. The gateway can keep a session on the same target and key so provider caches stay warm.
One place to measure. Hit rates and cost avoided appear in one dashboard.

Weak at:

Business-level knowledge that only your app has. Keep that in the app.

The recommended layout

For most teams the answer is a combination:

Gateway: exact cache, semantic cache where it fits, prompt-cache planning, and session stickiness.
Provider: prompt caching, driven by the gateway.
Application: only domain-specific caching, plus stable prompt structure (static content first).

Setting it up in ZRouter

The examples below show configuration only. Keep real API keys and connection strings in your secret manager or environment, never in committed files.

Step 1: Provider prompt caching (already on)

ZRouter's provider prompt-cache planner is enabled by default. To make that explicit in your deployment:

PROVIDER_PROMPT_CACHE_PLANNER_ENABLED=true

For Gemini, you can tune the lifetime of the cached-content objects ZRouter creates. The default is 300 seconds, and values from 30 to 3600 are accepted:

GEMINI_PROMPT_CACHE_TTL_SECONDS=300

If your client already sends its own cache directives (for example Anthropic cache_control blocks), ZRouter passes them through unchanged. To turn off automatic planning but keep pass-through, set the flag to false.

Step 2: Exact response cache

Point ZRouter at Redis in config.yaml:

cache:
  response:
    simple:
      redis:
        url: redis://localhost:6379
        ttl: 3600

Or with environment variables only:

RESPONSE_CACHE_SIMPLE_ENABLED=true
REDIS_URL=redis://localhost:6379
REDIS_TTL_RESPONSES=3600

Step 3: Semantic cache (optional)

Add an embedder and a vector store. Qdrant is shown here. pgvector, Pinecone, and Weaviate are also supported.

cache:
  response:
    semantic:
      enabled: true
      embedder:
        provider: openai
        model: text-embedding-3-small
      vector_store:
        type: qdrant
        qdrant:
          url: http://localhost:6333
          collection: aigateway_semantic

Useful tuning variables:

SEMANTIC_CACHE_THRESHOLD=0.92            # similarity required for a hit
SEMANTIC_CACHE_TTL=3600                  # seconds an entry lives
SEMANTIC_CACHE_MAX_CONV_MESSAGES=3       # recent conversation included in the key
SEMANTIC_CACHE_EXCLUDE_SYSTEM_PROMPT=false

If you use pgvector or Pinecone, make sure the configured dimension matches your embedding model's output size (1536 for text-embedding-3-small).

When both layers are enabled, a semantic hit is also remembered in Redis for that exact request. The next identical request skips the embedding and the vector search and is still reported as HIT (semantic).

Step 4: Send traffic and check the headers

Any OpenAI-compatible client works. Send the same request twice:

curl -i https://your-gateway/v1/chat/completions \
  -H "Authorization: Bearer $ZROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"What are your API rate limits?"}]}'

The second response includes:

X-Cache: HIT (exact)

A paraphrased question, with the semantic layer enabled, can return X-Cache: HIT (semantic).

Step 5: Bypass when you need fresh output

For "regenerate" buttons, live data, or debugging, skip both response cache layers for one request:

-H "Cache-Control: no-cache"

Use no-store when the response should not be written to the cache either.

Step 6: Keep sessions sticky for agents

If you run coding agents through ZRouter, session keeping works with zero configuration. Known session headers such as X-Session-Id, X-Claude-Code-Session-Id, and Codex's Session-Id keep a session on the same load-balanced target and the same provider key, so provider prompt caches stay warm for the whole task.

Step 7: Measure

Cache hits are recorded as usage entries with cache_type set to exact or semantic:

# Hits only, for one team
curl "https://your-gateway/admin/usage/summary?user_path=/team/alpha&cache_mode=cached" \
  -H "Authorization: Bearer $AIGATEWAY_MASTER_KEY"

Use /admin/cache/overview for a cache-only summary. Default usage reports exclude hits, so cost totals keep matching your provider invoice.

What ZRouter handles so you do not have to

A quick checklist of the details covered in Part 3, and how the gateway deals with them:

Concern	How ZRouter handles it
Tenant isolation	Keys include the auth scope and account-bound user path
Parameter changes	Exact keys include the final body and resolved model; semantic keys include all request options
Policy changes	The guardrail chain is part of the exact cache key
Formatting differences	JSON is canonicalized before hashing
Traffic spikes	Concurrent identical misses are coalesced per process
Streaming	Raw event streams are stored and replayed
Failover answers	Not cached under the primary's key
BYOK traffic	Bypasses the shared response cache
Provider differences	One planner adds the right prompt-cache directive per provider

Wrapping up the series

AI caching is one of the few optimizations that cuts cost and latency at the same time. The key ideas from all four parts:

Know the layers. Exact and semantic caches reuse answers. Prompt caching reuses the model's reading of your prompt. Embedding caching reuses vectors.
Measure the benefit. Track hit rate and cost avoided per layer.
Match the layer to the workload. Response caches for repetitive, stable traffic. Prompt caching nearly everywhere, especially for agents.
Put shared caching at the gateway. It sees every request, knows the tenant and the final model, and keeps provider caches warm.

If you want to try it, ZRouter is open source under the MIT license and speaks the OpenAI API, so you can point an existing client at it and turn on each layer one step at a time.

When to Use AI Caching, and When Not To

Saif Ali — Sat, 24 Oct 2026 13:19:30 +0000

When to Use AI Caching, and When Not To

We start with a simple test, go workload by workload, then cover the pitfalls. Examples refer to ZRouter settings where it helps to be concrete.

The three-question test

Before enabling any response cache (exact or semantic), ask:

Do the same requests repeat? If most prompts are unique, a response cache will mostly miss.
Is a reused answer still correct? If the right answer depends on time, live data, or the specific user, reuse is risky.
Is a reused answer acceptable? Some products need variety (creative writing, brainstorming). Others need consistency (policy answers).

If the answers are yes, yes, and yes, use a response cache. If any is no, skip response caching for that traffic and lean on provider prompt caching instead, which never reuses answers and is safe almost everywhere.

Workload by workload

FAQ bots and support assistants

Use: exact cache, semantic cache, prompt caching.

This is the best case. Questions repeat constantly, answers are stable, and consistency is a feature. Start the semantic threshold high and lower it only when you have evidence. ZRouter's default threshold is 0.92, which is a sensible starting point. Set a TTL that matches how often your underlying content changes. If your help center updates weekly, a one-hour TTL (ZRouter's semantic default, SEMANTIC_CACHE_TTL=3600) is safe.

Retrieval-augmented generation (RAG)

Use: exact cache, prompt caching, embedding caching. Semantic cache with care.

In RAG, the answer depends on the retrieved documents as much as on the question. An exact cache is safe because the retrieved context is part of the request body, so a document change produces a new key. A semantic cache that matches only on the question could return an answer built from old documents. ZRouter reduces this risk because only the final user message is matched by meaning, while the rest of the request, including injected context in other messages and a window of recent conversation (SEMANTIC_CACHE_MAX_CONV_MESSAGES, default 3), still has to match. Even so, keep TTLs short when your corpus changes often.

Embedding caching pays off when you re-index. Identical chunks hit the exact cache on /v1/embeddings instead of being billed again.

Coding agents and multi-step agents

Use: prompt caching. Avoid response caching.

Agents send long, stable prefixes (instructions, tool schemas, file contents) followed by new, specific steps. Each response must reflect the latest state of the task, so a reused answer is almost always wrong. But the prefix is the perfect target for prompt caching, and on long sessions it is where most of the money goes.

Two things decide whether prompt caching actually hits:

Prefix stability. Put static content first and changing content last. A timestamp at the top of a system prompt breaks the cache on every request.
Routing stability. The provider cache lives on a specific provider, model, and often API key. If your router load-balances turn 5 to a different target than turn 4, the cache is cold.

ZRouter handles the second point with session keeping: requests in a detected session (for example from Claude Code, Codex CLI, or OpenCode session headers) stay on the same target and the same provider key.

Chat apps with personal context

Use: prompt caching. Response caching only for non-personal requests.

If answers mention the user's name, account, or history, a shared response cache is a data leak waiting to happen. Make sure any response cache is scoped per user or per account. ZRouter includes the authenticated access scope and the server-derived user path in every exact and semantic cache key. Customer API keys are bound to /accounts/ACCOUNT_ID, and client headers cannot override that binding, so two accounts never share cached answers, even for identical prompts.

Batch and offline jobs

Use: exact cache.

Evaluation runs, classification over a dataset, and nightly summaries often re-run the same inputs after a crash or a code change. An exact cache makes reruns cheap. Since timing does not matter, consider a long TTL for the duration of the job.

Creative and generative features

Avoid: response caching.

If users click "regenerate" to get a different poem, serving the same poem defeats the feature. Bypass the cache for those requests. In ZRouter, send Cache-Control: no-cache or Cache-Control: no-store on any request that should skip both response cache layers.

Real-time or time-sensitive answers

Avoid: response caching, or use a very short TTL.

"What is the status of my order?", "What is the BTC price?", and "Is the API down?" need live data. Fresh tool calls and retrieval are the right answer here, and prompt caching can still reduce the cost of the static instructions.

Quick reference

Workload	Exact	Semantic	Prompt cache	Embedding cache
FAQ / support	Yes	Yes, high threshold	Yes	Optional
RAG	Yes	Careful, short TTL	Yes	Yes
Coding agents	Rarely useful	No	Yes, most value	No
Personal chat	Only if scoped	Only if scoped	Yes	No
Batch jobs	Yes	Rarely	Yes	Yes
Creative	No	No	Yes	No
Real-time data	No, or short TTL	No	Yes	No

Pitfalls that cause wrong answers

Semantic thresholds that are too loose

At a low threshold, "cancel my subscription" and "renew my subscription" can look similar. The cost of one wrong hit is usually higher than the savings from many right ones. Start strict, monitor wrong-hit reports, and loosen slowly.

Ignoring parameters

Two requests with the same prompt but different models, temperatures, tools, or response formats are different requests. A cache that keys only on the prompt text will serve a JSON-mode answer to a plain-text request. ZRouter's semantic keys include all top-level request options and provider extensions, and its exact keys include the resolved provider and model, so a change in any of them is a miss.

Caching across policy changes

If you add a guardrail that redacts personal data, cached responses from before the change were never checked by it. ZRouter includes the resolved guardrail chain in the exact cache key. Editing, adding, or removing a guardrail step produces a miss rather than replaying a response the new policy never saw.

Caching failures and fallbacks

Never cache errors. Be careful about caching answers from a fallback model under the primary model's key. ZRouter does not cache failed responses, and it does not cache responses served through failover.

Leaking data between tenants

The most serious pitfall. Any shared response cache must key on the tenant. If you run multi-tenant infrastructure, verify this before enabling caching, not after. Also check how bring-your-own-key traffic is handled. In ZRouter, BYOK requests bypass the shared response cache entirely.

Stale answers after content changes

Pick a TTL that matches how often the truth changes. When you ship a big content update, expect old entries to live until they expire, or clear the cache.

Unstable prompt prefixes

For prompt caching, small changes at the start of a prompt (a date, a random ID, a reordered tool list) silently disable the cache. If your prompt-cache hit rate is near zero, look at the first few hundred tokens of consecutive requests and diff them.

Key takeaways

Use response caching for repetitive, stable, non-personal traffic.
Use prompt caching almost everywhere, and especially for agents and long conversations.
Scope every response cache to the tenant and include every parameter that changes the answer in the key.
Give users and developers a way to bypass the cache per request.

Next in the series: [Part 4: Where to Put Your AI Cache]

The Real Benefits of AI Caching: Cost, Speed, Limits, and Reliability

Saif Ali — Fri, 23 Oct 2026 10:11:19 +0000

The Real Benefits of AI Caching: Cost, Speed, Limits, and Reliability

In Part 1 we described the four layers of AI caching: exact response caching, semantic caching, provider prompt caching, and embedding caching. This post answers the obvious follow-up: what do you actually get for turning them on?

The short answer is five things: lower cost, lower latency, more headroom under rate limits, better resilience, and more consistent answers. Below we go through each one, with worked examples you can adapt to your own traffic. We use ZRouter to show how each benefit shows up in practice.

A note on the numbers: provider prices change often, so the examples use round, illustrative figures. Plug in your own provider's current rates to get real estimates.

1. Lower cost

Response caching removes calls entirely

A response cache hit costs nothing at the model provider. Your savings are simply your hit rate multiplied by your spend on those requests.

Worked example. A customer support assistant handles 100,000 requests a month. Each request averages 1,500 input tokens and 300 output tokens. Assume a model priced at $2.50 per million input tokens and $10 per million output tokens.

Input: 100,000 × 1,500 = 150M tokens → $375
Output: 100,000 × 300 = 30M tokens → $300
Total: $675 per month

Support traffic is highly repetitive. If the exact cache catches 15% of requests and the semantic cache catches another 20%, then 35% of calls never reach the provider:

Model spend: 65% × $675 = $439
Semantic lookups: about 85,000 embedding calls (every request that missed the exact cache) at roughly 20 tokens of final user text each. That is around 1.7M embedding tokens, which costs cents at typical embedding prices.

That is about $236 saved per month on one assistant, before counting the latency improvement.

Prompt caching shrinks the repeated part

For workloads where every answer must be fresh, like agents, coding tools, and long chats, response caching rarely hits. Prompt caching still helps, because most of each request is the same prefix.

Worked example. A coding agent session runs 40 turns. Each turn sends a 25,000-token stable prefix (system prompt, tool definitions, project files) plus a growing conversation. Ignore the conversation growth and focus on the prefix:

Without prompt caching: 40 × 25,000 = 1M input tokens at full price.
With prompt caching: the first turn pays full price (some providers add a small write premium). Turns 2 to 40 read 975,000 prefix tokens at the provider's cached rate.

Providers discount cached input heavily. Depending on provider and model, cached input tokens are billed at roughly 10% to 50% of the normal input price. At a 90% discount, the prefix cost falls by close to 85% across the session. At a 50% discount, it still falls by close to half.

The catch is that you only get this if the prefix is actually identical every turn and the request reaches the same provider, the same model, and often the same API key. That is harder than it sounds, and it is one of the main reasons to handle it at a gateway (see Part 4). ZRouter's prompt-cache planner adds the right directive for each provider automatically, and its session keeping feature pins every request in a detected session to the same target and the same provider key so the cache stays warm.

2. Lower latency

A model call has two slow parts: reading the prompt (time to first token) and writing the answer (generation time).

Response cache hits skip both. A Redis lookup takes a few milliseconds. A typical chat completion takes one to ten seconds. For the user, a cached answer feels instant.
Semantic cache hits add an embedding call and a vector search before the answer, usually tens of milliseconds, which is still far faster than generating.
Prompt cache hits cut time to first token, because the provider does not reprocess the long prefix. On very long prompts this can be the difference between a two-second pause and a near-immediate start.

Latency matters beyond user patience. Agents chain many calls together. Saving a second on each of 40 steps saves 40 seconds on one task.

In ZRouter, streaming responses are cached as the raw event stream, so a cached streaming answer still arrives as a stream. Your chat UI renders it the same way, just much faster.

3. More headroom under rate limits

Providers limit requests per minute and tokens per minute. When you hit a limit, requests fail or queue.

Every response cache hit is a request that does not count against your provider limits. Every prompt cache hit can reduce the input tokens that count against token-per-minute limits on providers that discount cached tokens in their limit accounting.

A subtler benefit comes from request coalescing. Picture a product launch, where hundreds of users ask the same question in the same minute. A plain cache misses on all of them until the first answer returns. ZRouter coalesces concurrent identical misses in each gateway process: one request goes to the provider, and the rest wait and replay its answer. A spike of identical traffic becomes one provider call instead of hundreds.

4. Better resilience

A cache is also a buffer against provider trouble. During a brief outage or a period of heavy throttling, popular questions can still be answered from cache because the lookup happens before the request is sent anywhere.

ZRouter is careful about one edge case here. If a request is answered by a failover target rather than the primary target, the response is not cached. Otherwise the backup model's answer would be stored under the primary model's key and served long after the primary recovered. Entries stored before the outage keep serving until they expire.

5. More consistent answers

Language models do not always give the same answer to the same question. For some products, that variation is a problem: a pricing FAQ, a policy explanation, or a compliance answer should not change from one user to the next.

A response cache makes repeated questions return the same reviewed answer until the entry expires. That makes support quality easier to audit and reduces "the bot told me something different yesterday" tickets.

What caching does not buy you

To keep expectations honest:

It does not make novel requests cheaper. A brand new question with a brand new context gets full price, except for any cached prefix.
It does not improve answer quality. A cache replays answers. A bad answer cached is a bad answer served many times, which is why expiry (TTL) matters.
Semantic caching is not free. Each miss still pays for an embedding and a vector search. If your traffic is almost never repetitive, those lookups are pure overhead.

Measuring the benefit

Turning a cache on is easy. Knowing whether it pays off takes data. Track at least:

Hit rate per layer (exact, semantic, prompt).
Cost avoided: provider spend you would have paid without the hits.
Latency for hits versus misses.
Wrong-hit reports for the semantic layer, which tell you if your threshold is too loose.

ZRouter records each cache hit as a usage entry tagged cache_type: exact or cache_type: semantic, with the caller's user path. By default, usage totals only count provider-bound requests so your cost reports match your provider bill. Add cache_mode=all to include hits, or cache_mode=cached to see only hits:

# Provider spend only (default)
curl "https://your-gateway/admin/usage/summary" \
  -H "Authorization: Bearer $AIGATEWAY_MASTER_KEY"

# Only cache hits
curl "https://your-gateway/admin/usage/summary?cache_mode=cached" \
  -H "Authorization: Bearer $AIGATEWAY_MASTER_KEY"

There is also a cache overview at /admin/cache/overview, and every cached response carries an X-Cache header you can log on the client side.

Key takeaways

Response caching saves the entire call and is the biggest win for repetitive traffic.
Prompt caching is the biggest win for long, fresh, multi-turn work like agents and coding tools.
Caching also buys latency, rate-limit headroom, resilience, and consistency.
Measure hit rates and cost avoided per layer, or you are guessing.

Next in the series: [Part 3: When to Use AI Caching, and When Not To]

What Is AI Caching? A Practical Guide to the Four Layers

Saif Ali — Thu, 22 Oct 2026 17:56:30 +0000

What Is AI Caching? A Practical Guide to the Four Layers

Every call to a large language model costs money and time. You pay for each input token and each output token, and you wait while the model reads your prompt and writes its answer. In a busy application, a large share of those calls are repeats: the same help-center question, the same system prompt, the same 20-page document sent with every turn of a conversation.

AI caching is the practice of noticing that work and not paying for it twice.

This is the first post in a four-part series. Here we cover what AI caching is and the different layers it can happen at. Part 2 covers the benefits in numbers, Part 3 covers when caching helps and when it hurts, and Part 4 covers where to put each layer in your stack. Throughout the series we use ZRouter, an open source, OpenAI-compatible AI gateway, as a working reference, because it implements every layer described here in one place.

Caching, but for models

Classic web caching is simple. A request for /logo.png always returns the same bytes, so you store them and serve the copy next time.

AI requests are messier for three reasons:

Outputs are not deterministic. Ask a model the same question twice and you may get two different wordings. Caching an answer means choosing one and reusing it.
Inputs are large and mostly repeated. A chat request carries the full conversation history, a long system prompt, tool definitions, and maybe retrieved documents. The new part of each request is often a few dozen tokens at the end.
The same question has many spellings. "How do I reset my password?" and "password reset steps?" mean the same thing but share almost no characters.

Because of this, AI caching is not one technique. It is a set of layers, each catching a different kind of repetition.

Layer 1: Exact response caching

What it is: Store the complete response to a request. When an identical request arrives, return the stored response without calling the model.

What counts as identical: The request body after normalization, plus anything that changes the meaning of the request: which model it resolved to, which account sent it, and which policies ran on it.

What you save: The whole call. No input tokens, no output tokens, and the response comes back in milliseconds instead of seconds.

Example: A documentation assistant receives "What are your API rate limits?" two hundred times a day, sent by a widget that builds the exact same request every time. After the first call, the other 199 are free.

In ZRouter, the exact cache is backed by Redis and covers /v1/chat/completions, /v1/responses, /v1/messages, and /v1/embeddings. A hit is marked with a response header so you can see it:

X-Cache: HIT (exact)

Two details make the ZRouter version more useful than a naive "hash the body" approach:

Canonicalization. JSON key order and whitespace are normalized before hashing, so two clients that format the same request differently still share one entry.
Request coalescing. If ten identical requests arrive at the same moment and none is cached yet, only one goes to the provider. The other nine wait and replay its answer. This prevents a burst of traffic from turning into a burst of identical paid calls.

Streaming is handled too. A streaming miss stores the raw server-sent event bytes, and a streaming hit replays them as text/event-stream, so your client code does not change.

Layer 2: Semantic caching

What it is: Store responses along with an embedding (a vector representing meaning) of the prompt. When a new prompt arrives, embed it, search for a stored prompt whose vector is close enough, and if one is found, return its response.

What counts as a match: Similarity above a threshold, usually expressed as cosine similarity between 0 and 1. A high threshold such as 0.92 means "almost the same question". A lower threshold catches more paraphrases but risks wrong answers.

What you save: The model call, at the price of one embedding call and one vector search. Embeddings are typically far cheaper than generation, so the trade is usually good for repeated question patterns.

Example: A support bot gets "how do I reset my password", "I forgot my password, what now?", and "password reset link not working?" The first two are the same need and can share an answer. The third is a different problem, and a well-tuned threshold keeps it separate.

ZRouter's semantic layer uses an embedding provider you choose plus a vector store: Qdrant, pgvector, Pinecone, or Weaviate. A hit is labelled:

X-Cache: HIT (semantic)

ZRouter is deliberately conservative about what it matches on. Only the final user message is compared by meaning. Everything else (model, temperature, tools, other request options, and a limited window of conversation context) must match exactly as part of the key. That way a semantic hit never crosses from one model or configuration to another. It also never applies semantic matching to /v1/embeddings, because an embedding has to represent the exact text it was requested for.

Layer 3: Provider prompt caching

What it is: The model provider keeps the processed form of a long, repeated prompt prefix in memory. When your next request starts with the same prefix, the provider skips reprocessing it and bills those tokens at a discount.

What counts as a match: A shared prefix. The system prompt, tool definitions, and earlier conversation turns must be byte-identical and appear first. Anything after the first difference is processed normally.

What you save: Part of the input cost and part of the time to first token. Output is still generated fresh, because the question at the end is new.

Example: A coding agent sends a 30,000-token system prompt and project context on every turn. With prompt caching, turns two through fifty reuse that prefix. The new tokens at the end are billed normally, and the cached prefix is billed at the provider's reduced cached-input rate.

This layer is different from the first two. The response is not reused at all. You still get a new, fresh answer for every request. What gets reused is the model's work of reading the prompt.

Each provider exposes it differently. Some cache automatically, some need a cache key, some need explicit breakpoints, and some need you to create a cached-content object ahead of time. ZRouter handles this with a provider prompt-cache planner that is on by default. After it knows which provider and model will serve a request, it adds the right directive:

OpenAI: a stable prompt_cache_key, plus explicit breakpoints on models that support them.
Anthropic: automatic ephemeral caching.
Amazon Bedrock (Converse): a cache point after the stable prefix.
Google Gemini (AI Studio): a cached-content object, created only after the same prefix is seen again within five minutes.

If your client already sends its own cache directives, ZRouter leaves them alone.

Layer 4: Embedding caching

What it is: Store the vector for a piece of text so you do not embed the same text twice.

What you save: Embedding calls. Each one is cheap, but RAG pipelines and semantic caches make a lot of them.

ZRouter uses this in two ways. Requests to /v1/embeddings participate in the exact cache, so re-indexing the same documents does not re-bill you. Inside the semantic cache, identical lookup text is reused in memory for five minutes within the same account and embedder, and concurrent identical lookups share a single embedding call.

A fifth layer you do not control: the KV cache

Inside the model server, every transformer keeps a key-value (KV) cache of the tokens it has already processed in the current generation. It is the reason the second word of an answer comes out faster than the first. Provider prompt caching is essentially that KV cache, kept alive between your requests. You do not configure it directly, but it explains why prompt structure matters so much, which we cover in Part 3.

The layers side by side

Layer	What is reused	Match rule	Saves	Answer is fresh?
Exact response cache	Full response	Identical request	Entire call	No
Semantic cache	Full response	Similar final prompt, same everything else	Entire call, minus one embedding	No
Provider prompt cache	Processed prompt prefix	Identical leading tokens	Part of input cost and latency	Yes
Embedding cache	Vectors	Identical text	Embedding calls	Not applicable

The layers stack. A request can miss the exact cache, miss the semantic cache, and still get a prompt-cache discount from the provider. In ZRouter, all of them run in one gateway, in that order, after routing, workflows, and guardrails have produced the final request.

Key takeaways

AI caching is a stack of layers, not a single switch.
Response caches (exact and semantic) skip the model entirely but reuse an old answer.
Prompt caching keeps every answer fresh while cutting the cost of the repeated prefix.
The right mix depends on how repetitive your traffic is and how much freshness matters.

Next in the series: Part 2: The Real Benefits of AI Caching, where we put numbers on cost, latency, rate limits, and reliability.

AI-Powered Code Editors: Build, Fix, and Deploy Apps with AI Builder and CLI

Saif Ali — Wed, 21 Oct 2026 09:42:17 +0000

AI-Powered Code Editors: Build, Fix, and Deploy Apps with AI Builder and CLI

You ask an AI coding tool to build a feature. The first screen looks promising. Then you edit a component locally, the browser preview shows an older version, and the next AI change no longer matches the files you just tested.

That is where a good AI development workflow earns its place. You need a clear way to move between prompts, editable files, checks, preview, and deployment.

This guide shows how to use AI-powered code editors alongside NEXUS AI Builder and its CLI. You will create a small app, work on it in your editor, capture a failed build, review an AI fix, and deploy the reviewed Builder snapshot.

The commands below are real NEXUS AI CLI commands. The scenario is a worked example, rather than a claim that a customer completed these steps or that this particular app has already been generated and deployed.

What are AI-powered code editors?

An AI-powered code editor adds assistance to the place where you read and change source files. Depending on the product, it may offer inline suggestions, code explanations, project search, edits across files, or an agent that runs commands and checks its work.

For example, Cursor Agent can edit code and run terminal commands. AI in Visual Studio Code supports workflows for understanding code, fixing bugs, and building features.

The useful question is how those capabilities fit your task. Can you inspect the generated code? Run your own tests? Keep the preview synchronized? Identify the files that will be deployed?

Where the editor, AI Builder, and CLI fit

Tool	Its role in this workflow
Code editor	Read files, make precise edits, review diffs, and use your preferred coding assistant
NEXUS AI Builder	Generate and revise a project, inspect its browser preview, and save shared checkpoints
NEXUS AI CLI	Bring Builder files to your machine, sync changes, run the review workflow, and submit a deployment

You can keep your preferred editor. The CLI connects your local work to the Builder project, while the browser gives you a place to check the interface and behavior.

The app we will build

Imagine you are making a support-ticket dashboard prototype. You need to show a teammate a working ticket list, a creation form, search, and status changes.

For this first version, tickets will be saved in browser localStorage. That keeps the local setup focused on the editor and Builder workflow. It also gives the prototype a clear limit: records belong to that browser and origin. They are not shared across users or synchronized between preview and deployment URLs.

The acceptance checks are simple:

A ticket requires a nonempty title.
Search finds tickets regardless of capitalization.
A ticket can move from Open to In Progress to Resolved.
Saved tickets remain after refreshing the same browser URL.
Type checks, tests, and the production build pass.

Later, we will cover what changes when the prototype needs a shared database and authentication.

Step 1: Install the CLI and confirm access

You need Node.js 18 or later, a NEXUS AI account, an available AI provider for Builder, and a local editor. The generated app may require a newer Node.js version; inspect its package and setup instructions before running it.

In your normal terminal:

npm install -g nexusapp-cli@latest
nexus --version
nexus auth login
nexus auth whoami
nexus builder providers

nexus auth login opens browser sign-in. whoami confirms the account and organization you will work in. builder providers shows the AI providers available to your account.

AI requests use the applicable Builder allowance or provider configuration. Choose a provider and model available to your account rather than copying a model name from an old tutorial.

Step 2: Create an app with specific requirements

Run this command from the parent directory where you want to work:

nexus builder new --name support-desk-demo \
  "Build a Next.js App Router and TypeScript support ticket dashboard. Include a ticket list, title and description fields, priority, and Open, In Progress, and Resolved statuses. Add case-insensitive search and required-title validation. For this prototype, use seeded example tickets and browser localStorage, with no backend or database. Handle server rendering safely. Add working npm scripts named typecheck, test, and build. Include tests for title validation and case-insensitive search, plus README instructions for setup and local use."

The CLI creates a project and prints its project ID. Copy that actual ID, then set a shell variable:

PROJECT_ID='REPLACE_WITH_THE_RETURNED_PROJECT_ID'

This is an ordinary shell variable used in the remaining commands. It is not an AI model ID or a deployment ID.

The prompt defines a small, inspectable product and asks for checks that match its behavior. Review whether the generated project meets those requirements before continuing.

Step 3: Inspect the files and browser preview

nexus builder files --project "$PROJECT_ID"
nexus builder open --project "$PROJECT_ID"

files lists the current Builder files. open prints the browser Builder URL; open that link yourself.

Try the ticket form, search, status changes, and a page refresh. A plausible screenshot does not confirm that those interactions work.

Use the browser Builder for visual iteration. Depending on the preview mode and project, you can inspect the app beside its conversation and use the available visual tools. Keep the application behavior in your acceptance checks, even when a change is mostly visual.

The NEXUS AI documentation describes the browser Builder and its preview workflow.

Edit code directly inside AI Builder

You can also stay in the browser and open Builder's Code tab. Its built-in editor lets you read and change generated files, create or delete files, search and replace, and save with Cmd+S or Ctrl+S. Saved changes become checkpoints.

Use this for a precise change while keeping the preview nearby. For example, inspect the ticket form component and adjust its empty-title error message, then save and check the form again. The AI conversation and direct file editing work on the same project.

The browser editor is useful when you want to avoid a local setup. Pulling the project in the next step gives you access to your own editor, installed development tools, and local commands.

Step 4: Pull the project into your code editor

Download the Builder snapshot into a new local folder:

nexus builder pull --project "$PROJECT_ID" --out ./support-desk-demo
cd support-desk-demo

Open that folder in Cursor, Visual Studio Code, or your preferred editor. Read the generated README.md and package.json first.

List the available package scripts and install dependencies:

npm run
npm install

The example prompt requested typecheck, test, and build. If any are missing, have Builder add the corresponding working scripts and checks, then sync the updated project. Treat a missing test command as unfinished setup, rather than treating the absence of a failure as a pass.

The local folder also contains Builder checkout metadata. Keep it so the CLI can identify the project and compare local changes with the shared snapshot.

Make one focused editor change

For example, ask your editor's AI assistant:

Review the existing ticket search implementation. Make search ignore
leading and trailing whitespace as well as capitalization. Keep the
current UI and storage behavior. Update the relevant tests and show
which files changed.

You could make the same change manually. Either way, inspect the diff and keep the change focused enough to verify.

Step 5: Test locally and send the changes back

From the pulled project folder:

npm run typecheck
npm test
npm run build

These are project commands requested in the generation prompt, rather than built-in NEXUS commands. Confirm they exist and inspect what they run. If a check fails, use the captured-failure workflow in Step 7 before publishing the change.

Once the changes pass your checks, push them back from this folder:

nexus builder push --project "$PROJECT_ID" --from .
nexus builder check --project "$PROJECT_ID"

push synchronizes local edits with Builder. If it reports a conflict, resolve it before continuing. When files changed on both sides, the CLI can merge nonoverlapping edits and leave conflict markers where manual resolution is needed.

builder check performs static syntax and relative-import checks against the shared Builder files. It does not replace the local type checker, test suite, production build, or browser checks you just ran.

Refresh the Builder preview after a successful push. Repeat the search test with a term containing spaces, such as login.

Step 6: Use live sync when you want a tighter editing loop

For continuous local editing, run this from the pulled project folder:

nexus builder dev --project "$PROJECT_ID" --dir . --open

The CLI starts a live sync session and attempts to open a supported editor if its launcher is available. If it cannot open one, open the folder manually.

You can now edit local files and use Builder chat in the same terminal session. Changes made in the browser Builder can also arrive in the local workspace through sync.

At the dev> prompt, inspect the session:

/status
/diff

Use /status to see local changes, incoming changes, and pending drafts. Use /diff to review the actual differences.

Finish one focused change before having another assistant rewrite the same files. Shared files still need coordination, even when synchronization is automatic.

To stop live sync:

/exit

For a single sync pass instead of a running session, use your normal terminal:

nexus builder sync --project "$PROJECT_ID" --dir .

Step 7: Fix a failed build using its actual output

Suppose a change adds an invalid component prop and the production build fails. A useful AI fix starts with the files and error that produced that failure.

Start interactive Builder chat from the local project folder:

nexus builder chat --project "$PROJECT_ID" --dir .

At the builder> prompt, capture the build:

/run --capture npm run build

If it fails, ask Builder to work from that captured result:

/fix --last-run
/diff
/rerun

Here is what each step does:

Command	Result
`/run --capture npm run build`	Executes the local build and saves bounded output, exit status, and the tested source state
`/fix --last-run`	Sends that failure context to Builder and requests a proposed fix
`/diff`	Shows the changes for review
`/rerun`	Explicitly repeats the captured command against the current local files

The proposed fix is applied locally as a pending draft. The shared Builder snapshot stays unchanged until you accept it. Automatic sync is held while the draft is pending.

If the rerun succeeds, run the other relevant checks before accepting:

/run npm run typecheck
/run npm test

Inspect the outputs. If the fix meets the requirements and the checks pass:

/accept

If it is the wrong change:

/reject

Acceptance saves the local draft as a shared Builder checkpoint, including earlier local edits that are part of that workspace. Review the full diff before accepting.

When the shared Builder project changed while you were testing, acceptance can stop to prevent overwriting newer work. Follow the CLI's instructions, resolve the differences, and retest against the updated files.

The captured-failure workflow is described on the NEXUS AI Builder page. It gives you an explicit sequence: capture, propose, inspect, rerun, and save.

Keep shell commands and Builder commands separate

Where you are typing	Example
Normal terminal	`nexus builder chat --project "$PROJECT_ID" --dir .`
`builder>` or `dev>` prompt	`/run --capture npm run build`
`builder>` or `dev>` prompt	`/diff`, `/accept`, `/reject`

Slash commands belong inside the interactive Builder session. Type /exit to return to your normal terminal before running standalone nexus commands.

Step 8: Review checkpoints and the final preview

Back in your normal terminal:

nexus builder versions --project "$PROJECT_ID"
nexus builder open --project "$PROJECT_ID"

The versions list includes checkpoint message IDs. Keep the ID of a known working state with your project notes.

If you need to restore an earlier Builder file state, choose its actual message ID:

CHECKPOINT_ID='REPLACE_WITH_SELECTED_CHECKPOINT_MESSAGE_ID'
nexus builder revert "$CHECKPOINT_ID" --project "$PROJECT_ID"

Reverting changes the current Builder files and saves a new checkpoint. Sync the result locally and rerun the relevant checks. It does not restore a deployed database or change an already running deployment.

For the ticket prototype, finish with the five acceptance checks from the start of this guide. Check a narrow viewport as well as a desktop screen, and test the empty-result state in search.

Step 9: Deploy the reviewed Builder snapshot

Make sure your local changes have been accepted or successfully pushed before deploying. The deployment command uses the shared Builder snapshot, rather than uncommitted changes sitting only on your laptop.

From your normal terminal:

nexus builder deploy \
  --project "$PROJECT_ID" \
  --name support-desk-demo \
  --provider docker

nexus deploy list --project "$PROJECT_ID"

Here, --provider docker selects NEXUS AI managed deployment infrastructure. It is separate from the AI provider that generated your code.

builder deploy submits a new deployment and prints its ID. It does not run your local checks first, and this command does not have a --wait flag in CLI 4.3.2.

Copy the returned deployment ID:

DEPLOYMENT_ID='REPLACE_WITH_THE_RETURNED_DEPLOYMENT_ID'
nexus deploy status "$DEPLOYMENT_ID" --watch

After inspecting the status, press Ctrl+C to stop watching. For a build failure, inspect build logs. For an app that starts but behaves incorrectly, inspect runtime logs:

nexus deploy logs "$DEPLOYMENT_ID" --type build --lines 100
nexus deploy logs "$DEPLOYMENT_ID" --type runtime --lines 100

Open the actual public URL from the status output. Create a ticket, search for it, change its status, and refresh the page.

Because this prototype uses localStorage, you will create fresh demo records on the deployed origin. Records you made on the preview origin will not appear automatically.

Checkpoint: the deployed app passes the same user checks as the reviewed version. A submitted deployment request alone is not confirmation that the app works.

The CLI Builder guide covers project creation, file editing, checkpoints, preview links, and snapshot deployment.

When the prototype needs a shared database

Once several people need access to the same tickets, browser storage no longer meets the requirements. Treat that as a new feature with its own review and checks.

Ask Builder to plan the data change before implementing it:

Turn this ticket prototype into a team app using a backend and
PostgreSQL through Prisma. Replace browser-local ticket storage with
server-side persistence. Keep the current search and status behavior.
Explain the schema, migration, environment requirements, and tests
before changing files. Do not deploy the change yet.

For a persistent-data app, Builder supports a Next.js and Prisma workflow. Add authentication and authorization as a separate follow-up so you can verify which users may access which tickets.

Database work introduces new acceptance checks: records must persist across sessions, permissions must hold, migrations must run against the intended database, and preview data must be distinguished from deployed data. Follow the generated environment instructions and configure credentials through the platform's secure settings.

Before migrating an existing database, take a backup and test the recovery procedure. See the companion database backup and migration guide, including its current engine-specific caveats.

Already have a Git repository from your AI code editor?

Use the source deployment path when your existing repository is the release source. You do not need to create a duplicate Builder project just to deploy it.

After committing and pushing the code you intend to deploy, replace the repository URL and branch below:

nexus deploy source \
  --repo https://github.com/YOUR_USERNAME/YOUR_APP.git \
  --branch main \
  --name editor-built-app \
  --provider docker \
  --environment DEVELOPMENT \
  --wait

This is an alternative to the Builder snapshot deployment in Step 9. Select the path that matches the source you want to release.

For a private repository, configure repository access in NEXUS AI. For a database-backed app, configure its required database, environment, and migrations as part of that deployment setup.

The NEXUS AI developer workflow covers working in an editor, using GitHub, and deploying through developer tools.

Common problems and the next useful step

Problem	What to do
The preview shows older code	Confirm the local edits were pushed or accepted, inspect `/status`, and refresh the preview.
An AI fix changed more than expected	Inspect `/diff` before acceptance. Reject the draft if it does not meet the requested scope.
`check` passes but the build fails	Capture the actual build with `/run --capture npm run build`; static syntax checks cover a narrower set of problems.
A draft blocks sync or another prompt	Review it, then use `/accept` or `/reject`.
The files contain merge markers	Resolve the conflicting edits, rerun checks, and sync the resolved files.
The AI provider list is empty	Check account configuration and available providers with `nexus builder providers`.
A project script is missing	Inspect `package.json` and ask for a working script and its checks before using it.
The deployed version misses an edit	Confirm whether you deployed a shared Builder snapshot or a Git branch, then inspect that source.
The terminal rejects `/diff`	Enter Builder chat first; slash commands do not run directly in your shell.

Frequently asked questions

Do I need to switch code editors?

No. Pull the Builder project and edit the local files in your preferred editor. Use push or live sync to return the changes to Builder.

Is AI Builder the same as an AI code editor?

They overlap in generating and editing code, but serve different parts of this workflow. The editor gives you direct local file work. Builder maintains the shared project, conversation, and browser preview. The CLI connects those files and release actions.

Does every AI change go straight into the shared project?

In interactive chat with a local checkout attached, proposed file changes use the pending-draft workflow. You review and accept them before they become the shared checkpoint. Other Builder operations, such as browser edits or standalone snapshot commands, have their own effects; check the source you are changing.

Can I choose a different AI provider or model?

In interactive Builder chat or dev mode, /provider and /model show available choices. Access depends on your account and configured providers. These settings control code generation, while the deployment provider selects where the app runs.

Does a Builder checkpoint replace Git?

Use checkpoints to revisit Builder file states. Keep Git for the repository history, branches, review, and CI workflow your team needs. Neither a source checkpoint nor a Git commit is a database backup.

What should I check before sharing the app URL?

Confirm deployment status, inspect relevant logs, and complete the main user journey on the public URL. For apps with a database or authentication, include persistence and access-control checks.

Start with one feature you can verify

Choose a small app or a single change in an existing project. Write down the expected behavior, build the first version, and keep the files visible in your editor.

Then use the same loop each time: inspect the change, run the relevant checks, save the reviewed version, and verify it after deployment. Explore NEXUS AI Builder when you are ready to connect that workflow to your next app.

AI Coding: What Matters Most Before Your Next Database Migration

Saif Ali — Tue, 20 Oct 2026 18:05:39 +0000

Protect your app’s data with one backup and restore workflow for PostgreSQL, MongoDB, MySQL, and Redis.

Author: NEXUS AI Team

Category: Knowledge Base · Databases · CLI

Command syntax verified: NEXUS AI CLI 4.3.2, October 20, 2026

Reading time: 11 minutes

You used AI to write code that changes your database. Before running that migration or releasing the app, you need a backup, a way to test recovery, and a clear path back if the change goes wrong.

This guide explains how to back up and restore your database from the CLI when working with AI-generated code. It covers PostgreSQL, MongoDB, MySQL, and Redis through one NEXUS AI command workflow.

With NEXUS AI, the commands stay familiar whether your app uses PostgreSQL, MongoDB, MySQL, or Redis:

nexus db services
nexus db backup <service-id>
nexus db backups <service-id>
nexus db restore <service-id> <backup-id>

You choose the database service and the backup. NEXUS AI selects the database's native backup and restore tools and uses the service's stored connection settings. Your laptop needs the NEXUS AI CLI; the engine-specific tools run on the platform.

This guide shows how that simplifies release preparation and recovery, using a worked example of an orders application. The commands are real CLI commands. The application names describe an example you can adapt, rather than a reported customer incident.

Which databases this guide covers

Use this workflow for database services running alongside a deployment on NEXUS AI managed infrastructure, such as services added with --services postgres, mongodb, mysql, or redis on the docker provider.

Standalone databases and managed cloud database instances have a separate snapshot workflow under nexus managed-db. An external database connection does not automatically become a deployment-service backup target. See the database backup overview.

Before starting, you need:

Node.js 18 or later and a NEXUS AI account with access to the deployment.
A running database service visible to your current organization.
For the recovery rehearsal, a separate test deployment with a database of the same engine and a compatible version.

Why developers only need one command workflow

Without a shared interface, each engine introduces its own utility, connection options, credentials, backup format, and restore procedure. NEXUS AI puts those mechanics behind the database service ID.

Database	What NEXUS AI uses behind the scenes	Your backup command
PostgreSQL	`pg_dump` custom-format backup	`nexus db backup <service-id>`
MongoDB	`mongodump` directory packaged into a compressed backup	`nexus db backup <service-id>`
MySQL	`mysqldump` SQL backup	`nexus db backup <service-id>`
Redis	RDB snapshot	`nexus db backup <service-id>`

Listing, scheduling, and downloading also use the same commands. Restore uses the same CLI syntax across engines, with each engine's own data replacement behavior underneath.

You still decide what to back up, where to restore it, and whether the recovered application is correct. The CLI removes the need to assemble a different backup command for each database.

Scenario: protect an orders app before a release

Assume you maintain these two deployments:

Deployment	Purpose
`orders-api`	Production application and its PostgreSQL service
`orders-recovery`	Isolated recovery-test application and a compatible PostgreSQL service in the same organization

Your coding AI has proposed a migration that changes how order statuses are stored. Before running that generated code, you want to save the current data, review the migration, and prove that you can restore the backup into the recovery environment.

Keep the recovery app disconnected from live payments, email delivery, and production job processing. A restored copy contains the data represented by the backup. Use an environment approved for that data, and sanitize it before broader development use.

Step 1: Install the CLI and confirm your account

npm install -g nexusapp-cli@latest
nexus --version
nexus auth login
nexus auth whoami

Confirm that the reported account and organization are the ones that own your deployments. You do not enter individual database passwords for the backup commands.

Step 2: Find the database service IDs

List deployments and their database services:

nexus deploy list
nexus db services orders-api
nexus db services orders-recovery

The service table includes SERVICE ID, TYPE, DEPLOYMENT, STATUS, AUTO-BACKUP, and RETENTION.

Copy the PostgreSQL service ID from each deployment. A service ID identifies the database service; it is different from the deployment ID.

For the commands below, set these shell variables after replacing both example values with your actual service IDs:

PROD_SERVICE_ID='REPLACE_WITH_PRODUCTION_DATABASE_SERVICE_ID'
RECOVERY_SERVICE_ID='REPLACE_WITH_RECOVERY_DATABASE_SERVICE_ID'

The variables are ordinary Bash or Zsh variables used to keep the examples readable. They are not special NEXUS AI settings.

Step 3: Take a backup before the change

nexus db backup "$PROD_SERVICE_ID"
nexus db backups "$PROD_SERVICE_ID"

The first command creates the backup and prints its ID, engine type, size, and creation time. The second lists the service's backups, including their status.

Identify the backup you just created and check that its status is completed. Save that specific ID:

PRE_RELEASE_BACKUP_ID='REPLACE_WITH_COMPLETED_BACKUP_ID'

Keep the ID in your release notes with the service and release you are protecting. A backup taken before the change and a backup taken after a failure serve different purposes.

If you need a precise application-wide checkpoint, coordinate writes while taking it. Backing up several services one after another does not produce an atomic snapshot across all of them.

Step 4: Download a copy if you need separate retention

nexus db backup-download "$PROD_SERVICE_ID" "$PRE_RELEASE_BACKUP_ID"

The CLI saves the backup locally using its original filename. For a custom PostgreSQL backup path:

nexus db backup-download "$PROD_SERVICE_ID" "$PRE_RELEASE_BACKUP_ID" \
  --out ./backups/orders-before-status-change.dump

The default-filename command works across all four engines and keeps the engine's appropriate file extension. Store downloaded files in your approved backup location, outside your source repository.

Downloading creates a local copy. It does not test that the backup can be restored.

Step 5: Rehearse recovery in the separate database

If the recovery database contains data you may need, back it up first:

nexus db backup "$RECOVERY_SERVICE_ID"

Then restore the production backup into that recovery database:

nexus db restore-to "$RECOVERY_SERVICE_ID" "$PRE_RELEASE_BACKUP_ID"

Read the confirmation prompt and confirm the target. restore-to writes into the selected target service. The source database is not the restore target.

The source backup and target service must belong to the same organization and use the same database engine. Use a compatible engine version as well. This command does not convert a PostgreSQL backup into MySQL or another engine.

Check the recovery app:

nexus deploy status orders-recovery
nexus deploy logs orders-recovery --type runtime --lines 100

Open its URL and verify the application behavior. For this orders example:

Open several known orders and confirm their customer, total, and status.
Confirm the expected orders appear in the relevant filtered lists.
Perform a controlled test write with external side effects disabled.
Refresh the page and confirm the test change persists.

Database restore success confirms the restore operation finished. These application checks confirm the recovered data works with the app version you intend to run.

Checkpoint: you have a completed pre-release backup and a verified PostgreSQL recovery copy. You can now test the status migration in recovery before releasing it to production.

Step 6: Recover production if the release needs a data rollback

Suppose the migration changes existing statuses incorrectly. First stop application writes and any workers that could modify the affected data, using your application's maintenance controls. Prepare the application version compatible with the old schema and data before reopening traffic.

Do not assume that stopping an entire deployment leaves its database available. Backup and restore need a running database container.

Preserve the current production state before replacing it:

nexus db backup "$PROD_SERVICE_ID"
nexus db backups "$PROD_SERVICE_ID"

Record this new backup ID separately. It preserves the post-change state for investigation or later reconciliation.

Then restore the pre-release backup into its original service:

nexus db restore "$PROD_SERVICE_ID" "$PRE_RELEASE_BACKUP_ID"

Confirm the prompt only after checking the service and backup IDs. In-place restore can overwrite or remove current data. Changes made after the chosen backup may need reconciliation, including orders accepted between that backup and maintenance mode.

Restore the compatible application release through your normal release procedure, check logs, and repeat the order checks before resuming writes. A database restore does not roll back application code, and a code rollback does not restore database data.

The NEXUS AI CLI reference documents the same-service restore and separate-target restore-to commands.

Apply the same commands to MongoDB, MySQL, and Redis

MongoDB restore caveat: source verification for this draft identified a path mismatch when restoring platform-created MongoDB backups. Resolve and test that restore path before relying on it for MongoDB recovery. The CLI command names below are supported, but this guide does not claim a successful live restore test for every engine. The worked recovery example above uses PostgreSQL.

Redis backup caveat: the current implementation waits a fixed interval after requesting a background save, rather than checking that the save completed. Verify snapshot freshness and recovery on a representative dataset before relying on that backup path.

For a different engine, run nexus db services and copy that database's service ID. The core sequence stays the same:

SERVICE_ID='REPLACE_WITH_DATABASE_SERVICE_ID'

nexus db backup "$SERVICE_ID"
nexus db backups "$SERVICE_ID"

Copy a completed backup ID from the output:

BACKUP_ID='REPLACE_WITH_COMPLETED_BACKUP_ID'

nexus db backup-download "$SERVICE_ID" "$BACKUP_ID"

When you intend to restore that service, after coordinating writes and preserving any current state you need:

nexus db restore "$SERVICE_ID" "$BACKUP_ID"

You do not add a database-engine flag to these commands. NEXUS AI resolves the engine from the service.

What still differs between engines

Engine	What to account for during restore
PostgreSQL	Restore resets the target's `public` schema before loading the backup. Objects and data can be removed.
MongoDB	Restore uses collection drop behavior for collections being restored. Check the recovered documents through the app.
MySQL	Restore imports SQL into the target database. Objects not addressed by that dump can remain.
Redis	Restore replaces the RDB file and stops and starts the Redis container. Allow for interruption and reconnects.

The shared interface does not make every restore an identical byte-for-byte replacement of the whole service. Keep recovery targets compatible, and test the data your application depends on.

For Redis used as a job queue, restoring an earlier dataset can bring back jobs that already ran. Coordinate workers and account for duplicate side effects. If Redis only stores disposable cache data, decide whether recovery should rebuild that cache instead.

Enable daily backups with the same command

After the release, enable a daily backup policy for the chosen service:

nexus db backup-schedule "$SERVICE_ID" --enable --retention 14
nexus db services

Check that the service reports automatic backups as on and retention as 14d. Enable the policy separately for each database service that needs it.

Scheduled backups use a daily interval, with a default retention of seven days and a minimum of seven days. The command above raises retention to fourteen days. It does not set a particular wall-clock execution time.

While the schedule is enabled, the retention window applies to NEXUS AI-created backups, including on-demand backups. Download release checkpoints that you need to retain beyond that window. Uploaded backups are excluded from that automatic rotation.

These are scheduled and on-demand recovery points, rather than continuous point-in-time recovery. Take an on-demand backup before a risky change even when daily backups are enabled. See the backup schedule and retention documentation.

To disable future scheduled backups:

nexus db backup-schedule "$SERVICE_ID" --disable

Other useful commands

Inspect backups as JSON

nexus db services --json
nexus db backups "$SERVICE_ID" --json
nexus db backup "$SERVICE_ID" --json

The last command creates a new backup and returns its record as JSON. It is not a read-only inspection command.

Generate a temporary download link

nexus db backup-download "$SERVICE_ID" "$BACKUP_ID" --share --ttl 900

This creates a link lasting fifteen minutes. Anyone with the link can download the file until it expires. The supported lifetime is 30 to 3,600 seconds.

Skip restore confirmation in a reviewed automation job

nexus db restore "$SERVICE_ID" "$BACKUP_ID" --yes

--yes skips the interactive confirmation. It does not verify that you chose the intended data. Use it only when your automation has already established the service, backup, and maintenance conditions.

Troubleshooting

Problem	What to check
A command or flag is missing	Update `nexusapp-cli`, then run `nexus db --help`.
No database services appear	Confirm the organization with `nexus auth whoami`. Check whether the database is a deployment service or a standalone managed database.
`Service has no running container`	The database must be running for backup and restore. Restore database availability before retrying.
`Backup not found`	Check the organization, source service, and backup ID. In-place `restore` needs a backup belonging to that service.
Engine mismatch during `restore-to`	Choose a target service of the same engine as the backup.
A downloaded file is missing later	Download again while the platform backup still exists. The local download location and platform retention are separate.
Restore finishes but the app fails	Check the app's connection settings, schema compatibility, and runtime logs. A restore does not change the deployed code version.

Quick command reference

The angle-bracket values below are placeholders, not literal shell input.

nexus db services [deployment-name-or-id]
nexus db backup <service-id>
nexus db backups <service-id>
nexus db backup-download <service-id> <backup-id>
nexus db restore <service-id> <backup-id>
nexus db restore-to <target-service-id> <backup-id>
nexus db backup-schedule <service-id> --enable --retention 14

For developers moving between Python, Node.js, SQL databases, document storage, and Redis, this is the practical benefit: one backup workflow to learn and reuse. Find the service, create the backup, retain the recovery point, and rehearse the restore through your application.

Deploy a full-stack Python app with MongoDB using CLI or MCP

Saif Ali — Tue, 20 Oct 2026 11:04:28 +0000

You have a Python app working on your laptop. Now you need somewhere to run it, a database that keeps its data, and a way to find out what went wrong when a request fails.

This guide walks through that next step. You will build a small notes app with a browser interface, a Flask backend, and MongoDB for storage, then deploy it on NEXUS AI.

You can deploy from your terminal with the CLI or ask an AI assistant to deploy through MCP. Both paths use the same application and database setup. Choose one deployment path to avoid creating two copies.

Step 1: Understand what you are deploying

The app has three parts:

Part	What it does
Frontend	An HTML page with a form and a list of saved notes
Python backend	Flask routes that read and write notes through PyMongo
MongoDB	Stores notes outside the app container

The request flow is:

Browser
   |
   | HTTPS
   v
Flask app on NEXUS AI
   |
   | PyMongo connection over the deployment network
   v
MongoDB service

Flask serves the HTML and the API from the same origin, so this example needs one web deployment and no separate frontend build.

We will use NEXUS AI managed infrastructure with provider: docker. Its MongoDB sidecar service supplies the app's connection settings, including MONGO_URI and MONGO_DATABASE. The MongoDB sidecar option is not supported on the Cloud Run, ECS Fargate, or Azure Container Apps deployment paths.

Before starting, have these ready:

Python 3.12 and Git.
Docker for the local MongoDB example.
A GitHub account and an empty repository named python-mongo-notes.
A NEXUS AI account with access to managed container deployments and enough quota for the app and database.
Node.js 18 or later for the NEXUS AI CLI, or an MCP client such as Cursor or Claude Code.

This is a shared public demo: everyone who opens the URL can read and add notes. Use sample content. Add authentication and per-user access controls before storing private information.

Step 2: Create the project

Run these commands in Bash or Zsh on macOS or Linux:

mkdir python-mongo-notes
cd python-mongo-notes
python3 -m venv .venv
source .venv/bin/activate
mkdir templates

Create requirements.txt:

Flask>=3.1,<4
pymongo>=4.10,<5
gunicorn>=23,<24
python-dotenv>=1,<2

Install the dependencies:

python -m pip install -r requirements.txt

These ranges keep the example within the intended major versions. Before maintaining a release, lock the exact versions you have tested.

Create .gitignore:

.venv/
__pycache__/
.env
.env.*

Your final project will contain:

python-mongo-notes/
  app.py
  templates/
    index.html
  requirements.txt
  Dockerfile
  .dockerignore
  .gitignore

Step 3: Add the Python backend

Create app.py:

import os

from dotenv import load_dotenv
from flask import Flask, jsonify, render_template, request
from pymongo import MongoClient
from pymongo.errors import PyMongoError

load_dotenv()

app = Flask(__name__)
app.config["MAX_CONTENT_LENGTH"] = 16 * 1024

client = MongoClient(
    os.environ["MONGO_URI"],
    serverSelectionTimeoutMS=5000,
)
db = client[os.getenv("MONGO_DATABASE", "appdb")]


@app.get("/")
def index():
    return render_template("index.html")


@app.get("/api/notes")
def list_notes():
    documents = db.notes.find().sort("_id", -1).limit(50)
    return jsonify([
        {"id": str(note["_id"]), "text": note["text"]}
        for note in documents
    ])


@app.post("/api/notes")
def create_note():
    data = request.get_json(silent=True)
    if not isinstance(data, dict):
        return jsonify(error="Send a JSON object."), 400
    text = data.get("text")
    if not isinstance(text, str) or not 1 <= len(text.strip()) <= 500:
        return jsonify(error="Enter a note between 1 and 500 characters."), 400
    text = text.strip()
    result = db.notes.insert_one({"text": text})
    return jsonify(id=str(result.inserted_id), text=text), 201


@app.get("/health")
def health():
    try:
        client.admin.command("ping")
    except PyMongoError:
        return jsonify(status="unhealthy", database="unavailable"), 503
    return jsonify(status="ok", database="connected")

MONGO_URI comes from the environment. No database credentials belong in the source code or browser JavaScript. PyMongo manages connection pooling through MongoClient. See the official MongoClient documentation.

The /health route actually pings MongoDB. A working homepage alone does not tell you whether the app can reach its database.

Step 4: Add the browser interface

Create templates/index.html:

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Python + MongoDB Notes</title>
  <style>
    body { font: 18px system-ui; max-width: 640px; margin: 60px auto;
           padding: 0 20px; color: #142033; }
    input, button { font: inherit; padding: 12px; }
    input { width: 60%; }
    button { background: #0a60ff; color: white; border: 0;
             border-radius: 6px; cursor: pointer; }
    li { padding: 12px 0; overflow-wrap: anywhere; }
  </style>
</head>
<body>
  <h1>Your notes, saved with Python + MongoDB</h1>
  <form id="note-form">
    <label for="note">New note</label><br>
    <input id="note" maxlength="500" required>
    <button>Save note</button>
  </form>
  <p id="message" role="status"></p>
  <ul id="notes"></ul>
  <script>
    const form = document.querySelector('#note-form');
    const input = document.querySelector('#note');
    const message = document.querySelector('#message');

    async function loadNotes() {
      const response = await fetch('/api/notes');
      if (!response.ok) throw new Error('Could not load notes.');
      const notes = await response.json();
      const list = document.querySelector('#notes');
      list.replaceChildren();
      for (const note of notes) {
        const item = document.createElement('li');
        item.textContent = note.text;
        list.appendChild(item);
      }
    }

    form.addEventListener('submit', async (event) => {
      event.preventDefault();
      try {
        const response = await fetch('/api/notes', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ text: input.value })
        });
        if (!response.ok) throw new Error('Could not save your note.');
        input.value = '';
        message.textContent = 'Note saved.';
        await loadNotes();
      } catch (error) {
        message.textContent = error.message;
      }
    });

    loadNotes().catch(error => { message.textContent = error.message; });
  </script>
</body>
</html>

The browser sends requests to /api/notes. Only the Python process connects to MongoDB. Using textContent also makes note content display as text instead of executable HTML.

Step 5: Run the complete app locally

Start a local MongoDB container:

docker run -d \
  --name python-mongo-notes-db \
  -p 127.0.0.1:27017:27017 \
  -v python-mongo-notes-data:/data/db \
  mongo:7

This local development database has no authentication and binds only to your machine's loopback interface. The named volume stores data separately from the container. On later runs, start the existing container with docker start python-mongo-notes-db.

Create a local .env file:

MONGO_URI=mongodb://127.0.0.1:27017/appdb
MONGO_DATABASE=appdb

Start Flask:

flask --app app run --port 8000

Open http://127.0.0.1:8000, save a note, and refresh the page. The note should still appear.

In another terminal, check the database connection:

curl --fail-with-body http://127.0.0.1:8000/health

Expected response, with key order allowed to vary:

{"database":"connected","status":"ok"}

Checkpoint: the page loads, a note survives a refresh, and /health reports a database connection. Fix local errors before deploying.

Step 6: Package the app and push it to GitHub

Create Dockerfile:

FROM python:3.12-slim
WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
COPY templates ./templates
EXPOSE 8000
CMD ["sh", "-c", "exec gunicorn --bind 0.0.0.0:${PORT:-8000} --workers 2 --access-logfile - --error-logfile - app:app"]

Gunicorn runs Flask in the deployment container and listens on all container interfaces. Its logs go to stdout and stderr so you can inspect them through NEXUS AI. Flask's development server is for local use; see Flask's deployment guide.

Create .dockerignore:

.git
.venv
__pycache__
.env
.env.*

Push the project to the empty GitHub repository you created. Replace YOUR_USERNAME first:

git init
git add app.py templates/index.html requirements.txt Dockerfile .dockerignore .gitignore
git commit -m "Add Python and MongoDB notes app"
git branch -M main
git remote add origin https://github.com/YOUR_USERNAME/python-mongo-notes.git
git push -u origin main

The explicit file list keeps your local .env out of the commit. For this tutorial, use a public repository containing only demo code. A private repository requires repository access configured in NEXUS AI.

Step 7: Deploy using the CLI

If you prefer an AI assistant, skip to Step 8. You only need one deployment path.

Install the CLI and sign in:

npm install -g nexusapp-cli@latest
nexus auth login
nexus auth whoami

Deploy the GitHub repository with its MongoDB service:

nexus deploy source \
  --repo https://github.com/YOUR_USERNAME/python-mongo-notes.git \
  --branch main \
  --name python-mongo-notes \
  --provider docker \
  --environment DEVELOPMENT \
  --framework python \
  --dockerfile Dockerfile \
  --services mongodb \
  --wait

Here is what matters in that command:

Flag	Purpose
`--dockerfile Dockerfile`	Builds the app using your repository's Dockerfile
`--provider docker`	Selects NEXUS AI managed container infrastructure
`--services mongodb`	Adds the MongoDB sidecar and injects its connection environment
`--environment DEVELOPMENT`	Labels this tutorial deployment as development
`--wait`	Waits for the deployment to reach a terminal build/run state

Do not pass your local .env file to this deployment. Its 127.0.0.1 address points to the app container in the cloud, which is a different machine from your laptop. The platform supplies the database connection instead.

Save the deployment ID from the output. Inspect its status and public URL:

nexus deploy status python-mongo-notes

For command details and version-specific options, see the NEXUS AI CLI reference.

Step 8: Deploy using MCP instead

MCP, the Model Context Protocol, lets an AI assistant call NEXUS AI deployment tools. You describe the task, and the assistant sends structured tool calls.

For Claude Code, connect the server:

claude mcp add --transport http nexus-ai https://mcp.nexusai.run/mcp

Complete browser sign-in when prompted. For Cursor, add the same URL in its MCP settings. The NEXUS AI MCP setup guide explains the connection flow.

Allow the deployment read, create, and log scopes needed for this workflow: deployments:read, deployments:create, and deployments:logs.

Then send your assistant this prompt, replacing the repository URL:

Deploy https://github.com/YOUR_USERNAME/python-mongo-notes.git from branch main to NEXUS AI. Name it python-mongo-notes, use the docker provider and DEVELOPMENT environment, build with the repository Dockerfile, and attach the mongodb service. Keep health checks enabled. Confirm my account and available providers first. Follow the returned deployment ID until it is running, inspect health, and return its public URL. If deployment fails, inspect the build and runtime logs. Do not print database credentials.

The assistant should first call nexusai_whoami and nexusai_providers_list with empty argument objects. The source deployment tool call then uses these arguments:

MCP tool: nexusai_deploy_source

{
  "repoUrl": "https://github.com/YOUR_USERNAME/python-mongo-notes.git",
  "repoBranch": "main",
  "name": "python-mongo-notes",
  "provider": "docker",
  "environment": "DEVELOPMENT",
  "framework": "python",
  "dockerfile": "Dockerfile",
  "services": ["mongodb"],
  "healthCheckEnabled": true
}

These JSON blocks are MCP tool arguments for your assistant, not terminal commands. Tool names may have a client-specific prefix. If your connection does not expose the services field, update the connection before proceeding with this recipe.

With the returned ID, the assistant checks progress:

MCP tool: nexusai_deploy_status

{
  "deploymentId": "REPLACE_WITH_RETURNED_DEPLOYMENT_ID"
}

Once running, it checks platform health:

MCP tool: nexusai_deploy_health

{
  "deploymentId": "REPLACE_WITH_RETURNED_DEPLOYMENT_ID"
}

A queued deployment is not yet a working application. Keep the actual returned ID and follow status through completion.

Step 9: Verify the deployed app and MongoDB

Whichever deployment path you used, copy the actual public HTTPS URL from the deployment output. Set it in your terminal, replacing the example:

APP_URL='https://REPLACE_WITH_YOUR_APP_HOSTNAME'
curl --fail-with-body "$APP_URL/health"

The expected result is status: ok and database: connected. This explicit request tests the application's database check even if platform health checks use another route.

Create a note through the deployed API:

curl --fail-with-body -X POST "$APP_URL/api/notes" \
  -H 'Content-Type: application/json' \
  -d '{"text":"My Python app is live with MongoDB."}'

curl --fail-with-body "$APP_URL/api/notes"

Open the public URL in your browser. The same note should appear. Add another note through the form and refresh.

Checkpoint: the HTTPS page works, the API can write and read notes, and the database health check succeeds. You have verified the full request path from browser to Python to MongoDB.

Step 10: Read logs and deploy updates

If something fails, start with the logs.

CLI:

nexus deploy logs python-mongo-notes --type build --lines 100
nexus deploy logs python-mongo-notes --type runtime --lines 100

MCP tool: nexusai_deploy_logs

{
  "deploymentId": "REPLACE_WITH_RETURNED_DEPLOYMENT_ID",
  "type": "runtime",
  "limit": 100
}

Use "type": "build" to inspect build failures.

Symptom	First thing to check
`ModuleNotFoundError`	The missing dependency is in the committed `requirements.txt`
`KeyError: 'MONGO_URI'`	MongoDB was attached and its environment reached the app
MongoDB connection timeout	The database is running, and the deployed URI was not overwritten with localhost
Authentication failure	The injected database credentials match the existing database; inspect settings without printing secrets
App starts but URL fails	Gunicorn binds to `0.0.0.0` and the configured container port
Latest edits are missing	You committed and pushed them to the deployment's branch

To release an update, commit and push the changed files, then run:

nexus deploy redeploy python-mongo-notes --wait

Or ask your assistant to use nexusai_deploy_redeploy:

{
  "deploymentId": "REPLACE_WITH_RETURNED_DEPLOYMENT_ID"
}

For this repository-based deployment, omit name and provider overrides to use an in-place rebuild. Afterward, repeat the /health and notes checks. Existing notes should still be present; verifying this also catches accidental attachment to a new or empty database.

Before using the app for real users

The tutorial gives you a working deployment foundation. Before collecting real data, add authentication, per-user note ownership, and rate limits. Confirm persistent database storage and a backup policy, then test restoring a backup. App container storage and database backups serve different purposes.

If you choose MongoDB Atlas later, supply its connection URI through secure deployment settings and configure Atlas network access for your deployment's outbound traffic. Atlas permits connections from entries in its IP access list. For MCP workflows, keep credential-bearing URIs out of chat and tool arguments; configure them through the authenticated dashboard.

Start by saving one note, refreshing the page, and seeing it again at your public URL. From there, you can build on an application whose frontend, Python backend, and MongoDB connection have all been checked.

How to Choose the Best AI for Coding in 2026

Saif Ali — Tue, 20 Oct 2026 11:01:56 +0000

The best AI for coding depends on the work you want to finish. Cursor is worth evaluating for an editor-centered workflow, Claude Code for terminal-based development, Codex for agent-assisted repository tasks, and GitHub Copilot for teams working through GitHub and supported editors. NEXUS AI is worth evaluating when you want to build, preview, and deploy a full-stack app in a connected workflow.

If you are new to coding AI, the number of choices can make a simple decision feel complicated. Start with one question: what should the tool help you finish?

Explaining an unfamiliar function, fixing a bug across several files, and getting a database-backed app online are different jobs. This guide explains the differences, compares five options, and gives you a step-by-step way to choose.

The recommendations below are editorial judgments based on documented capabilities. They are not a benchmark ranking or the result of a controlled performance test.

What is coding AI?

Coding AI is software that uses artificial intelligence to help people write, understand, debug, test, and maintain code. Depending on the tool and its permissions, it may suggest a line of code, explain an error, edit project files, or run commands to check its work.

For example, you could ask a coding assistant:

This Python endpoint returns a 500 error when a customer submits an empty email address. Find the cause, add validation, and verify that valid submissions still work.

Useful output would include a focused fix, evidence that the behavior changed, and an explanation you can review.

The main types of AI coding tools

Type	What it helps with	What to check
Chat assistant	Explanations, small examples, and debugging conversations	Whether it has the current project context
Editor assistant	Suggestions and changes while you work in an IDE	Which files it reads and how you review edits
Coding agent	Tasks involving multiple files, commands, and verification	What it actually ran and whether the changes passed review
AI app builder	Creating an application from a description and iterating on a preview	Whether the backend, database, and deployment meet your needs

A model and a coding tool are also different things. The model produces responses; the tool gives it access to files, terminals, project instructions, and other services. Evaluate the complete setup you will use.

Best AI for coding: a practical shortlist

Use this table to pick candidates for your own project. The tools have overlapping capabilities, and the right choice can change with your task, client, and plan.

Tool	Consider it when you want to…	Check before committing
Cursor	Work closely with an agent inside a code editor	Whether its editing and review workflow suits your project
Claude Code	Use an agent through your terminal or preferred development environment	Command permissions, project instructions, and usage limits
OpenAI Codex	Delegate repository work through terminal, IDE, or cloud workflows	Execution environment, available integrations, and how you review results
GitHub Copilot	Keep AI assistance close to GitHub issues, pull requests, and supported editors	Which capabilities your plan, client, and organization allow
NEXUS AI	Build and preview an app, then deploy it with its supporting services	Runtime support, deployment provider, service availability, and resource costs

Cursor: consider it for an editor-centered workflow

Cursor's Agent can search a codebase, edit files, and execute terminal commands. That makes it a reasonable candidate when you want to discuss a change while keeping the relevant code and terminal nearby. These capabilities are described in the Cursor Agent documentation.

Try it on a feature that crosses your interface and backend. Review whether it follows existing conventions and whether its changes are easy to inspect. A convincing explanation should be backed by a working change.

Claude Code: consider it for terminal-based development

Claude Code reads code, edits files, and runs commands. It is available through the terminal and other development surfaces, including IDE integrations. See the Claude Code overview.

It is worth evaluating if your day already involves command-line tools, scripts, Git, and test commands. Give it a bounded maintenance task, then inspect the diff and the verification output. Check how it handles project instructions before giving it broader work.

OpenAI Codex: consider it for agent-assisted repository work

Codex provides terminal, IDE, and cloud ways to work on coding tasks. The CLI can read and change files and run commands in the selected directory; the IDE extension brings an agent into supported editors. See the official Codex CLI and Codex IDE extension guides.

Evaluate it on a task with a clear finish line: reproduce a bug, implement a focused fix, and run the relevant checks. Choose the execution surface that fits your repository and access requirements, then review the resulting changes.

GitHub Copilot: consider it for GitHub-based teamwork

GitHub Copilot offers code suggestions, codebase questions, change review, and agent-assisted tasks. Its available capabilities depend on the client, plan, and organization settings. See GitHub's Copilot documentation.

It deserves a place on the shortlist when your team already organizes development through GitHub. Test the path from an issue to a proposed change and review. Check whether the features you need are available to everyone who will use the workflow.

NEXUS AI: consider it for building and deploying an app

NEXUS AI combines an AI app builder with previews and full-stack deployment tools. Its platform covers application services, databases, workers, storage, logs, and recovery workflows. See the NEXUS AI product overview.

Consider it when your goal includes a running application people can use. You can also bring code from another coding tool: the developer workflow supports editing locally, using GitHub, and deploying through developer tools.

Test the full path that matters to you: create a feature, inspect the preview, save real data, deploy, and read it back. Confirm which services your selected provider supports before choosing the deployment setup.

How to choose a coding AI tool in seven steps

Step 1: Write down one task you actually need done

Avoid starting with a large request such as “build my entire business.” Choose a task you can assess:

Explain how an existing login flow works.
Fix a reproducible validation bug.
Add a searchable list to an existing dashboard.
Build a small app with one database-backed form.

Describe what success looks like. For a form, that might mean valid input saves, invalid input produces a useful error, and the saved entry survives a page refresh.

Step 2: Pick two tools that fit that task

For an existing repository, shortlist tools that work comfortably with its files and checks. For a new app, include a builder that lets you inspect the generated project and verify backend behavior.

Keep the trial manageable. Two tools completing the same realistic task tell you more about your workflow than several unrelated demos.

Step 3: Give each tool the same context

Start from the same repository version and provide the same requirements. Include your framework, relevant files, existing conventions, and verification command.

Here is a reusable prompt:

Goal: Add validation to the customer signup endpoint.
Context: This is an existing Python application. Follow its current style.
Required behavior: Reject missing or malformed email addresses with a
clear client error. Valid submissions must continue to work.
Scope: Keep changes focused on this behavior.
Verification: Reproduce the invalid-input case, implement the fix,
and run the relevant checks.
Report: Explain the changes, what you verified, and any remaining issue.

For a new app, make the data behavior equally explicit:

Build a small customer directory with a browser interface, a backend,
and persistent database storage. Let me add and search customers.
Validate required fields and show useful errors. Verify that a saved
customer remains after refreshing the page. Show me how to run and
deploy the app.

Step 4: Review correctness and scope

Read the changed files. Confirm that the behavior matches the request and that the tool did not introduce unnecessary dependencies or unrelated edits.

Ask it to show the actual verification output. Run the important checks yourself when needed, especially where authentication, payments, or data access are involved. A test that merely repeats an implementation detail may miss the behavior you care about.

Step 5: Test the user journey

Open the app and complete the task as a user would. Submit valid and invalid input, refresh the page, and check that saved data is still available.

For a repository fix, revisit the original failure. For a generated app, confirm that the interface calls a working backend and that the backend reaches its intended database.

Step 6: Compare cost and review effort

Subscription price is only part of the decision. Record usage limits, paid features you require, time spent correcting output, and how much review the change needed.

For an application, also account for hosting, database, storage, and external API costs. Check the vendors' current plan pages before buying; access and limits can change.

Use a simple worksheet:

Criterion	What to record
Correctness	Did the requested behavior pass the relevant checks?
Context	Did it follow your project's conventions?
Scope	Were the changes focused and understandable?
Review effort	How long did you spend checking and correcting the result?
Cost	What did this task consume, and what would repeated use cost?
Delivery	Could you run, deploy, and verify the result?

Prefer a tool whose results you can understand and maintain. Generating more code is useful only when that code helps finish the task.

Step 7: Deploy and verify the result

For an app project, include deployment in your evaluation. NEXUS AI supports deploying repository code from its CLI or through MCP, so you can continue using your chosen coding assistant. Its deployment documentation covers the supported workflows.

For example, the following CLI path deploys a repository with a PostgreSQL service on NEXUS AI managed infrastructure. It assumes your app is committed and pushed, has a supported runtime, and reads its database connection from DATABASE_URL. Replace the repository URL first:

npm install -g nexusapp-cli@latest
nexus auth login

nexus deploy source \
  --repo https://github.com/YOUR_USERNAME/YOUR_APP.git \
  --branch main \
  --name coding-ai-demo \
  --provider docker \
  --environment DEVELOPMENT \
  --services postgres \
  --wait

nexus deploy status coding-ai-demo
nexus deploy logs coding-ai-demo --type runtime --lines 100

Adding a database service supplies infrastructure and connection settings. Your application still needs its database client, schema, and any migrations it requires. The CLI reference documents the commands and options.

If you prefer MCP, connect your compatible assistant to NEXUS AI. For Claude Code:

claude mcp add --transport http nexus-ai https://mcp.nexusai.run/mcp

Complete browser sign-in, then ask:

Deploy my GitHub repository to NEXUS AI managed infrastructure with PostgreSQL in the DEVELOPMENT environment. Confirm account access and available providers first. Keep health checks enabled, follow the returned deployment ID, and inspect logs if it fails. Return the public URL and observed health. Keep credentials out of chat.

The assistant's nexusai_deploy_source tool call would use arguments such as:

{
  "repoUrl": "https://github.com/YOUR_USERNAME/YOUR_APP.git",
  "repoBranch": "main",
  "name": "coding-ai-demo",
  "provider": "docker",
  "environment": "DEVELOPMENT",
  "services": ["postgres"],
  "healthCheckEnabled": true
}

This JSON describes an MCP tool call; it is not a shell command. Choose either the CLI or MCP creation path for this example. Then follow status with nexusai_deploy_status and inspect health with nexusai_deploy_health, passing the actual returned deploymentId to each. See the MCP connection guide.

After deployment, open the actual public URL, complete the main user action, and confirm that saved data can be read back. A running process is one checkpoint; a working user journey completes the evaluation.

Frequently asked questions about coding AI

What is the best AI for coding?

Choose by workflow. Start with Cursor for editor-centered development, Claude Code for terminal work, Codex for agent-assisted repository tasks, GitHub Copilot for GitHub-based teamwork, or NEXUS AI for building and deploying an app. Compare two candidates on the same task before committing.

Can a beginner use coding AI?

Yes. Start with a small feature and ask the tool to explain the files, decisions, and verification steps. Learn enough to understand how the app starts, where it stores data, and what happens when input is invalid. Ask for help when you cannot assess a change.

Is free coding AI enough?

It may be enough for your workload. GitHub offers a Copilot Free option, while access to advanced capabilities depends on the plan and client. Review the current access options, then test whether the allowance fits your real tasks.

Can AI code an entire application?

An AI app builder or coding agent can generate files across an application. You still need to verify the requirements, data access, error handling, dependencies, and deployment. Start with one complete user journey and expand after it works.

Which AI is best for Python coding?

Use the same task-based evaluation. Ask candidates to work with your actual Python framework and dependencies, reproduce a failure, make a focused change, and run the relevant checks. Include database and deployment behavior if those are part of your application.

Can I use one coding AI and deploy somewhere else?

Yes, when you can access the source and meet the destination's runtime requirements. NEXUS AI supports a workflow where you edit locally, push to GitHub, and deploy through developer tools. Check your app's build settings, secrets, and supporting services before moving it.

Put your shortlist to work

Pick a small project you care about and define what “done” means. Give two tools the same task, review their changes, and compare the result you can run.

NEXUS CLI App Builder

Saif Ali — Tue, 20 Oct 2026 10:58:53 +0000

The NEXUS CLI Builder is the terminal interface to the same AI App Builder used on the NEXUS AI website. You describe an app, the Builder creates files in a project, and later prompts revise that project. You can inspect and edit the files locally, check common syntax and import errors, open the browser preview, and explicitly deploy a snapshot. It is not a separate code generator: the terminal and browser use the same Builder session and checkpoints.

This guide follows one app from first prompt to deployment, then covers the commands you need when you want more control.

Who is it for, and when should you use it?
Use the NEXUS CLI Builder if you prefer a terminal workflow, want to make a quick app without first creating a project in the website, or want to combine AI changes with edits in your own code editor. It is also useful when you need a repeatable sequence of commands for an existing Builder project. A terminal beginner can stay in nexus builder chat; developers can pull the files, edit them locally, and push the result.

Use the browser Builder when you need to see and interact with the preview, use its visual editing tools, or inspect the app alongside the conversation. The NEXUS CLI prints a link to that page; it does not render a visual preview inside the terminal. If you already have an independent Git repository rather than a Builder snapshot, the regular source deployment workflow may be a better fit than starting a new Builder app.

The NEXUS CLI Builder does not silently deploy each AI response. Generation, checks, preview, and deployment are separate steps so you can inspect the result first.

Before you start
Install Node.js 18 or later, then install and sign in to the CLI:

npm install -g nexusapp-cli@latest
nexus auth login
nexus auth whoami
nexus auth login opens a browser for sign-in. If you already use a NEXUS AI API token in a noninteractive environment, set NEXUSAI_TOKEN instead. Run nexus --help or nexus builder --help to confirm the commands supported by your installed version.

Your account also needs an available AI provider. To see the provider IDs and whether they are allowed for your account, run:

nexus builder providers
In interactive chat, /provider and /model present numbered choices, so you do not need to memorize IDs. If no provider is available, add or enable one under AI Providers on NEXUS AI Dashbaord. AI requests and automated fixes may use your Builder quota.

Example: build a small issue tracker
Start a new interactive build from your normal terminal:

nexus builder chat
The builder> prompt is inside the chat. Type a description of the app, not a shell command. Your first app prompt creates a NEXUS AI project automatically; you do not need to create one in the UI. For example:

builder> Build an issue tracker for a small team. Include an issue list, status and priority filters, an issue detail view, and a form to add an issue. Use realistic sample data. Keep the layout usable on a phone.
The CLI prints New build: ... (), streams file and tool activity, and reports a saved checkpoint. Keep the project ID. It is the address you use to resume the app and run non-chat Builder commands. The app name and exact files depend on the generated result; this prompt is a concrete starting specification, not a guarantee that every requested behavior is production-ready.

To revise the same app, keep talking at builder>:

builder> Make the issue form validate the title and show an inline error instead of submitting an empty issue.
builder> Add an empty-state message when the selected filters match no issues.
builder> /files
builder> /check
Describe the change you want and its expected behavior. A focused follow-up is easier to inspect than asking for many unrelated features at once. /files lists the current snapshot. /check reports static diagnostics such as a file and line number; it does not run the app.

Type /help at any point for the chat commands. Use Up and Down to revisit earlier prompts and commands in the current CLI session. The history is held in memory and is not saved across restarts.

Choose an AI provider or model
You can choose before the first prompt, or switch while working on an app:

builder> /provider
Choose an AI provider:

Claude ...
OpenAI ... Provider number (Enter to cancel): 1 builder> /model builder> /status The numbered entries reflect providers and models actually available to your account; they may differ from this example. The selection affects subsequent AI messages. Switching providers resets the model to that provider's default until you select another model. /status shows the current project, provider, model, and file count.

Here “AI provider” means the model service generating code. Later, builder deploy --provider ... means the infrastructure provider running the deployed app. They are different settings.

Start another app or resume this one
Inside chat, /new clears the current chat target; your next ordinary prompt creates another project. It does not delete the previous app. /exit leaves Builder chat. If you launched chat from the interactive nexus shell, you return to that shell; otherwise you return to your normal terminal.

To return to the issue tracker after leaving, run this in your normal terminal, replacing the placeholder with the ID printed when you created it:

nexus builder chat --project
For a single prompt without entering chat, use:

nexus builder new "Build a responsive issue tracker with a list, filters, and an add-issue form"
That also creates a project automatically and prints its ID. For an existing project, a one-shot follow-up is:

nexus builder ask --project "Add an empty state to the filtered issue list"
Inspect the files and fix a failed check
Outside chat, the equivalent commands are:

nexus builder files --project
nexus builder check --project
check examines supported text files for syntax errors and checks relative imports against the current Builder files. For example, an error might point to an import in src/App.tsx whose target file does not exist. If the reported issue is appropriate for AI repair, run:

nexus builder fix --project
nexus builder check --project
fix passes the diagnostics to the AI, saves its changes, and checks again. It attempts repair up to twice by default; --attempts 3 changes that limit (maximum 5). Each AI attempt may consume Builder quota. Inspect the result after an automated fix: a passing static check means the checked syntax and relative imports passed, not that the app builds, behaves correctly, is secure, or has passing tests.

If you prefer to guide the fix yourself, ask a targeted question instead:

nexus builder ask --project "The check reports a missing relative import in src/App.tsx. Fix that import without changing the issue form behavior."
nexus builder check --project
Use the path and diagnostic actually printed for your app. After that, open the browser preview and test the affected flow. For serious apps, also run the project's own build, tests, and security review after pulling the code locally.

Edit the generated app in your own editor
Builder files live in the project snapshot. nexus builder dev keeps a local folder in live two-way sync with it, so you can use VS Code, Cursor, or any editor as your IDE while still chatting with the AI:

nexus builder dev --project --dir ./issue-tracker --open
The first run downloads the app into ./issue-tracker (an empty or new folder) and --open opens it in VS Code or Cursor. Then:

Your edits sync up. Save a file and it becomes a new Builder checkpoint within a second or two, so the browser preview shows it.
AI edits sync down. Type a prompt at the dev> prompt (or in the browser Builder) and the changed files appear in your folder. Before each AI turn, your pending edits are saved first so the AI works on your latest code.
Edits on both sides are merged. If you and the AI changed different parts of the same file, both changes are kept. If you both changed the same lines, the file gets git-style markers:
<<<<<<< local

const title = "Recipe Box";

const title = "Cookbook";

nexus
Edit the file to keep what you want and remove the markers. A file with markers is never saved to Builder, and it syncs as soon as the markers are gone. Your other edits keep syncing in the meantime.

At the dev> prompt, /open opens the folder in your editor, /sync syncs immediately, /check and /fix work as in chat, /preview prints the browser preview link, and /exit stops syncing. Press Ctrl+C to cancel a running AI turn.

To sync once instead of continuously (for example in a script), use:

nexus builder sync --project --dir ./issue-tracker
It exits with code 1 when conflicts are waiting for you.

What syncs: text files, including new and deleted files. .git, node_modules, build output, and .env files stay local and are never uploaded. The limit is 200 files and 2 MB of content; symlinks and binary or non-UTF-8 files are not supported. Keep secrets out of Builder files. If most of the folder's files disappear at once, sync stops instead of deleting them in Builder; restore them, or pass --allow-deletes if you really removed them.

Manual pull and push
If you prefer explicit steps, pull copies the snapshot to an empty folder and push saves your edits back:

nexus builder pull --project --out ./issue-tracker
nexus builder push --project --from ./issue-tracker
push refuses to save if Builder changed after your pull. Run nexus builder sync on the same folder instead: it merges both sides. If you removed files that were in the pulled snapshot, push asks you to confirm with --delete:

nexus builder push --project --from ./issue-tracker --delete
Use --delete only when you intentionally removed those files. For a single file, you can skip the pull/push cycle:

nexus builder put src/components/IssueCard.tsx --from ./IssueCard.tsx --project
nexus builder rm src/components/OldIssueCard.tsx --project
The Builder path in put or rm must match the project's file structure. put adds or replaces that file and rm deletes it; neither automatically edits imports elsewhere. Run check after either operation. File changes made through the CLI are shared with the browser Builder.

What if someone edited the app after I pulled it?
push compares the checkpoint recorded by pull with the current project checkpoint. If another Builder chat, browser edit, or file operation changed the app, push rejects your stale copy rather than overwriting those changes. Run nexus builder sync --project --dir on the same folder: it brings in the other changes, keeps yours, and marks any lines you both changed for you to resolve. Do not simply add --delete: that flag confirms missing files, not a stale checkpoint.

Review checkpoints and open the preview
Each successful AI response or file edit produces a Builder checkpoint. To list them:

nexus builder versions --project
The output includes message IDs. If you need an earlier file state, choose the intended ID and restore it:

nexus builder revert --project
Revert saves the restored files as a new checkpoint. It does not erase the intervening history. Review the listed version before reverting because it changes the current snapshot.

To inspect behavior, get the browser Builder URL:

nexus builder open --project
Inside chat, /preview prints the same kind of link. Open it in a browser to run the preview, click through the issue list and form, and make sure the result matches your requirements. open only prints the URL; it does not launch a preview inside the terminal.

Deploy the reviewed snapshot
Deployment is explicit. After reviewing the files, running check, and testing the browser preview, deploy the current Builder snapshot:

nexus builder deploy --project --name issue-tracker --provider docker
nexus deploy list --project
The command creates a deployment from the current files and prints a deployment ID. docker here is a deployment provider, not an AI model. Other providers may be available for your account; for example, --provider gcp_cloud_run selects Google Cloud Run. You can supply --region and --framework when needed. The CLI uses the project's framework hint, or nextjs if none is set, unless you override it. Check the generated app before deployment if that default does not fit.

builder deploy does not automatically run builder check, your package's test suite, or a production build first. It submits a new deployment; it is not an in-place edit of an existing deployment. Deployment uses your normal infrastructure quotas and may incur charges. Check deployment status and logs through the normal deployment commands or dashboard if the build fails.

Quick command map
Goal In builder> chat From the normal terminal
Start a new app First ordinary prompt, or /new then a prompt nexus builder chat or nexus builder new "..."
Continue an app Ordinary prompt nexus builder chat --project or nexus builder ask --project "..."
Get help /help nexus builder --help
Select AI /provider, /model, /status nexus builder providers; start supports provider/model IDs
Inspect files /files nexus builder files --project
Check and repair /check, /fix nexus builder check and nexus builder fix with --project
Open preview /preview nexus builder open --project
Edit locally Leave chat, then use terminal commands pull, edit, push; or put/rm for one file
Review or restore Leave chat, then use terminal commands versions and revert with --project
Deploy Leave chat, then use terminal command nexus builder deploy --project --name --provider
Common sticking points
“Where is my project?” The first Builder prompt creates it automatically. Copy the printed project ID. /status shows the ID during chat; use chat --project to resume later.
“My provider is missing.” Run nexus builder providers. In chat, /provider lists supported providers allowed for your account. Configure one under website Settings → AI Providers if the list is empty.
“check passed, but preview or deployment failed.” Static checks do not execute the app or replace its build/tests. Inspect the browser preview and deployment logs, then fix the actual runtime or build failure.
“push says the Builder changed since pull.” Preserve your edits, pull into a new empty directory, merge, and push that newer copy.
“I typed /help and my shell rejected it.” Slash commands work only at the builder> prompt. From your normal terminal, use nexus builder --help.
“I pressed Up but yesterday's command is gone.” Chat history is only kept for the current CLI process. The Builder project's files and checkpoints persist; use the project ID to resume them.
For broader platform guidance, see the documentation. For exact Builder flags in your installed CLI, run nexus builder --help or nexus builder --help

DEV Community: Saif Ali

Configuration Management in Markdown

Introducing Korta: Configuration Management in Markdown

Then run:

Configuration you can read

You can use labels to select a subset:

Who Korta is for

Why now

Why open source

Where to start

Databases on NEXUS AI: Managed, Portable, and Ready for AI-Built Apps

Databases on NEXUS AI: Managed, Portable, and Ready for AI-Built Apps

What NEXUS AI databases are

Where to use each option

App database service: for prototypes and single apps

Managed database on NEXUS AI: for data shared by several apps

Managed cloud database: for production and compliance

Connected external database: for data you already have

Quick decision guide

The management point of view

Portability: your data is never trapped

Flexibility: one workflow, many shapes

Who it is for

Why NEXUS AI is a strong fit for AI deployments

What to know before you start

Frequently asked questions

What databases does NEXUS AI support?

Can I export my data and leave?

Is it safe to let an AI agent run queries?

What is the difference between nexus db and nexus managed-db?

Can I move from a database on NEXUS AI to Amazon RDS later?

How long are backups kept?

Start with the database your app needs today

Virtual Models: One Model Name, Any Model Behind It

Virtual Models: One Model Name, Any Model Behind It

What is a virtual model?

The problem it solves: model choice is wired into your code

Why virtual models fit today's AI agents and apps

Models change faster than release cycles

Agents call models from many places

Many tools only let you set one model

Reliability needs more than one provider

Cost needs to follow the task

Teams need governance in one place

When to use a virtual model

Route by job: name models after what they do

Routing strategies, and what each is best for

1. Single target (alias)

2. Round-robin (default)

3. Lowest cost

4. Failover

5. Schedule (time of day)

6. Super Intelligence Router (request difficulty)

7. Adaptive (ZRouter Pro)

8. Plugin strategies

Choosing a strategy

Combine strategies by chaining virtual models

More things a virtual model can carry

A practical rollout plan

Key takeaways

Where to Put Your AI Cache: Client, Gateway, or Provider

Where to Put Your AI Cache: Client, Gateway, or Provider

Option 1: Cache in the application

Option 2: Cache at the provider

Option 3: Cache at the gateway

The recommended layout

Setting it up in ZRouter

Step 1: Provider prompt caching (already on)

Step 2: Exact response cache

Step 3: Semantic cache (optional)

Step 4: Send traffic and check the headers

Step 5: Bypass when you need fresh output

Step 6: Keep sessions sticky for agents

Step 7: Measure

What ZRouter handles so you do not have to

Wrapping up the series

When to Use AI Caching, and When Not To

When to Use AI Caching, and When Not To

The three-question test

Workload by workload

What is the difference between `nexus db` and `nexus managed-db`?