DEV Community: Dave Kurian

AI provider portability: build one model boundary without hiding provider differences

Dave Kurian — Tue, 01 Sep 2026 09:21:59 +0000

AI provider portability works when your application owns one stable model boundary while adapters preserve the differences that matter. Do not flatten every provider into a fake universal API. Normalize the inputs and outputs your product actually needs, keep provider-specific options behind an explicit escape hatch, and record which provider produced each result.

That design lets a production app change vendors, add a fallback, or run an evaluation without rewriting every route and screen. It also avoids the opposite failure: scattering provider-specific request shapes through handlers, background jobs, tests, and UI code until a price or availability change becomes a migration project.

What should an AI provider boundary own?

Your boundary should own product behavior, not vendor vocabulary. A useful first version accepts a task, an input, a policy, and an execution context. It returns text or structured output, usage metadata when available, and a typed error when the request fails.

export type AiTask = {
  name: string;
  input: string;
  system?: string;
  schema?: Record<string, unknown>;
  maxOutputTokens?: number;
  providerHint?: string;
};

export type AiResult = {
  text: string;
  provider: string;
  model: string;
  usage?: {
    inputTokens?: number;
    outputTokens?: number;
  };
  raw?: unknown;
};

export interface AiProvider {
  complete(task: AiTask): Promise<AiResult>;
}

The raw field is deliberate. If you throw away the original response, debugging and provider-specific features become harder. Keep it out of normal product logic, redact it before persistence, and expose it only to code that explicitly opts in.

The adapter should translate from this contract into the provider’s request format. OpenAI’s Responses API reference documents response inputs, developer or system instructions, structured input items, background responses, and response output objects. Anthropic’s Messages API reference documents alternating user and assistant turns, a top-level system parameter, content blocks, and a required maximum token limit. Those are similar concepts with different shapes. The adapter is where the translation belongs.

How do you keep the common path small?

Start with the narrowest capability your product needs. If the first feature is “classify a support message into one of four labels,” do not design a universal abstraction for every tool call, image input, citation, and streaming mode on day one.

const task: AiTask = {
  name: 'support-label',
  input: messageText,
  system: 'Return one label: billing, bug, account, or other.',
  maxOutputTokens: 20,
};

const result = await ai.complete(task);
const label = parseSupportLabel(result.text);

The route knows the task name and its acceptance rules. It does not know whether the selected provider expects max_tokens, max_output_tokens, a content array, or a different message role. That knowledge stays in the adapter.

Keep capability declarations next to the adapter:

export type ProviderCapabilities = {
  structuredOutput: boolean;
  streaming: boolean;
  toolCalls: boolean;
  imageInput: boolean;
};

export type RegisteredProvider = {
  id: string;
  model: string;
  provider: AiProvider;
  capabilities: ProviderCapabilities;
};

A capability check should fail before a request is sent. If a task needs structured output and the selected provider cannot guarantee it, return a typed configuration error or choose an explicitly approved alternative. Do not silently downgrade to free-form text and hope a parser survives.

How should adapters handle provider differences?

Keep one adapter per provider and one translator per capability. Avoid a giant conditional that checks provider names throughout the application.

class OpenAiAdapter implements AiProvider {
  constructor(private readonly client: OpenAiClient) {}

  async complete(task: AiTask): Promise<AiResult> {
    const response = await this.client.responses.create({
      model: 'configured-model',
      instructions: task.system,
      input: task.input,
      max_output_tokens: task.maxOutputTokens,
    });

    return {
      text: response.output_text,
      provider: 'openai',
      model: response.model,
      usage: {
        inputTokens: response.usage?.input_tokens,
        outputTokens: response.usage?.output_tokens,
      },
      raw: response,
    };
  }
}

The client and response fields in this example represent an adapter boundary; use the installed provider client’s current types rather than copying names into a different implementation. The important property is that the rest of the app receives AiResult, not a vendor response object.

For a second provider, write a separate adapter that maps its conversation and content format to the same result. Keep provider-specific features explicit:

type ProviderOptions = {
  common: {
    maxOutputTokens?: number;
  };
  openai?: { background?: boolean };
  anthropic?: { cacheControl?: 'ephemeral' };
};

If a route passes anthropic.cacheControl, it has opted into a provider-specific contract. That is better than pretending the option has identical semantics everywhere. Portability is not sameness; it is controlled change.

How do you design fallback without duplicate side effects?

Fallback belongs around model calls that are safe to repeat, not around an entire business operation. If a request also charges a card, sends an email, or writes a record, separate those effects from model selection.

async function completeWithFallback(
  task: AiTask,
  providers: RegisteredProvider[],
): Promise<AiResult> {
  let lastError: unknown;

  for (const candidate of providers) {
    if (task.schema && !candidate.capabilities.structuredOutput) continue;

    try {
      return await candidate.provider.complete(task);
    } catch (error) {
      lastError = error;
      if (!isRetryableModelError(error)) throw error;
    }
  }

  throw new Error(`No approved provider completed ${task.name}`, {
    cause: lastError,
  });
}

The retry classifier needs a small, reviewed set of conditions. A timeout or temporary upstream failure may be retryable. An invalid request, policy refusal, malformed schema, or authentication error generally needs a different path. Do not retry every exception, and do not send the same prompt to three providers without recording that choice.

Use an idempotency key for the surrounding job. Background jobs for AI features explains why a worker needs explicit state and safe retries; the same rule applies when a model fallback is inside that worker. Store the task ID, attempt number, provider, model, status, and validation result. If the worker restarts after the first provider returned a response, it should know whether to reuse, validate, or deliberately retry it.

How do you preserve structured output across providers?

Treat structured output as a validation contract, not as a formatting preference. The provider may offer a schema feature, but your application still needs to validate the returned value before using it.

import { z } from 'zod';

const SupportLabel = z.object({
  label: z.enum(['billing', 'bug', 'account', 'other']),
  confidence: z.number().min(0).max(1),
});

function parseSupportLabel(text: string) {
  const value = JSON.parse(text);
  return SupportLabel.parse(value);
}

If your project does not use this validation library, use the validator already established in the repository. The invariant is what matters: parse, validate, and reject before a model result changes application state.

Keep fixtures for every provider adapter. One fixture should be a valid result, one should be truncated, one should contain unexpected fields, and one should represent a provider error. Then run the same task set through each adapter and compare the normalized result, not only the raw text.

For quality measurement, link each result to an evaluation case. The LLM evaluation loop covers the practical cycle: define cases, run a baseline, change one variable, inspect failures, and keep the evidence. Portability is useful only if switching providers does not erase your ability to compare behavior.

What should you observe and budget?

Record provider, model, latency, input and output usage when returned, retry count, validation status, and a redacted task identifier. LLM observability for production apps describes why the trace should connect the request to retrieval, model calls, tool authorization, validation, and the final outcome.

type AiTrace = {
  taskId: string;
  taskName: string;
  provider: string;
  model: string;
  durationMs: number;
  attempts: number;
  validation: 'passed' | 'failed' | 'skipped';
  inputTokens?: number;
  outputTokens?: number;
};

Do not log prompts or outputs by default when they may contain private customer data. Sample deliberately, redact before storage, and set retention by task type. A provider switch is a data-flow change, so review where prompts travel and where responses are retained before changing the routing table.

Your budget should include failed attempts. If fallback turns one request into two paid calls, the cost model must show that. Compare successful task rate, validated-result rate, latency, retry rate, and cost per accepted result. A cheaper token price is not a cost win if validation failures create manual work.

When is portability worth the extra code?

Portability is worth paying for when you have a real reason to change or compare providers: regional availability, workload-specific quality, cost variation, data handling requirements, or a fallback policy. It is not worth building a ten-provider abstraction before one production task exists.

Keep the first adapter small, put it behind the task boundary, and add a second provider only when a measured requirement justifies it. OTF’s full-stack app templates are one starting point for builders who want owned application code plus AI-tool configuration before they add this boundary; the provider adapters and policy decisions remain yours.

AI provider portability is a code-ownership decision as much as a vendor decision. Normalize the product contract, preserve differences behind adapters, validate every result, and log enough metadata to compare accepted outcomes. That gives you room to change providers without pretending their APIs or behavior are interchangeable.

Sources

Originally published at otf-kit.dev — full-stack app templates for web and mobile. See the templates →

LLM observability for production apps: what to trace, measure, and fix

Dave Kurian — Mon, 31 Aug 2026 18:26:12 +0000

LLM observability starts with the whole request

LLM observability is the practice of connecting a user request to the model calls, retrieval steps, tool calls, latency, cost, and final outcome that followed. In production, logging the generated text alone is not enough. When an answer is slow, expensive, unsafe, or simply wrong, builders need to see which step caused the failure.

The practical approach is to create one trace for the application request and attach a span to every meaningful operation: prompt construction, retrieval, model invocation, tool authorization, tool execution, validation, and response delivery. Record measurements that help you make a decision, but do not automatically store sensitive prompts or user data.

This guide focuses on a small observability contract that works across model providers and can be implemented before adopting a specialized vendor.

Define the signals before the dashboard

Start with questions, not charts. A useful production system should answer questions such as:

Which workflow is slow?
Is latency coming from retrieval, the model, a tool, or a retry?
Which model and configuration produced the response?
How often are outputs rejected by validation?
Which requests exceeded their cost or latency budget?
Did a tool call fail because of authorization, invalid input, or an upstream outage?

Group the signals into four categories:

Traces show the path of one request through the system.
Metrics show rates and distributions across many requests.
Logs preserve structured events and error details.
Evaluations measure whether the output was useful, correct, safe, or complete.

OpenTelemetry provides a vendor-neutral framework for capturing traces and metrics, and its context-propagation model is designed to correlate signals across service boundaries. Its GenAI semantic conventions have moved to a dedicated repository, so check the current conventions before fixing your own attribute names. Sources: OpenTelemetry and OpenTelemetry context propagation.

Make the trace structure boring

A consistent trace shape is more valuable than a clever one. For one user request, use a structure similar to this:

request
├── prompt.build
├── retrieval
│   ├── search
│   └── rerank
├── model.call
├── tool.authorization
├── tool.execute
├── output.validate
└── response.write

Not every request uses every span. The important rule is that each span has a stable name, a start and end time, a status, and a correlation ID. Use the same names for the same operations across workflows so that a dashboard can compare them.

For a model span, record the provider, model identifier, request mode, input and output token counts when available, time to first token when relevant, total duration, retry count, and finish category. Keep the raw prompt and response out of general-purpose telemetry by default. If you need samples for debugging, store them in a controlled system with access restrictions and an explicit retention policy.

For retrieval spans, record the index or collection identifier, query type, number of candidates, selected document count, retrieval duration, and whether the context was empty. Avoid placing full document contents in spans. A document reference, content hash, or redacted excerpt is usually enough to reproduce the path without copying private data into every telemetry backend.

Measure the failures users actually feel

Average latency hides the tail. Track p50, p95, and p99 latency separately for the whole request and for major spans. A workflow can have a reasonable average while a meaningful group of users waits through a slow retrieval call or repeated model retries.

Measure the following rates by workflow and model configuration:

Request success and failure
Timeout and cancellation
Retry frequency
Output-validation rejection
Tool-authorization rejection
Empty retrieval context
Fallback-model usage
Human escalation

For cost, track estimated input and output usage by workflow, tenant, and model. Treat estimates as estimates when provider billing data arrives later or pricing changes. A budget alert should be based on a documented calculation, not a dashboard label that looks precise.

For quality, connect an evaluation result to the same request or trace identifier. A response that completed in 800 milliseconds is not necessarily successful if it cited the wrong record or omitted a required field. The evaluation loop should be separate from the runtime trace, but linked to it.

Use structured events for decisions

A log line such as agent failed is almost useless. Emit structured events with a stable event name and a small set of safe fields.

{
  "event": "llm_request_finished",
  "trace_id": "trace-identifier",
  "workflow": "support-answer",
  "model": "provider-model-id",
  "status": "validated",
  "latency_ms": 842,
  "input_tokens": 1240,
  "output_tokens": 318,
  "retry_count": 0,
  "tool_calls": 1
}

The identifier in this example is a correlation value, not a secret. Never put API keys, authorization headers, access tokens, or unredacted personal data into logs. Apply the same discipline to exception messages: upstream errors often contain request payloads or URLs that were not meant for a general log destination.

Use explicit status values such as completed, timed_out, blocked_by_policy, invalid_output, and upstream_error. These categories let operators separate product failures from infrastructure failures and policy decisions.

Debug by comparing traces

When an incident arrives, compare a successful trace with a failing trace from the same workflow. Look for the first meaningful divergence:

Did retrieval return fewer or different documents?
Did the prompt exceed the intended context budget?
Did the model configuration change?
Did a tool authorization check reject a valid action?
Did a retry reuse a non-idempotent operation?
Did validation reject a response that the model considered complete?

Do not begin with the final answer and guess backward. Follow the trace from the request boundary. This avoids blaming the model for a timeout caused by a slow database or blaming retrieval for a schema mismatch introduced after generation.

Add a trace link to operational errors, evaluation failures, and support tickets where appropriate. That creates a short path from “the answer was wrong” to the exact request path that produced it.

Protect telemetry from becoming a new data leak

Observability data can be more sensitive than application logs because it may contain user prompts, retrieved documents, model outputs, and tool arguments together. Define a telemetry policy before enabling verbose capture.

Use field allowlists rather than trying to redact everything after collection. Hash or classify identifiers when operators do not need the original value. Restrict access by tenant and role. Set retention periods for raw samples, traces, metrics, and aggregate reports separately.

Sample routine successful traces more aggressively than failures, but keep enough metadata to understand traffic changes. Never sample away all blocked, timed-out, or validation-failed traces; those are the traces most useful during an incident.

An implementation sequence for builders

A safe rollout can happen in four passes:

Add a request ID and trace context at the application boundary.
Instrument model, retrieval, tool, validation, and response spans.
Add a small metric set for latency, failures, retries, tokens, and policy blocks.
Link offline evaluations and support reports back to trace IDs.

Only after this foundation is stable should you add expensive payload capture or complex dashboards. Start with one production workflow and verify that a deliberately induced timeout, validation failure, and tool rejection are all visible and distinguishable.

For related production patterns, compare this guide with background jobs for AI features, the LLM evaluation loop, and safe AI agent tool permissions. The broader lesson is the same: reliability comes from explicit boundaries, measurable outcomes, and recoverable operations. OTF’s templates can be a starting point when you want to turn those production conventions into an owned application structure.

Sources

How to design safe tool permissions for AI agents in production apps

Dave Kurian — Mon, 31 Aug 2026 13:42:06 +0000

The model is not the authority

An AI agent can choose a tool, fill in arguments, and explain what it intends to do. None of those things make it authorized to perform the action.

That distinction matters as soon as an agent can access customer records, create invoices, update a project, send a message, or call an external service. A helpful demo often gives the model a broad function such as runAction and trusts the model to decide when it is appropriate. A production application cannot make that the security boundary.

Treat the model as an untrusted decision-maker inside a user session. The application must authenticate the user, check authorization, validate the tool arguments, enforce limits, and record the outcome independently of the model’s reasoning.

The goal is not to make the model perfectly obedient. The goal is to make the application safe when the model is mistaken, manipulated, or unavailable.

Start with a tool inventory

Before writing prompts, list every action the agent can take. Give each tool a plain-language purpose, required inputs, affected resources, and worst-case consequence.

A read-only search tool may expose private data if its query ignores tenant boundaries. A calendar tool may create an unwanted commitment. A “send” tool may contact hundreds of people if the recipient list is not constrained. A database update may be reversible in theory but damaging in practice.

Classify tools into four levels:

Read-only access to information
Reversible changes to a user-owned resource
External communication or financial action
Irreversible or high-impact operations

The classification should determine the controls. Do not use the same approval path for searching documentation and deleting an account.

Keep the inventory close to the implementation. If the tool registry changes but the security review does not, the review is already stale.

Give every tool the smallest possible permission

Least privilege is more reliable than a longer system prompt.

Instead of exposing a general-purpose function that accepts an arbitrary URL, record ID, or SQL expression, create narrow operations with constrained inputs. Prefer get_invoice(invoiceId) over query_database(sql). Prefer draft_email(recipientId, templateId) over send_email(to, subject, body) when the product does not need unrestricted sending.

A narrow tool is easier to test, authorize, rate-limit, and explain to the user. It also reduces the damage caused by prompt injection in retrieved content.

Keep tenant and user identity outside model-controlled arguments whenever possible. The server should derive the current user and tenant from the authenticated session. If the model supplies an identifier, verify that the resource belongs to the same authorization scope before reading or changing it.

Do not give the model secrets. A tool implementation can use a server-side credential without placing that credential in the prompt, conversation history, or model-visible result.

Separate intent from authorization

The agent can propose an action. Your application decides whether that action is permitted.

A useful flow has distinct stages:

The user asks for an outcome.
The model proposes a tool and structured arguments.
The server validates the schema and resource ownership.
The authorization layer checks the user, role, tenant, and policy.
The application requests confirmation if the action is sensitive.
The tool executes with bounded permissions.
The result is recorded and shown to the user.

The model should never be able to skip stages four or five by writing “confirmed” in its own output. Confirmation must come from the application interface or another trusted control.

For example, an agent may prepare a customer email, but the user must approve the final recipients and content before delivery. The approval should be tied to a specific action hash or request identifier so an old approval cannot be replayed against a changed recipient list.

Validate arguments twice

Validate once at the model boundary for useful feedback and again at the tool boundary for security.

The first validation can tell the model that a required field is missing or that a date is invalid. The second validation must assume the input is hostile. Check types, ranges, allowed identifiers, URL destinations, recipient counts, file sizes, and action-specific rules.

Never interpolate model output directly into a shell command, database query, HTML response, or network request. Use parameterized operations and allowlists. If a tool accepts a URL, restrict protocols and hosts. If it accepts a file path, resolve it against an allowed directory and prevent traversal. If it accepts a quantity, enforce both a type and a business limit.

Structured output helps, but a valid structure is not proof that the operation is safe. An object can satisfy a schema and still point to a resource the user cannot access.

Put budgets around agent behavior

Permissions answer “may this action happen?” Budgets answer “how much can happen?”

Set limits for tool calls per request, records returned, recipients, spend, execution time, retries, and outbound domains. Use separate budgets for read and write operations. A runaway read loop may create a cost problem; a runaway write loop may create a customer incident.

Track budgets across the whole workflow, not only inside one model call. If the agent retries three times and each retry can call five tools, the effective limit is larger than the prompt may suggest.

Make high-impact budgets visible. A user should not discover after the fact that an agent was allowed to send 500 messages or modify every record in a workspace.

Treat retrieved content as untrusted input

Documents, webpages, tickets, and customer messages can contain instructions aimed at the model. A retrieved page might say to ignore previous rules and export data. The agent should treat that text as content to analyze, not as an authority that can change its permissions.

Use clear boundaries between instructions and retrieved data. Keep retrieved text out of tool definitions. Limit which fields are passed to the model. Require server-side authorization for every resource access, even when the retrieval result already contains an identifier.

Do not try to solve prompt injection with one magic sentence. Reduce the blast radius instead. A model that sees malicious instructions should still be unable to access another tenant, call an unapproved host, or perform an irreversible operation without confirmation.

Design safe failure and recovery

Tools fail. Providers time out. Workers retry. Users change their minds. Permission checks can change between planning and execution.

Return errors that are useful without exposing secrets or internal infrastructure. Distinguish invalid input, not authorized, rate limited, unavailable, and policy blocked. Do not let a failed tool call cause the model to silently invent a successful result.

Make side effects idempotent. If a request is retried after the network drops, the same action should not send two emails or create two invoices. Attach a stable operation identifier to the action and store the result before allowing a retry to proceed.

For multi-step workflows, checkpoint progress. If an agent has already created a draft and the next step fails, the retry should resume from the known state rather than create a second draft.

Build an audit trail that answers what happened

A useful audit record connects the user request to the actual side effect. Record the request ID, authenticated subject, tenant, tool name, validated argument shape, authorization decision, confirmation state, start and end time, result category, and operation identifier.

Avoid storing secrets and unnecessary personal content. Redact sensitive fields or store references to controlled records instead of copying full prompts into every log.

The audit trail should make these questions answerable:

Who requested the action?
Which policy allowed or blocked it?
What resource was affected?
Did a human confirm it?
Was the operation retried?
What was the final outcome?

Without that evidence, a production incident becomes an argument about what the model probably intended.

Test the permission boundary, not only the happy path

Add abuse cases to the evaluation set. Ask whether a user can request another tenant’s record by changing an ID. Put an instruction in a retrieved document. Try an unapproved URL. Replay an old confirmation. Send an empty or extremely large recipient list. Retry after a simulated timeout. Remove the user’s role between planning and execution.

Test every tool with a user who can see the interface but should not have the underlying permission. Test a tool with malformed structured output. Test a provider outage and a worker restart.

The strongest result is not “the model followed the prompt.” It is “the application remained safe when the model did not.”

A practical release checklist

Before shipping an agent tool, confirm that:

The tool has one narrow purpose.
User and tenant scope come from the server session.
Arguments are validated at both the model and tool boundaries.
Resource ownership is checked immediately before execution.
Sensitive actions require trusted confirmation.
Limits exist for calls, records, spend, recipients, and retries.
External writes are idempotent.
Retrieved content cannot grant permissions.
Secrets never enter model context or logs.
Audit records explain the authorization and outcome.
Abuse cases are part of the regression suite.

This design takes more thought than handing an agent a large toolbox. It also gives builders something far more valuable than a convincing demo: a system that can explain what it is allowed to do, stop when it reaches a boundary, and recover without surprising the user.

If you are building with OTF, use this permission model as part of the production foundation around any AI feature. The model can be replaced. The authorization boundary should remain clear.

Source

OWASP Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/

A practical LLM evaluation loop for AI features that need to ship

Dave Kurian — Sun, 30 Aug 2026 17:57:53 +0000

Prompt testing is not an evaluation strategy

An AI feature can look excellent in a demo and still fail the first week of real use. A user asks a question in a different way, a retrieved document is incomplete, a tool returns an unexpected shape, or a model update changes the tone of the answer. The team then opens the prompt, changes a sentence, and tests three examples by hand.

That process feels productive because the output changes immediately. It is not a reliable way to know whether the product improved.

A production AI feature needs an evaluation loop. The loop should make a change observable, repeatable, and reversible. You need a set of representative inputs, a definition of acceptable behavior, automated checks for obvious failures, and a way to learn from what users do after the answer appears.

The useful question is not “does this prompt work?” It is “does this version perform better on the jobs our users actually need done?”

Start with a small, owned dataset

Do not wait for a perfect benchmark. Start with the examples your product already creates.

Collect successful requests, failed requests, user corrections, support tickets, abandoned flows, and inputs that required a human handoff. Remove personal information and secrets. Then label each example with the behavior that matters. A support answer might need to cite the correct policy, refuse an unsupported request, and avoid inventing a deadline. A code-generation feature might need to preserve an API contract, produce valid syntax, and avoid changing a protected file.

Fifty carefully chosen examples can reveal more than a thousand random prompts. Divide them into a development set and a holdout set. Use the development set while improving the feature. Keep the holdout set hidden from day-to-day prompt editing so it can tell you whether the change generalizes.

Version the dataset. Add a short reason whenever an example changes. If a case was added because a user found a serious failure, keep that context. Your evaluation set is not disposable test data; it is the product’s growing memory of what “good” means.

Define behavior before choosing a grader

A single score such as “quality: 8.4” is not enough. Define the dimensions that make an output useful or dangerous.

For an AI assistant that answers product questions, useful dimensions might be:

Factual accuracy
Completeness
Citation or evidence use
Appropriate refusal
Clear next step
Tone and readability

For an agent that can call tools, add:

Correct tool selection
Valid arguments
Authorization compliance
Side-effect safety
Recovery after a failed tool call

Write a short rubric for each dimension. A good rubric tells the grader what a passing answer must contain and what counts as a failure. It should also identify severe failures that cannot be averaged away. Leaking another tenant’s data is not merely a low score; it is a release blocker.

This is where many evaluation projects go wrong. The team asks a grader whether an answer is “good” and then trusts the number. A useful grader is closer to a reviewer with a checklist: it can explain which requirement was missed and assign a result that maps to a product decision.

Combine deterministic and judgment-based checks

Use the cheapest reliable check first.

Deterministic checks are ideal for structure and safety. Validate JSON against a schema. Confirm required fields exist. Reject unknown tool names. Check that a citation field is present when the workflow requires evidence. Enforce maximum length, latency, token, and tool-call budgets. Verify that an action requiring confirmation has not executed without confirmation.

These checks should run on every change because they are fast and predictable.

Reference-based checks compare the result with known facts or required entities. Exact string matching is often too brittle, especially when several answers can be correct. Instead, check whether required facts appear, forbidden claims are absent, and important names, dates, or identifiers are correct.

Judgment-based checks are useful for qualities such as helpfulness, completeness, and tone. They can be performed by a human reviewer or a separate model with a strict rubric. Keep the evaluator separate from the system being tested where possible. Record the rubric version and evaluator model alongside the result.

Never let a judgment score replace a safety check. A fluent answer can still contain a forbidden claim, expose private data, or trigger an unauthorized action.

Evaluate the whole workflow

The model response is only one part of the user experience.

Suppose a retrieval assistant gives a wrong answer. The root cause might be the model, but it might also be that the correct document was never retrieved, the tenant filter was missing, the context was truncated, or an old result was displayed in the interface. If you evaluate only the final paragraph, you cannot tell which subsystem needs attention.

Capture evaluation data for the workflow stages that matter:

The user input and normalized task
Retrieval queries, filters, and result identifiers
Context assembled for the model
Model output and structured fields
Tool calls and authorization decisions
Validation and retry results
The final response shown to the user

Keep sensitive content out of logs unless your data policy explicitly allows it. You can often retain redacted samples, hashes, identifiers, and structured outcomes instead of every raw prompt and document.

For asynchronous workflows, evaluate recovery too. A job that succeeds on the first attempt but duplicates an email after a retry is not reliable. Include timeouts, provider errors, cancellation, partial completion, and stale status in the test set.

Turn every production failure into a regression case

The most valuable evaluation case is often the one that just failed in production.

When a user corrects an answer, save a sanitized version of the input, the bad behavior, the expected behavior, and the condition that caused the failure. Add it to the development set immediately. Move it to the holdout set after the fix has been tested. That prevents the team from celebrating a local improvement while silently overfitting to the same examples.

Track failure categories, not only pass rates. You want to know whether prompt injection attempts are increasing, whether retrieval failures cluster around one document type, whether one customer segment sees more refusals, or whether a model change increases tool-call errors.

Slice results by language, workflow version, customer tier, input length, document type, and task difficulty. Aggregate numbers hide important regressions. A new version can improve the average while breaking the exact high-value workflow your best customers use.

Make the evaluation a release gate

An evaluation is useful when it changes what you do.

Set a small release policy. For example, a change may ship only when deterministic checks pass, no critical safety case regresses, the holdout score does not fall below its threshold, and the latency or cost budget remains acceptable. The exact thresholds should reflect the product, but the decision must be explicit before the result arrives.

Store each run with the prompt version, model identifier, retrieval configuration, tool definitions, dataset version, grader version, and timestamp. Without this metadata, a result cannot explain what changed.

A simple runner can make the contract visible:

type EvalCase = {
  id: string;
  input: string;
  requiredClaims?: string[];
  forbiddenClaims?: string[];
  expectedTools?: string[];
};

type EvalResult = {
  id: string;
  passed: boolean;
  failures: string[];
  latencyMs: number;
};

async function evaluateCase(testCase: EvalCase): Promise<EvalResult> {
  const started = Date.now();
  const output = await runFeature(testCase.input);
  const failures = checkOutput(output, testCase);

  return {
    id: testCase.id,
    passed: failures.length === 0,
    failures,
    latencyMs: Date.now() - started,
  };
}

The implementation will evolve, but the principle stays stable: every result should say what was tested, what failed, and whether the change is safe to release.

Use real user outcomes as the final signal

Offline evaluations are necessary, but they are not the finish line. Watch what users do after the response.

Do they accept it, edit it, retry, abandon the workflow, open a support request, or ask for a human? For generated code, does the patch pass tests and survive review? For a support answer, does it resolve the case without a follow-up correction?

These signals are not perfect labels. A user may accept a bad answer or reject a good one for reasons unrelated to quality. Still, they reveal cases your curated dataset missed. Sample them responsibly, remove sensitive data, and feed the important failures back into the evaluation set.

That creates a durable loop: collect representative cases, define acceptable behavior, run cheap checks, review meaningful judgments, ship only against explicit thresholds, observe real outcomes, and turn failures into new tests.

The result is a better way to build with AI. You can change the prompt, model, retrieval strategy, or tool workflow without relying on instinct alone. You can explain why a version shipped. Most importantly, you can catch a regression before a user has to become your evaluator.

OTF is built for teams that want to move from an AI-shaped prototype to a product with clear conventions and a real release path. Keep the evaluation loop close to the code, make the pass criteria visible, and let every production lesson improve the next build.

Sources

OpenAI Evals — framework and registry for evaluating LLMs and LLM systems: https://github.com/openai/evals

An AI app security checklist for builders moving from demo to production

Dave Kurian — Sun, 30 Aug 2026 11:19:57 +0000

A production AI app needs more than a safe-looking system prompt. Before you ship, you need a boundary around every input, model call, tool invocation, database write, and user-visible output. This AI app security checklist gives you that boundary in concrete terms.

The short version: authenticate the caller, authorize each resource, treat retrieved text as untrusted data, validate model output before using it, keep tools narrow, store secrets outside code, cap spend and rate, and log enough context to investigate a bad result without logging private data by accident. These controls matter whether your first version was written with Cursor, Claude Code, or another AI coding tool.

Start with the threat model, not the prompt

Write down the path from user input to side effect. For a support assistant, it might be:

A signed-in user submits a question.
The server loads documents for that user's workspace.
The model drafts an answer.
A tool fetches an account record.
The server saves the answer.

Now mark every boundary. Which values come from the browser? Which come from a document? Which can cause a write? Which identity is attached to the request? If the answer is “the model decides,” you have found a security gap, not an architecture.

OWASP's current GenAI LLM Top 10 lists prompt injection, insecure output handling, sensitive information disclosure, excessive agency, and denial of service among the risks application teams need to address. Use the OWASP GenAI LLM Top 10 as a threat checklist, then map each item to a server-side control.

Authenticate and authorize on the server

A model should never be the source of truth for identity. The request handler must derive the user and workspace from a verified session, then check access to every record before it enters the prompt or leaves the database.

Do not accept userId, workspaceId, or role from the request body and trust it. A browser can send any JSON it wants. The same rule applies to tool arguments generated by a model: parse them, validate them, and run authorization against the authenticated principal.

A small policy function makes the boundary visible:

type Principal = {
  userId: string
  workspaceId: string
  role: "member" | "admin"
}

type ToolRequest = {
  name: "get_invoice" | "create_ticket"
  invoiceId?: string
  subject?: string
}

async function authorizeTool(
  principal: Principal,
  request: ToolRequest,
) {
  if (request.name === "get_invoice") {
    if (!request.invoiceId) throw new Error("invoiceId is required")

    const invoice = await db.invoice.findFirst({
      where: {
        id: request.invoiceId,
        workspaceId: principal.workspaceId,
      },
      select: { id: true, status: true, total: true },
    })

    if (!invoice) throw new Error("Invoice not found")
    return invoice
  }

  if (request.name === "create_ticket") {
    if (principal.role !== "admin") throw new Error("Not allowed")
    if (!request.subject || request.subject.length > 200) {
      throw new Error("Invalid subject")
    }
    return { ok: true }
  }

  throw new Error("Unknown tool")
}

The database query includes the workspace boundary. That is more useful than checking permissions in a prompt or relying on a hidden convention in generated code.

Treat prompts, documents, and model output as data

Prompt injection is not limited to a malicious user typing “ignore previous instructions.” It can arrive through a web page, uploaded file, ticket, repository issue, or customer message. OWASP's prompt injection prevention guidance describes direct and indirect attacks, including instructions hidden in external content.

Separate instructions from data in your application design. Put untrusted content in a clearly marked field, tell the model to summarize it rather than obey it, and assume that wording alone will not stop every attack. More important, do not give the model a tool whose permission exceeds the user's permission.

The output needs a boundary too. If the model returns JSON, parse it with a schema. If it returns Markdown, render it with an escaping policy. If it returns an email address, URL, SQL fragment, shell command, or database identifier, validate that value for its specific use before passing it onward. “The model usually returns the right shape” is not validation.

For a structured response, the safe sequence is:

const raw = await model.generate({
  instructions: "Return a support classification.",
  input: userText,
})

let parsed: unknown
try {
  parsed = JSON.parse(raw.text)
} catch {
  throw new Error("Model returned invalid JSON")
}

const result = classificationSchema.parse(parsed)

await db.ticket.update({
  where: { id: ticketId, workspaceId: principal.workspaceId },
  data: {
    category: result.category,
    priority: result.priority,
  },
})

The schema should constrain enums, lengths, numeric ranges, and optional fields. Keep the write conditional on the same tenant scope used during authorization.

Give tools the smallest possible permissions

Start with read-only tools. Add writes only when you can name the user action, authorization rule, validation rule, and rollback behavior. A “manage account” tool is too broad; get_invoice with one workspace-scoped identifier is easier to inspect.

Require confirmation for consequential actions such as sending messages, changing billing details, deleting records, or publishing content. A model can prepare an action, but the user or a separate server-side policy should approve it. Record who approved it, what arguments were approved, and whether the final arguments changed before execution.

Also set limits around loops. A tool-using agent should have a maximum number of steps, a maximum input size, a request timeout, and a budget for model calls. Return a controlled failure when a limit is reached. Do not let a retry loop turn a malformed request into a large bill.

If the feature runs for longer than a request, pair these controls with a queue, stable job identity, and checkpoints. The background-jobs guide covers why retries and side effects need idempotency once an AI workflow crosses process boundaries.

Protect secrets and personal data

Keep provider keys on the server and load them through the deployment's secret store. Never put them in browser bundles, prompts, screenshots, test fixtures, or exception messages. Review logs as if a customer will read them: redact tokens, authorization headers, passwords, full payment details, and unnecessary personal data.

Use separate credentials for development, staging, and production where the provider supports it. Restrict production access to the smallest group that needs it. Add spend alerts and hard limits at the provider level, then enforce per-user and per-workspace limits in your own application. Provider controls catch account-wide drift; application controls stop one tenant from consuming everyone else's budget.

Set retention deliberately. If you store prompts and responses for debugging, define how long they remain, who can access them, and how a user can request deletion. Hash or truncate identifiers in analytics where the full value is not needed. Security is not only preventing remote access; it is also reducing what an incident can expose.

Log security events without logging everything

You need an audit trail for sign-ins, denied authorization checks, tool requests, confirmation decisions, model errors, rate-limit events, and high-impact writes. Each event should include a timestamp, request or job ID, actor ID, workspace ID, action, result, and a safe reason code.

Do not make the prompt itself the audit trail by default. Store a redacted summary or a content hash unless the raw text is required for a documented support workflow. Make logs append-only for normal application roles, and test that a user cannot read another workspace's events.

Finally, test the failures you are trying to prevent. Add cases for cross-workspace IDs, expired sessions, oversized inputs, malformed JSON, prompt injection inside retrieved documents, repeated job delivery, unauthorized tool calls, and budget exhaustion. Run them in CI and against a staging project before production access is enabled.

A practical ship gate

Before calling the feature production-ready, answer yes to these questions:

Is identity derived from a verified server-side session?
Does every read and write enforce the tenant boundary?
Are retrieved documents and user messages treated as untrusted data?
Is every model output parsed and validated for its next use?
Are tools narrow, permission-checked, rate-limited, and confirmation-gated when needed?
Are secrets outside source code and client bundles?
Are spend, input size, time, and step limits enforced?
Can you investigate a denied action without exposing the customer's prompt?
Are retries safe for every external side effect?
Have these cases been tested with hostile input, not only the happy path?

OTF's templates page includes a free AI configs pack for Cursor, Claude, and Lovable. That is a useful starting point for getting an AI coding tool oriented to your project, but the security boundary still belongs in your server code, database policies, provider settings, and tests. Start with the checklist above, then ship one narrow tool at a time.

Sources

Cursor prompts that keep agent sessions focused from first edit to final check

Dave Kurian — Sun, 30 Aug 2026 10:35:12 +0000

Cursor can make a small feature feel like a conversation: describe the change, watch files move, answer a question, and review the result. That workflow is useful until the task becomes larger than one edit. Then a vague prompt creates a vague session. The agent changes the right screen but misses the empty state, updates a handler without checking authorization, or stops after the code compiles while the feature still cannot be used.

The fix is not a longer paragraph full of implementation guesses. It is a prompt that gives the agent a bounded job, a definition of done, and a way to show its work.

Cursor’s CLI changelog makes this more important. Its August 11, 2026 release notes describe delegated subagents, single-turn runs that wait for subagents to finish, queued steering while a turn is active, and durable goals. Those are useful controls for longer sessions, but they do not decide what “finished” means for your product. Your prompt still has to do that.

Start with the user-visible result

Open with the result a user should be able to see or complete. Avoid starting with a file name or a list of technologies. The agent needs context, but the acceptance test should come first.

A weak prompt looks like this:

Add an invite flow to the settings page.

It leaves too many decisions open. Who can invite someone? What happens to an existing email? Is an invitation sent immediately? What does the sender see when the request fails?

A better first pass is specific about the job:

Build the workspace invite flow for an owner or admin.

A successful flow lets the user:
- enter one email address
- see whether the invitation was accepted by the server
- understand when that address already belongs to the workspace
- retry after a temporary failure

Do not change the member list layout or billing screens.

This gives the agent a product boundary before it sees an implementation boundary. It also gives you a useful review question: can a workspace admin complete the invite flow and understand every outcome?

Separate facts from decisions

Agent sessions become noisy when the prompt mixes known requirements with guesses. Mark what already exists, then identify decisions the agent should not make without checking.

For example:

Known:
- The current settings route already loads the workspace id.
- The server has an authenticated request helper used by the member list.
- The member list displays pending invitations.

Decide by inspecting the repository:
- where the invite request belongs
- which existing validation and error conventions apply
- how tests are organized for authenticated requests

Do not invent a second request helper or duplicate member status logic.

This pattern keeps the prompt honest. You are not pretending to know the repository before reading it, and you are not asking the agent to redesign unrelated code. “Inspect first” is often more valuable than prescribing a path that the codebase does not use.

Give the agent a reconnaissance pass

For a feature that crosses a screen, a server action, and a database, ask for a short inspection before asking for edits. Make the first response a map, not a code dump.

Before changing files, inspect the existing invite, member, and authentication paths.
Return:
1. the files you think will change
2. the data flow from the form to persistence
3. existing authorization checks
4. the relevant test commands
5. any ambiguity that needs a decision

Do not edit yet.

This is a useful stop point. You can correct a wrong assumption while the cost is still one message. It also creates a record of the agent’s model of the codebase. If the proposed files are surprising, ask why before approving the implementation.

For a small isolated change, this pass may be unnecessary. For a production feature, it is cheap insurance against a polished edit in the wrong layer.

Define acceptance checks, not just implementation tasks

“Add validation” is an implementation task. “Reject malformed addresses without making a request, and show the server error without clearing the form” is an acceptance check.

Write checks that a person can verify:

Acceptance checks:
- An empty email cannot be submitted.
- A malformed email shows an inline message and makes no network request.
- A valid submission disables the submit control while the request is active.
- A successful response adds the pending invite to the visible list.
- An already-used address explains what happened without creating a duplicate.
- A failed response preserves the typed address and gives the user a retry path.
- A non-admin cannot create an invitation, whether the request comes from the screen or a direct call.
- Tests cover the permission check and the duplicate-address case.

Notice that the list includes more than the happy path. It covers client behavior, server behavior, authorization, and recovery. That is the difference between asking for a feature and asking for a demo-shaped result.

Tell the agent when to stop

Long sessions need explicit stop conditions. Without them, an agent can keep “improving” nearby code or silently broaden the change.

Add a scope rule:

Scope rules:
- Change only files required for this invite flow.
- Do not rename existing public functions.
- Do not migrate unrelated data.
- Do not add a dependency unless the repository has no existing solution; if one is needed, stop and explain why.
- If a test or check fails for an unrelated pre-existing reason, stop and report it instead of rewriting that area.

The last line matters. A failing check is information, not an invitation to make the diff larger. The agent should distinguish a regression caused by the change from a failure that was already present.

For a multi-step session, make the phases visible:

Work in these phases:
1. Inspect and report the plan.
2. Wait for confirmation.
3. Implement the smallest complete change.
4. Run the focused checks.
5. Review the diff for scope, authorization, loading, empty, error, and recovery states.
6. Stop and report files changed, checks run, and remaining risks.

If you are using an autonomous or headless run, the report in step six becomes part of your review surface. A session that exits after delegated work has completed is only useful if its output tells you what was actually checked.

Use a compact context block

The best prompts are not necessarily the longest. Put stable repository facts in a compact block so the task itself stays readable.

Context:
- This is a multi-tenant app.
- Every workspace-scoped read must be filtered by workspace id.
- Authorization is enforced on the server, not only in the UI.
- Existing tests use the focused test command documented in the repository.
- Preserve the current visual language and interaction patterns.

Task:
Add workspace invitations for owners and admins.

Evidence required before you finish:
- focused tests pass
- the diff shows the server permission check
- the UI covers loading, empty, success, duplicate, and failure states
- the final response lists commands and their results

A context block gives the agent constraints that are easy to forget while it is editing. It is also reusable: the multi-tenant and server-authorization rules can remain stable while the task changes.

Ask for evidence in the final response

“Done” is not a useful verification record. Ask for facts that let you review the session without replaying every turn.

In your final response, include:
- a two-sentence summary of the user-visible change
- files changed and why each changed
- commands run and whether each passed
- tests not run and the reason
- any assumptions or follow-up risks
- a short manual test checklist

This also helps when the agent delegates work. A completed subagent is not the same as a verified feature. The parent session should collect the result, compare it with the acceptance checks, and report gaps plainly.

A reusable prompt for production features

Here is the full shape in one place:

Goal:
Describe the user-visible result in one paragraph.

Context:
List only verified repository and product facts.

First pass:
*** the relevant paths and report files, data flow, authorization, tests, and ambiguities. Do not edit yet.

Acceptance checks:
List happy path, validation, loading, empty, error, recovery, authorization, and persistence behavior.

Scope rules:
Name areas not to change, rename, migrate, or add to without stopping.

Execution:
Implement the smallest complete change. Preserve existing conventions. Run focused checks after editing.

Evidence:
Report the diff, commands and results, untested areas, assumptions, and manual verification steps.

You can shorten this for a one-file change. Keep the same bones whenever the task touches user data, permissions, persistence, or more than one layer.

The practical rule

A good Cursor prompt does three jobs: it gives the agent a reason to change the code, a boundary around the change, and proof requirements for calling it finished. The current tooling makes longer agent sessions easier to steer and inspect, but the product judgment remains yours.

Start with the outcome. Ask for reconnaissance before edits. Turn edge cases into checks. Set stop conditions. Require evidence instead of accepting a confident summary. That structure keeps an agent session focused without forcing you to micromanage every line.

For more detail on the CLI behavior behind delegated work, steering, and completion handling, see the Cursor CLI changelog.

Why Cursor's AIUC-1 Certification and Quarterly Re-testing Matters for Your AI Coding Tools

Dave Kurian — Sat, 29 Aug 2026 09:28:28 +0000

Cursor earns AIUC-1 certification — and the quarterly re-test is the part that matters

Cursor earns AIUC-1 certification, an independent adversarial audit of its coding agent covering prompt injection, secrets exfiltration, MCP misuse, unsafe shell commands, and destructive filesystem actions. That's a real receipt. Most AI coding tools ship a "we take security seriously" paragraph and a SOC 2 badge from 2023. Cursor put itself in front of an external red team and committed to a recurring re-test cadence — not a one-and-done marketing logo.

This is worth taking seriously. Let me translate the announcement into a checklist production teams can actually use this week, then talk about the part of agent safety that doesn't change when the model does.

[[COMPARE: one-time security cert vs quarterly adversarial re-test]]

What AIUC-1 actually puts under the microscope

The audit spans the surface area where coding agents actually break things in production:

Secrets protection — can the agent be tricked into reading .env, printing process.env, or pasting tokens into a commit message, a log line, or a fetched URL?
Secure code generation — under pressure, does the model produce SQL with string-concatenated user input, eval() calls, hardcoded API keys, or known-vulnerable dependency patterns?
MCP security — when an MCP tool is in scope, can a malicious tool description, a poisoned resource, or a confused-deputy call get the agent to exfiltrate or overwrite?
Permissions — does the agent respect read-only vs write scopes, file allowlists, branch protection, the difference between "edit this PR" and "push to main"?
Unsafe commands — rm -rf, git push --force, curl | sh, chmod 777, kubectl delete ns. The foot-guns that turn an agent mistake into a Friday incident.
Destructive actions — wiping a database, dropping a bucket, force-pushing, rotating production keys by accident. The audit asks: does the agent ask first?

The point isn't that any one tool "passes." Adversarial testing surfaces tradeoffs, not trophies. The point is that the tradeoffs are now visible, versioned, and re-checked — not buried in a vendor PDF that ages into a logo on the marketing page.

The quarterly cadence is the actual headline

Read the announcement again. The receipt is the re-test.

AI coding agents are not stable software. The model underneath changes every few months. New MCP tools get added. New agent loops get shipped. The "agent" you audited in March is a different system in July. A single certification, even a rigorous one, certifies a snapshot. Quarterly re-testing is what turns that snapshot into a moving baseline — a real signal you can compare release-over-release, vendor-over-vendor.

For production teams, this is the enable. You can now require:

The vendor has an externally-published adversarial test suite, not a private checklist only they run.
The vendor commits to a public re-test cadence — quarterly is the new floor; biannual is stale.
The re-test results are diffable — what regressed, what improved, what was newly added.
The test surface covers the six categories above: secrets, code-gen, MCP, permissions, unsafe commands, destructive actions.

Diffable is the word that matters most. A pass/fail badge is binary; a diff is a signal you can plot, alert on, and argue about in a postmortem. That checklist didn't exist for AI coding tools in any meaningful form six months ago.

How to actually use this today

If you run a team already shipping with Cursor, Claude Code, or anything similar against real repos, here's what to do this week:

# 1. Pin the agent version, not just the CLI
cursor --version
claude --version

# 2. Audit the tool surface the agent can actually call
cat ~/.cursor/mcp.json
cat .mcp.json

// 3. Lock a deny-list at the repo layer, regardless of agent behavior
// .cursor/rules or equivalent agent config
{
  "deny": [
    "Bash(rm -rf:*)",
    "Bash(git push --force:*)",
    "Bash(curl * | sh)",
    "Read(.env*)",
    "Read(**/secrets/**)"
  ],
  "permissions": {
    "filesystem": ["read:./src", "write:./src"]
  }
}

<!-- 4. Add an adversarial self-review prompt your agent runs on its own diff -->
<!-- ai/prompts/security-review.md -->
Before any commit, re-read your diff and answer:
- Did I touch .env*, secrets/, *.pem, or any credentials?
- Did I produce string-concatenated SQL, eval, exec, or pickle?
- Did I run rm -rf, --force, or curl|sh?
- Did I push, merge, or delete anything outside the current branch?
If yes to any: stop, revert, and ask the user.

# 5. Run that prompt in CI on every PR
bun run ai/prompts/security-review.md --on-diff
# fail the build if the agent flags any of the four questions

That last step — running the adversarial prompt in CI, on every PR, against the agent's actual diff — is the closest thing you can do today to "AIUC-1 for your own codebase." Most teams that talk about agent safety have never actually done it.

Two patterns that survive any agent swap

Here's the part that matters once the Cursor news cycle moves on. The agent will change. The model underneath will change. The CLI will rename itself twice in a year. Two layers are durable across every swap:

1. The configuration layer. Deny lists, permission scopes, allowed file globs, MCP allowlists, branch protection rules. These are repo-level artifacts that mean the same thing to every agent. They're also the layer AIUC-1 covers least — it tests the agent's behavior, not your config.

2. The evaluation layer. Adversarial prompts that the agent runs against its own diffs before committing. Run in CI. Versioned with the repo. Diffable across PRs. If a model update silently regresses a safety property — and they do, every model update is effectively a new system — this is what catches it.

The two layers reinforce each other. A deny-list keeps the agent from running rm -rf; the eval layer catches the case where it tried. Without the eval, you wouldn't know the deny-list just saved you. Without the config, the eval fires too late.

[[CONCEPT: guardrails as code — config + eval layers survive any agent swap]]

If your safety story lives entirely inside the agent's prompt or the vendor's blog post, you don't have a safety story. You have a marketing page.

What AIUC-1 doesn't test — and what that means for builders

The audit is scoped to agent-internal behavior: does the agent resist being tricked? It is not scoped to:

Whether your repo's secrets were already leaked before the agent touched them.
Whether your CI runner has the right network egress rules.
Whether your branch protection actually blocks force-pushes.
Whether the third-party MCP server you added last Tuesday has a poisoned tool description.
Whether the agent's "edit this file" actually maps to a scoped, audited write.

In other words: AIUC-1 certifies the agent. It does not certify your system. The durable work — config, evals, CI gates, branch policy, MCP allowlists — is on you. That's the part nobody else can ship for you, and it's the part that has to be re-evaluated every time you change anything in the toolchain. Model swap, MCP upgrade, new agent loop — re-run the evals.

The durable layer underneath the tool churn

Use Cursor. Use Claude Code. Use whatever the next agent is, whenever it ships. Treat the AIUC-1 receipt as a baseline you can ask every vendor to match — adversarial testing, public results, quarterly cadence, diffable regressions.

And then build the part that doesn't change when the model does: repo-level deny lists, permission scopes, adversarial eval prompts in CI, and a checklist the agent has to clear before any commit lands.

This is exactly the work we treat as a kit feature at OTF, not an afterthought. Every full-stack kit we publish has to clear a 24-item design checklist before it's allowed out the door — a script enforces it, no human override, no "we'll fix it next week." Same pattern applies to agent guardrails: encode the rules as artifacts the agent must read (CLAUDE.md + .cursorrules), back them with 20+ tested prompts in ai/prompts/, run them in CI, diff the results release-over-release. The audit badge is a starting line. The portable guardrail layer — configs, prompts, enforced checklists — is the part that compounds across every model swap, every vendor change, every new MCP server you bolt on.

Open Source Cursor Alternatives: Honest Trade-offs for AI-Powered Editing

Dave Kurian — Sat, 29 Aug 2026 07:05:40 +0000

Six open source Cursor alternatives — and the trade-off each one makes

Open-source Cursor alternatives in 2026 are genuinely exciting. You swap the model in two env vars. You swap the agent in one binary. The agent code is on GitHub, the telemetry is opt-in, and you can point any of these at a local Ollama running on your laptop. Compared to closed editor lock-in, that's a real tailwind for builders who care about auditability and model portability.

What you give up is polish and pace. Closed editors spent two years fusing chat with the diff view, the file tree, the terminal, and the test runner. Most open alternatives bolt AI onto an editor that wasn't designed around it, or skip the editor entirely and live in the terminal. Knowing which trade-off each tool makes — and which one best reads your repo conventions — is the difference between a productive agent and one that hallucinates your codebase.

Here's the landscape, with real commands and the honest read on each.

The matrix

Tool	Surface	License	BYO model	Editor integration	Reads repo conventions	AI-tool configs
Continue	Editor plugin	Apache-2.0	Any OpenAI-compatible	VS Code, JetBrains	`~/.continue/config.json` + AGENTS.md	System prompts
Cline	Editor plugin	Apache-2.0	OpenAI / Anthropic-compatible	VS Code	CLAUDE.md / rules path	Per-agent rules
Roo Code	Plugin / fork	MIT	Any OpenAI-compatible	VS Code (fork + extension)	AGENTS.md, custom rules	Multi-mode rules
Aider	CLI	Apache-2.0	30+ providers	External (any editor)	`CONVENTIONS.md`, `--read` flag	Per-repo
OpenHands	Web / sandbox agent	MIT	Any OpenAI-compatible	None — runs in Docker	Per-session repo upload	Limited
Goose	CLI / Desktop	MIT	Anthropic, OpenAI, Ollama, …	External	`.goose/config.yaml`, extensions	Yes

Pricing is identical across the list: free, you pay the model provider (or run it locally). The differences are workflow, not cost.

Continue — the plugin you'll actually keep

Continue is the most direct replacement for Cursor's vibe — a sidebar chat, inline edits, autocomplete, an OpenAI-compatible endpoint. It lives in VS Code and JetBrains, the config is plain JSON, and you can run it against a local Ollama or any hosted model.

The setup is two env vars:

export OPENAI_API_KEY="sk-..."
export OPENAI_BASE_URL="https://openrouter.ai/api/v1"   # or

Then in ~/.continue/config.json:

{
  "models": [
    { "title": "Claude via OpenRouter",
      "provider": "openai",
      "model": "anthropic/claude-sonnet-4.6",
      "apiBase": "${env:OPENAI_BASE_URL}" }
  ],
  "systemPrompts": [{ "title": "Repo conventions",
    "text": "Read AGENTS.md before any task. Use design tokens from @otfdashkit/tokens, never raw values. Keep prop APIs identical across @otfdashkit/ui and @otfdashkit/ui-native." }]
}

Continue reads system prompts on every turn, so a top-level AGENTS.md plus a systemPrompts entry pointing at it gives the agent your conventions without restating them every chat.

Pick this if you want the inline-edit UX Cursor users love, with the editor and model both swappable.

Cline — the agent that wants to actually use the shell

Cline is a VS Code extension that treats the model like an agent with tools: read, edit, run, report. Apache-2.0, source fully visible, supports any model with an OpenAI- or Anthropic-compatible API.

The killer feature is the audit trail — every command the agent wants to run appears in the chat as a permission prompt. You approve, edit, or reject. Most closed-source agents hide that; Cline puts it in the conversation.

// A "rules" file Cline reads on every turn — name and path per your version
{
  "rules": [
    "Read AGENTS.md and ./CONVENTIONS.md before any task.",
    "Run `pnpm test` and `pnpm typecheck` before marking a task done.",
    "Never edit files under db/migrations/ without a linked ticket."
  ]
}

Pick this if you want an agent that does the whole "read, edit, run, fix" loop in your editor, with every action gated behind your approval.

Roo Code — Cline with a different fork

Roo Code started as a fork of Cline and now ships as both a VS Code fork and a standalone extension. License is MIT, the source builds in the open. Same tool-calling architecture as Cline, but it adds multi-agent modes (a planner, a coder, a tester) and a more opinionated UX.

Drop your conventions into its rules file, point AGENTS.md at the kit's tokens, and any of its modes picks them up.

Pick this if you want Cline's tool-calling model but find the planner/coder split useful, and you don't mind a fork.

Aider — terminal-native, Git-native

Aider is the one most terminal-native devs reach for first. MIT, Python install, takes a prompt and a file (or a directory) and returns a Git commit. No editor integration — Aider writes diffs, you review them in any tool that reads Git.

pip install aider-chat

# point at any model endpoint
export ANTHROPIC_API_KEY=sk-ant-...
aider --model claude-sonnet-4-6 \
      --map-tokens 2048 \
      src/components/Button.tsx

The repo-conventions story is direct — Aider reads CONVENTIONS.md from the repo root and you can pass additional prompt fragments with --read:

aider --read CONVENTIONS.md \
      --read ai/prompts/add-page.md \
      --message "Add a /docs/components/tabs page for the new Tabs component"

Aider is the one I see in the most CI pipelines — same binary, same flags, deterministic model, a diff at the end.

Pick this if you live in the terminal, want the same agent in CI as in your editor, and review changes through Git rather than a chat panel.

OpenHands — the autonomous backend

OpenHands (formerly OpenDevin) is closer to a self-hosted Devin than to an editor plugin. It's a Docker image that spins up a sandboxed environment, clones your repo, and runs an agent loop with full tool access. You send it a task over HTTP, it works in the background, you pull the PR when it's done.

# exact env var names vary by version — check the OpenHands docs for your build
docker run -it --rm \
    -v /path/to/repo:/workspace \
    -e LLM_API_KEY="$OPENROUTER_API_KEY" \
    -e LLM_BASE_URL="https://openrouter.ai/api/v1" \
    -e LLM_MODEL="anthropic/claude-sonnet-4.6" \
    ghcr.io/all-hands-ai/openhands:latest

The conventions story is weaker — OpenHands reads files on demand, but it doesn't auto-load an AGENTS.md the way editor plugins do. You ship conventions to it through the prompt, which is brittle but workable.

Pick this if you have long-running, low-supervision work (migrations, ports, batch refactors) you'd rather queue than drive interactively.

Goose — Block's open agent

Goose is the open-source agent from Block, MIT licensed. It runs as a CLI and a desktop app, supports a wide model range (Anthropic, OpenAI, Ollama, and more), and has a clean extension system — new tools drop in as Python files.

goose configure        # pick provider, paste API key
goose run "refactor the billing service to use the new token names"

Goose reads a per-repo config file, which is the natural slot for an AGENTS.md-style conventions doc.

Pick this if you want a model-agnostic CLI agent with a real extension story and the sanity check of a major vendor using it in production.

What every one of them needs from you

Here's the part none of these tools fix. The agent's productivity is bounded by one thing — what it knows about your repo before the first prompt. A frontier model with no context is a smart intern who joined today. A frontier model with documented conventions is the senior engineer who shipped the last three releases.

Every tool above reads some flavor of repo conventions. The substance is identical regardless of which agent you pick:

<!-- AGENTS.md -->
# Conventions

- Use tokens from @otfdashkit/tokens, never raw color or spacing values
- Every component lives in both @otfdashkit/ui (web) and
  @otfdashkit/ui-native — keep the prop API identical
- Run `pnpm test` and `pnpm typecheck` before reporting a task done
- Don't modify files under db/migrations/ without a linked ticket
- New component → add a page under app/(docs)/components/

Whether the agent reads it as AGENTS.md, CONVENTIONS.md, a rule file in its config, or a system prompt is a config shuffle. A paid kit ($99, Everything Bundle $149) ships the prompt library, CLAUDE.md, .cursorrules, and 20+ tested prompts under ai/ — point any of the six agents above at the repo, and the agent has the conventions, the token names, and the file layout it needs on the first turn.

[[COMPARE: swappable tool layer on the left — agent swapped every six months, model swapped every quarter, editor swapped every couple of years — vs durable context layer on the right — repo conventions, design tokens, prompt library, AGENTS.md — outlives every one of them]]

The tool is the swappable layer. The context is the durable one.

Which one should you pick?

Shipping a Next.js / Node service, you live in VS Code, you want the inline-edit UX → Continue.
Same stack, you want the agent to run shell commands and edit files with you approving each step → Cline (or Roo Code if you want the planner/coder split).
You're a terminal-first engineer and you want the same agent in CI as in your editor → Aider.
You have a backlog of long-running, low-supervision tasks (migrations, ports, refactors) you'd queue rather than drive interactively → OpenHands.
You want a model-agnostic CLI agent with a real extension system and a major-vendor sanity check → Goose.
You're not sure which, and you think the context layer matters more than the agent → pick whichever has the smoothest setup, point it at a kit with CLAUDE.md and ai/prompts/, and let the conventions outlive the tool.

Pick the agent. Keep the conventions. The model changes every quarter, the editor changes every couple of years — your repo's AGENTS.md lasts as long as the code does.

enable New Possibilities: Building with the Claude Agent SDK

Dave Kurian — Fri, 28 Aug 2026 20:09:26 +0000

The Claude Agent SDK put something genuinely new on the table for builders: an agent loop that reads a real filesystem, runs real commands, and edits real files — not just a chat box. That's the enable, and it's a real one. This post is what you can actually build with it from a builder's seat, and what the repo the agent lands in has to look like for the loop to be worth running.

What the SDK actually puts in your hands

Four commonly-discussed primitives show up in every serious walkthrough, and they're worth naming so the rest of the post makes sense:

A real-filesystem agent loop. The agent reads files, edits them, runs shell commands, and iterates on the result. It's not hallucinating an answer; it's acting on disk — with all the screw-ups and recoveries that implies.
Skills. Bundles of instruction plus capability — a folder of files that teach the agent a new trick without retraining the underlying model.
Hooks. Event-driven side effects. A tool call goes out, your code intercepts it, and you can block, log, or augment before it lands.
MCP servers. Pluggable tool registries. Point the agent at a server that speaks the protocol, and new tools show up — no SDK fork required.

[[DIAGRAM: a builder feeding an agent loop, with four rails converging on the same workspace — filesystem access, a skills folder, hook handlers, and an MCP server all feeding the same loop]]

The convergence is the part most demos skip. The agent isn't a chatbot with tools bolted on. It's a loop over a workspace, and everything you wire into that workspace becomes part of what the agent can do. That changes what "using the agent" means day to day — you're not prompting a model, you're configuring a runtime.

A custom skill, in shape

A skill is roughly a directory of files plus a short manifest describing what it is and when to use it. The agent discovers the folder, reads the manifest, and decides whether the user's intent matches. The match is where the design work happens — a vague manifest fires too often, an over-specific one never fires.

# tree of a real skill (shape — see current docs)
skills/
  deploy-staging/
    MANIFEST.md        # what it does, when to fire
    scripts/
      deploy.sh        # the actual work
    examples/
      last-run.log     # what success looks like

<!-- skills/deploy-staging/MANIFEST.md -->
# Deploy to staging

Fire when the user asks to ship, push, or deploy to staging.

## Steps
1. Run `scripts/deploy.sh` from the repo root.
2. Wait for "build complete" in the output.
3. Compare output against `examples/last-run.log` shape; report deltas.
4. Never run without `DEPLOY_CONFIRM=1` in env.

## When NOT to use this
- Production deploys (different skill).
- Local dev server (`npm run dev`).

(Manifest filename and exact schema vary by SDK version — treat the above as the shape of the pattern, not the contract.)

The manifest is the contract with the agent. You didn't teach the model anything — you handed it a folder and told it what's inside. That's a one-shot capability upgrade you can ship today, and the agent's next run picks it up.

A hook that catches the bash mistake

Hooks are how you keep an autonomous loop honest. The most useful pattern is a pre-tool-use gate: before the agent runs a shell command, your code gets a veto. This is where you encode the rules a junior engineer would have been told on day one — the agent never was.

// .claude/hooks/block-dangerous-bash.ts
// (Pseudocode — event shape and config keys vary by SDK version;
// see current docs before relying on the exact fields below.)

export default async function preToolUse(event) {
  if (event.tool !== "bash") return;
  const cmd: string = event.input.command;

  const forbidden = [
    /rm\s+-rf\s+~\//,        // home dir wipe
    /git push.*--force/,     // force push
    /curl.*\|\s*sh/,         // curl-pipe-shell
    /DROP\s+TABLE/i,         // destructive SQL
  ];

  for (const re of forbidden) {
    if (re.test(cmd)) {
      return { action: "deny", reason: `Blocked by pattern ${re}` };
    }
  }
}

// .claude/settings.json (illustrative config — see current docs)
{
  "hooks": {
    "PreToolUse": [".claude/hooks/block-dangerous-bash.ts"]
  }
}

The agent still sees the bash call in its reasoning trace. It just never lands. This is the cheapest way to keep an autonomous loop from doing something irreversible on a Tuesday afternoon, and it composes — you can stack ten hooks, one per concern, and the agent never knows.

Plug in an MCP server for one tool

MCP is the boring part that's quietly the most capable. You can hand the agent access to a tool it has no native knowledge of — an internal DB, a third-party API client, a search index — by pointing it at a server that speaks the protocol.

// .claude/mcp.json (illustrative — see current docs)
{
  "mcpServers": {
    "<your-integration>": {
      "command": "npx",
      "args": ["-y", "<your-mcp-server-package>"],
      "env": { "<YOUR_API_KEY>": "${env:YOUR_API_KEY}" }
    },
    "<internal-docs>": {
      "command": "node",
      "args": [".mcp/internal-docs-server.mjs"]
    }
  }
}

The agent now sees tools prefixed with <your-integration>_* and <internal-docs>_* in its available tool list — no SDK code change, no model retraining. New team, new tool, new server, same agent. The protocol is the contract, which is why MCP is a bigger deal than its quiet reputation suggests: it turns "the agent has tools" into "the agent has whatever tools your org decides to ship."

The repo is half the agent

The SDK is genuinely good. But — and this is the part that doesn't show up until you run the same agent on two different repos in the same week — the repo it lands in matters as much as the agent itself.

[[COMPARE: a convention-documented kit repo vs a year-old generated-spaghetti repo, run through the same agent loop]]

Same agent, same model, same prompt, same task. The delta on first-task success rate is consistent and significant. The model didn't get worse. The substrate did. The agent reads your repo the way a senior engineer reads your repo: if the conventions are clear, it follows them; if they're not, it invents ones that don't exist, and the code drifts a little every iteration.

Most "the AI slowed down" stories come from this, not from the agent. The agent is doing its job. The job description — the repo — is the problem.

Three things a substrate needs

None of this is expensive. All of it is mandatory if you want the agent to perform a tier higher than it does in an unstructured repo:

A root-level convention file. CLAUDE.md, .cursorrules, or AGENTS.md — whichever your harness reads. It states where components live, how tests run, the canonical way to add a feature, and what NOT to do. One file, top of the tree, written in plain prose.
A predictable layout. One folder per concern. app/, components/, lib/, db/, tests/. The agent doesn't need to be clever — it needs to know where to look. Predictability beats elegance every time for an automated reader.
A prompts/ folder of tested instructions. Not vibes — instructions you've actually run, that produced the output you wanted. Twenty verified prompts beats two hundred you wrote once and forgot. Each one is a checked-in contract: "when the user asks for X, do Y."

If your repo has all three, the same agent performs a tier higher than it does in a repo without them. We've measured this across multiple kits. The delta isn't subtle, and it's not a model-side win — it's a substrate-side win.

The pre-wired advantage

Use the SDK. And here's the part that doesn't change when the model does.

Every OTF kit — the SaaS Dashboard, the Fitness app, the Booking one — ships with the three substrate items already in place, so the agent has a substrate worth running on from minute one:

A CLAUDE.md at the root that names where things live and why. Plain prose, no magic.
A .cursorrules file saying the same thing in the format Cursor wants, so a second agent with a different harness reads the same conventions without you rewriting them.
A prompts/ folder of 20+ tested instructions: "add a new pricing tier", "wire a new Stripe webhook", "add a screen to the mobile app". Each hand-verified to produce working output — not aspirational, machine-checked.

On top of that, a 24-item design checklist is enforced by a script before any kit ships. The conventions aren't aspirational — they're checked at build time, so the substrate the agent lands in is the same substrate the kit shipped. No drift between the doc the agent reads and the code it's editing.

You drop the agent into the kit, and on day one it knows where the components are, how to add a feature, and what NOT to do. You're not spending week one teaching the agent your conventions. They're already written down.

[[IMG: clay character confidently running the agent loop inside a structured kit repo, with the chaos of an unstructured legacy repo shrinking in the background]]

That part doesn't change when Anthropic ships a new SDK version, when a competitor ships a different agent, or when the model under the hood gets swapped. The conventions are durable. The agent is the perishable layer.

What this enables

A second developer — the AI one — that ramps in a day instead of a quarter. Same loop, same hooks, same MCP servers. The only variable is the substrate it's dropped into. Make the substrate legible and the agent pays you back ten times. Leave it as generated spaghetti and the agent becomes the thing you blame for the mess.

The SDK is the lever. The repo is the lever's length. You decide which one to invest in.

Beyond Pretty Charts: The Essential Features of a Real Financial Dashboard

Dave Kurian — Fri, 28 Aug 2026 19:05:20 +0000

A pricing page screenshot tells you nothing. The difference between a dashboard template and a dashboard you'd actually run your business on lives in the boring parts — the eight edge cases that surface the first time a real user clicks a real button on a real Tuesday.

Here's the checklist I run every dashboard candidate through, what most templates ship instead, and what the SaaS Dashboard kit at saas.otf-kit.dev already has wired. The point isn't that templates are bad. The point is that pretty and wired are two different products, and the wiring is what compounds.

[[COMPARE: marketing screenshots of charts vs the eight edge cases the user actually hits]]

1. Server pagination on every table

Pretty templates ship a table that fetches everything. That's fine for the demo with 47 rows. It's a wall on day one of production, where transactions are 40M rows and the user clicked "page 2."

Server pagination is three decisions made correctly:

// page state lives in the URL, not component state
const [page, setPage] = useQueryState('page', parseAsInteger.withDefault(1))
const [pageSize, setPageSize] = useQueryState('size', parseAsInteger.withDefault(25))

const { data, isLoading, error } = useSWR(
  ['transactions', page, pageSize, range],
  ([, p, s, r]) => fetchTransactions({ page: p, size: s, range: r })
)

The URL is the source of truth so a refresh doesn't lose the user's place, the browser back button works, and a shared link deep-links to "March, page 4, 25 per page." That last one is the test most templates fail.

2. The three states every async surface owes you

Loading, error, empty. One component, three branches. Every screen, every table, every card.

function TransactionsTable({ range }: { range: DateRange }) {
  const { data, error, isLoading, refetch } = useTransactions(range)

  if (isLoading) return <TableSkeleton rows={10} />
  if (error)     return <ErrorState error={error} onRetry={refetch} />
  if (!data?.length) return (
    <EmptyState
      title="No transactions in this range"
      hint="Try widening the date filter or check the import job."
      action={<Button onClick={openImportGuide}>How to import</Button>}
    />
  )

  return <DataTable columns={columns} rows={data} />
}

The empty state is the one everyone forgets. It's also the one that tells you the template author actually thought about the user. A bare "No data" is a confession. A hint plus a CTA is a product.

3. Date-range filters that compose

A date picker bolted onto a table is a demo. A date picker that composes with pagination, with the URL, with the export button, and with whatever filter the user adds next is a system.

The composition rule: the filter is a URL param, the table reads from URL, the export reads from URL, the chart reads from URL. One source of truth, four surfaces.

// filters live in the URL; everything reads them
const [range, setRange]     = useQueryState('range',  parseAsString.withDefault('last_30d'))
const [account, setAccount] = useQueryState('account', parseAsString.withDefault('all'))
const [status, setStatus]   = useQueryState('status',  parseAsString.withDefault('all'))

When the filter and the table share state via the URL, the export button automatically exports the filtered view. When the agent regenerates the page, the filter survives because it was never in component state to begin with.

4. Exports that survive a slow database

Exports are where dashboards go to die. The user clicks "Export CSV," the request times out at 30 seconds, the browser shows nothing, the user clicks again, and now the job is queued twice.

async function onExport(range: DateRange) {
  const job = await startExportJob({ range })       // returns a jobId
  toast.promise(pollExport(job.id), {
    loading: 'Preparing your export…',
    success: (url) => { window.location.href = url; return 'Download ready' },
    error: (e) => `Export failed: ${e.message}`,
  })
}

Job-based, polled, with a real toast that survives a page refresh. The CSV isn't generated in the request — it's generated by a worker against the same query the table uses, so pagination and filters apply identically. If your template ships exports as a synchronous Blob download, it has never met a real database.

5. Access control at the data layer, not the button

Hiding a button is not access control. A user with reports.read: false who guesses the URL still sees the report. Access control has to fail closed at the data layer.

// server route guards the query, not the UI
export async function GET(req: Request) {
  const user = await requireUser(req)
  if (!user.can('reports.read')) return new Response('forbidden', { status: 403 })

  const rows = await db.transaction.findMany({
    where: { orgId: user.orgId, ...dateFilter(req) },
  })
  return Response.json(rows)
}

The UI hides the link. The server denies the query. The audit log records the attempt either way. This is the boring layer that makes the dashboard safe to hand to a paying customer on day one — and it's the layer most templates leave entirely to you.

6. Walking the live SaaS Dashboard kit

saas.otf-kit.dev is the live demo, not a screenshot of one. Auth and Stripe are wired — not stubbed — so the demo runs real sessions and real webhooks. The kit ships full-stack: auth, billing, DB, and Stripe already connected. You clone it, point your keys, and the dashboard is live.

What's wired out of the box:

Transactions table with server pagination, URL-backed filters, and the three states from section 2 — the empty state isn't "No data," it's the actual hint-with-CTA.
MRR / churn charts reading from the same query layer as the tables, so the chart and the table never disagree.
Customer list with role-based columns hidden at the server, not the client. A user without customers.export cannot export customers, full stop.
Export job with a polled download URL and a toast that survives the navigation that triggers the download.
Settings → Billing running the actual Stripe customer portal, not a mock.

The 24-item design checklist runs as a script before the kit ships, so spacing, focus rings, and dark mode aren't aspirational — they're enforced. The same component primitive renders in dark mode without a second stylesheet because the design tokens flip one theme across the whole surface.

7. Adding a new metric screen via the kit's prompts

The kit ships a tested prompt library — CLAUDE.md plus .cursorrules plus a folder of prompts the team has run end-to-end against this codebase. The point isn't to write a prompt. The point is that the prompt has already been run against this kit, on this stack, and the result is in the repo.

A new "Refunds by reason" metric screen:

# ai/prompts/06-new-metric-screen.md

Add a new metric screen to the SaaS Dashboard.

Inputs:
- New metric: refunds grouped by reason
- Where: under `/dashboard/reports`, visible to users with `reports.read`
- Visualisation: bar chart + a top-10 table
- Date range filter must compose with the existing URL state

Steps:
1. Add the query in `server/reports/refunds.ts` using the existing
   `dateFilter` helper and `requirePermission('reports.read')`.
2. Add the route in `app/dashboard/reports/refunds/page.tsx`.
3. Use `DataTable` for the table, `<Card>` for the chart, and the
   existing `DateRangePicker` — do not introduce new components.
4. Wire the export button to `startExportJob`.
5. Run `pnpm kit:check` (the 24-item script). All checks must pass.

Point Claude Code or Cursor at the prompt. The agent extends the kit instead of regenerating it, because CLAUDE.md already tells it the conventions — what the existing helpers are, where queries live, what the design tokens are. A prompt without CLAUDE.md is a coin flip; a prompt with it is a refactor.

[[CONCEPT: the agent extends the kit instead of regenerating it — CLAUDE.md is the contract]]

The new screen inherits pagination, filter composition, access control, and the three states for free, because those live in the kit, not in the screen. That is the entire bet: a wired layer underneath the screens, so every new screen ships the boring parts on day one.

What this gets you

A new metric in fifteen minutes, not a new metric after a sprint of "we should also add empty states." The eight-item checklist stops being a backlog and starts being the floor — every screen inherits it because the kit enforces it. That's the difference between a dashboard you demo and a dashboard you run a business on.

A template can win on screenshots for one launch cycle. It loses the moment the user paginates, filters, exports, or hits the empty state. Build for the second click, not the first. The kit is at saas.otf-kit.dev — clone it, point your keys, and the boring parts are already done.

[[IMG: the clay character at a laptop, screens behind it showing a new metric live in the dashboard — a small check badge floating over the export button, the empty-state card replaced by real rows, the agent prompt pinned to the side as the receipt of how it shipped]]

Master Cursor: From Install to Shipping Your First Feature

Dave Kurian — Fri, 28 Aug 2026 17:07:35 +0000

Cursor lives in the editor, not next to it. That is the entire pitch and it's a real one. Most "AI coding" tools bolt a chat panel to the side and call it integrated; Cursor is the editor with the model wired into the buffer, the selection, and the file tree. You select code, you type a sentence, the code changes. You don't copy a prompt into a website, you don't paste output back. For a working developer that single difference is worth more than every benchmark ever published — it's the difference between using a tool and demoing one.

This post is the one I wish I had on day one: install Cursor, open a real codebase (not an empty folder), write your conventions before the first prompt, learn the three core interactions, and ship one feature end to end. The whole thing takes an afternoon. The single non-obvious move — giving the agent a written rulebook — is what separates "it generated plausible code" from "it generated code I can merge."

1. Install and open a real repo

Download Cursor from the official site and sign in. The Pro tier is what you want for serious use; the free tier will cap you mid-session in a way that makes the tool feel worse than it is. Don't judge the product on the free tier.

Skip the "new folder" temptation. Open a real project — a Next.js app, an Express server, anything with at least 20 files and a real dependency graph. The model is dramatically more useful when it can read the surrounding code and copy the conventions it finds. An empty folder is a stress test, not a workflow.

Let the indexer finish. Cursor builds a vector index of your repo so that chat and agent mode can pull in the right files. It runs in the background; the spinner in the bottom-right tells you when it's done. The first thing most beginners do is start prompting before the index is ready and conclude the tool is bad. The tool is fine. Wait for the index.

2. The one thing beginners skip: write `.cursorrules` before your first prompt

Here's the lesson that costs the most when you skip it. The model's default behaviour is to invent conventions. It will pick a state management library, a folder layout, a test framework, an error-handling style, and a name for every function — and it will pick differently every prompt. You end up with a codebase that has six ways to do everything, none of which match the rest of the project.

The fix is a file at the repo root named .cursorrules. Cursor injects its contents into every prompt, automatically, in the background. You don't @-mention it, you don't paste it in. The model just reads it. This is the single highest-use thing you can do in the first five minutes.

# Project: SaaS dashboard
# Stack outcome, not stack names: shared components across web and mobile
# Owner: shipped product, not a demo

## Conventions
- Components live in `src/components/`. One file per component. Co-locate the test in `__tests__/`.
- Server actions return `{ data, error }`. Never throw across an action boundary.
- Money is integer cents. Always. The number of bugs this prevents is not zero.
- Use the design tokens from `@otfdashkit/tokens` for color, spacing, type. Hard-coded values are a code-review fail.
- Database access goes through the typed query layer in `src/db/queries/`. No raw SQL in route handlers.

## What to refuse
- Do not introduce a new dependency without flagging it. Prefer existing libs.
- Do not "fix" a file outside the task. Stay scoped.
- Do not add a comment that restates the code.

## Voice
- Senior engineer. Terse. Code over prose. No "let me explain", no "happy to help".

Three things to notice. First, this is outcomes not libraries — "shared components across web and mobile" rather than naming every framework. The model can adapt as the stack evolves. Second, there is a "refuse" section. Without it, the model will "improve" code you didn't ask it to touch and the diff becomes unreviewable. Third, it has a voice line. The model matches the tone of the prompt; tell it the tone you want.

Write this file, commit it, then open Cursor for the first time. Reverse this order and you will spend an hour deleting invented conventions.

3. The three interactions you actually need

Cursor has a lot of surface area. Ignore most of it. Three interactions cover 90% of real work.

Inline edit (Cmd+K). Select code, type what you want changed, hit enter. This is for surgical, local edits — rename this, add the type, swap the loop for a map, fix the off-by-one. It edits the buffer, you review the diff, you keep moving. Use it for anything you could have done yourself in under two minutes but don't want to.

Chat (Cmd+L). A side panel. Use it for questions, explanations, and "what does this file do." It does not edit files by default — it's a thinking partner, not a doer. The right time for chat is when you are stuck and want a second opinion faster than reading docs.

Agent mode (Cmd+I). This is the one people underestimate. You give it a multi-step task — "add a password reset flow that emails a token, expires after 1 hour, and updates the audit log" — and it edits multiple files, runs commands, reads errors, fixes them, and reports back. The quality is a function of three things: the .cursorrules file, how clearly you describe the goal, and how small the scope. Big vague prompts in agent mode produce big vague diffs. Small specific prompts in agent mode produce small specific diffs that you can actually review.

The mental model: Cmd+K is your hands, chat is your head, agent mode is your junior engineer. Use each for what it's good at.

4. One end-to-end feature, start to finish

Here is the workflow for shipping a real thing. The example is a "duplicate project" button on a SaaS dashboard. Real code, real steps.

Step 1 — scope it in chat. Open the project page file in the editor. Hit Cmd+L, type:

Read src/app/projects/[id]/page.tsx and src/db/queries/projects.ts. I want to add a "Duplicate" button to the project header that creates a copy of the project, its tasks, and its members under a new name. Do not write code yet. Tell me what you'd touch, what edge cases you see, and what questions you have.

This is the move beginners skip. You are using the model to think, not to type. Read its answer. Push back on assumptions. "What about the billing relationship?" "What about archived projects?" You are the engineer; the model is the rubber duck on steroids.

Step 2 — agent mode for the implementation. Once the scope is sane, hit Cmd+I and give it the actual task:

Add a server action duplicateProject(projectId, newName) in src/app/projects/actions.ts. It should: 1) load the project by id, 2) deep-copy it plus its tasks and members under a new id, 3) set the new name, 4) return the new project id. Use the existing db.transaction helper. Do not modify the project header component yet — that is the next step.

Notice: scoped, named function, single file, no scope creep. Agent mode will read the existing helpers and copy their style because your .cursorrules told it to. The diff is reviewable.

Step 3 — inline edit for the UI. Select the project header JSX, hit Cmd+K, type:

Add a "Duplicate" button next to the existing "Edit" button. On click, prompt for a name, call the action, then router.refresh().

That's it. Three interactions, three files touched, one feature shipped. Total time: under fifteen minutes for a working dev. The output is in your style because the .cursorrules told the model what your style is.

5. What changes when the conventions are pre-written

A .cursorrules file is one file, but it changes the economics of the whole tool. Without it, every prompt reinvents the project. With it, the first prompt and the fiftieth prompt look the same to the model — same conventions, same refusals, same voice. The output stops drifting. Your code reviews stop being a hunt for inconsistencies and start being a hunt for bugs, which is the job.

This is also why the content of the file matters. "Use TypeScript" is noise. "Money is integer cents. Always." is the kind of rule that prevents a four-hour debugging session six months from now. The best rules are the ones written from a real scar.

[[COMPARE: first prompt with no rules vs tenth prompt with .cursorrules]]

6. The part that doesn't change when the tool does

Cursor is a fantastic tool. So was the last one, and the one before that, and the one before that. The thing that survives every tool churn is the convention layer — the documented decisions about how your codebase works, written once, enforced everywhere, legible to humans and machines.

This is exactly why the kits we ship at OTF include CLAUDE.md, .cursorrules, and 20+ tested prompts at install time. Not because we think Cursor is risky — use Cursor, use Claude Code, use whatever ships next week — but because the conventions are the part that outlives the tool. The component is the same <Button> on web, iOS, and Android, themed by a single token file, audited by a 24-item design checklist before it ships. The AI config is the same kind of layer for the model. Pre-written, pre-tested, version-controlled with the code. The model comes and goes; the rules stay.

If you want to skip the part where you discover every footgun by hand, the SaaS Dashboard, Fitness, and Booking kits at otf-kit.dev ship with this whole setup — components, tokens, AI config, deployment script — for $99 each, or the everything bundle for $149. Free MIT components on npm if you just want the building blocks.

What this gets you

By the end of one afternoon you have: a real editor, a real repo, a real convention file, three interactions you understand, and one shipped feature. The next feature is faster because the model already knows the rules. The one after that is faster still. That compounding — not the model, not the editor, the documented conventions — is the actual product.

Start with .cursorrules. Everything else follows.

Why Builders Are Leaving MUI and the Honest Alternatives

Dave Kurian — Fri, 28 Aug 2026 16:05:16 +0000

MUI is one of the most productive React component libraries in the world, and for many teams it's the right call. It's accessible, comprehensive, and ships with enough components that you can build a real product without bolting on third-party pieces. The question isn't whether MUI is good — it is. The question is whether MUI's opinions match your product's opinions, and what to switch to when they don't.

MUI is the right answer for some teams

Stay with MUI if you're shipping a B2B SaaS where the Material look reads as "professional," you want a comprehensive single-vendor suite and don't mind the bundle, your designers and PMs like Material's vocabulary, or your mobile story is "responsive web," not native apps. For a CRUD-heavy internal tool, a Material-adjacent aesthetic is a feature, not a bug, and MUI's accessibility story is genuinely strong. The pain isn't MUI. The pain is MUI being the default when something else actually fits.

Other themed libraries: same shape, different opinions

If the complaint is "Material's specific look" but you still want a comprehensive suite, you can swap MUI for another library of the same shape and pick a different design vocabulary.

Library	Best for	Tradeoff vs MUI
Chakra UI	Lighter bundles, simpler theming, accessibility by default	Smaller ecosystem — advanced components like DataGrid and complex date pickers need third-party add-ons
Mantine	Dense component set with bundled hooks (forms, notifications, modals manager)	More npm dependency weight than MUI; CSS-modules-style API is a learning curve if you're coming from emotion
Ant Design	Enterprise density, complex tables, multi-locale admin tools	Heaviest bundle of the four; Chinese design vocabulary shows through theming work

Pick this if you want a comprehensive single-vendor suite but Material's elevation, ripples, and typography don't match your brand. Skip it if the underlying complaint is "I want my own design system" — these are still vendor opinions, just different ones.

The three things that drive builders away

Bundle weight. The full MUI suite ships with a meaningful per-import footprint. Tree-shaking helps, but the emotion-based runtime is always present, and the moment you pull in an icons package you'll notice. Mobile-first sites feel this first.

Material look bleeding through. Even after theming, the vocabulary keeps showing. The elevation system, the ripple on every Button, the FAB, the dense Tabs, the Typography scale — Material's design language is what you bought, and "make this look less Google" is harder than it should be.

Theme fights. Overriding MUI feels like a constant negotiation. Want a flat button with no ripple? You fight the default props. Want a Paper without the elevation overlay? You fight the theme. The library was built to express Material's design system, and any other design system is a translation:

const theme = createTheme({
  components: {
    MuiButton: {
      styleOverrides: {
        root: { borderRadius: 2, textTransform: 'none', boxShadow: 'none' },
      },
    },
    MuiPaper: {
      styleOverrides: {
        root: { backgroundImage: 'none' },
      },
    },
    // ... and this list never ends
  },
})

Each override is small. The accumulated cost is real.

Headless primitives: you bring the styles

The most aesthetic-opinion-free escape from MUI: a library that gives you behavior, accessibility, and keyboard handling, but no CSS. You bring a stylesheet or a utility-class system. Total control, small bundles, and every pixel is yours to write.

The two exemplars searchers compare are Radix Primitives (unstyled, composable, the de-facto choice for utility-class-first projects) and Headless UI (smaller surface, tightly scoped to what utility classes usually need).

import { Dialog } from 'your-headless-lib'

<Dialog.Root open={open} onOpenChange={setOpen}>
  <Dialog.Trigger asChild>
    <button className="rounded-md bg-slate-900 px-3 py-2 text-white">
      Open
    </button>
  </Dialog.Trigger>
  <Dialog.Portal>
    <Dialog.Overlay className="fixed inset-0 bg-black/40" />
    <Dialog.Content className="fixed left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2 bg-white p-6 rounded-lg shadow-xl">
      ...
    </Dialog.Content>
  </Dialog.Portal>
</Dialog.Root>

Pick this if you have a strong design system already, you want pixel-perfect control, and you don't mind writing CSS. Skip it if you want to ship a screen tonight and your design system is "whatever looks reasonable."

Utility-class-first sets: the current default

The CLI-copied-into-your-repo pattern. You install a tool, run a command, and a Button.tsx lands in your repo. From then on the component is yours — edit it freely, no upstream fight.

shadcn/ui is the exemplar searchers compare against: it composes headless primitives with utility classes, owns nothing, and ships CLI commands that drop components into your repo. Other implementations in this bucket follow the same pattern.

The upside is speed-to-pixel and ownership. The downside is exactly ownership — when the upstream ships a bugfix or a new component, you don't get it automatically; you diff and merge.

Pick this if you want speed, you want the components in your repo so you can edit them freely, and you accept that "upgrades" become a chore. Skip it if you want a maintained vendor you can outsource component bugs to.

Cross-platform SDKs: when mobile is on the roadmap

Different problem entirely. You also ship iOS and Android, ideally from one codebase, and "responsive web" stopped being the answer last year. Web-only sets don't help here — the native side is a separate decision.

A cross-platform SDK gives you a single component API that compiles to a web bundle, a native iOS bundle, and a native Android bundle from the same source. The renderers differ per platform (DOM, UIKit, Android views), but the API doesn't:

import { Button } from '@otfdashkit/ui'

<Button intent="primary" onPress={submit}>Save</Button>

Same JSX, same prop names, same intent values. Design tokens flip one theme across all three targets so a Card looks identical in Safari, in a native iOS app, and in a native Android app.

[[DIAGRAM: one component file renders to a web bundle, a native iOS bundle, and a native Android bundle from the same component API and shared design tokens]]

Pick this if iOS and Android are on the roadmap and "responsive web" isn't good enough. Skip it if mobile is "maybe someday" — the cross-platform overhead earns itself when you actually ship both targets.

Picking by constraint

Constraint	Pick
Material look is the look	Stay with MUI
Comprehensive suite, but not Material	Chakra, Mantine, or Ant Design
Pixel-perfect, design system already exists	Headless primitives (Radix, Headless UI)
Speed-to-pixel, own the code	Utility-class-first (shadcn/ui)
Web + iOS + Android from one codebase	Cross-platform SDK

The mistake is picking by trend. The right choice follows from your hardest constraint.

Where OTF fits

OTF is the cross-platform option. The free SDK ships roughly 200 components on npm under MIT, with the same component name and props on web, iOS, and Android. Design tokens flip a theme across all three targets so a single source of truth covers everything.

What makes the SDK different from a normal cross-platform component library is what ships with it for AI coding agents. Every kit includes CLAUDE.md and .cursorrules configs plus 20+ tested prompts in ai/prompts/ — so when Cursor or Claude Code opens the repo, the agent extends the kit instead of regenerating it. You ask for a DataTable in chat and you get the OTF DataTable, not a utility-class hallucination.

For shipping, one script wires a custom domain, DNS, TLS, and a mobile build in a single command. Quality is enforced by a 24-item design checklist that runs before any kit ships — no hand-waving about "looks right on my machine."

For the "just ship it" version of the same problem, OTF also publishes paid full-stack kits ($99 each, Everything Bundle $149) — SaaS Dashboard, Fitness, Booking — with auth, billing, DB, and Stripe wired, plus 15 landing templates at $9. The free SDK is enough to evaluate; the kits are what you reach for when the deadline is real. Live demos at saas.otf-kit.dev and fitness-preview.otf-kit.dev.

OTF doesn't compete with MUI, Chakra, Mantine, Ant Design, Radix, Headless UI, or shadcn/ui on the web-only question. Those are great answers when the deliverable is a web app. When iOS and Android are also on the deliverable, and you want one component source of truth instead of three, that's where a cross-platform SDK earns its place.