DEV Community

Sonam
Sonam

Posted on

A/B Test AI Prompts at the Edge with Telnyx Stateful Actors

#ai #edge #typescript #telnyx

Changing a prompt is easy. Knowing whether the new prompt is actually better is the hard part.

This example builds a small prompt A/B testing API on Telnyx Edge Compute. You send one task and two prompt variants. The app runs both variants through Telnyx AI Inference, stores the experiment in a Stateful Actor, and lets users vote on which result they prefer.

Code: https://github.com/team-telnyx/telnyx-code-examples/tree/main/edge-prompt-ab-tester

What it does

The app gives you a tiny evaluation loop:

  • create an experiment
  • compare two model responses
  • vote for variant A or B
  • close the experiment
  • check aggregate stats

Routes:

  • POST /experiments
  • POST /experiments/<id>/vote
  • POST /experiments/<id>/close
  • GET /experiments
  • GET /experiments/<id>
  • GET /stats
  • GET /health/liveness
  • GET /health/readiness

Create a prompt experiment 

curl -X POST https://edge-prompt-ab-tester-<id>.telnyxcompute.com/experiments \
  -H "Content-Type: application/json" \
  -d '{
    "task": "Write a one-sentence tagline for an edge compute platform.",
    "variant_a": "You are a concise marketing copywriter. Return just the tagline.",
    "variant_b": "You are a technical explainer. Return just the tagline, focus on latency."
  }'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "id": "exp-msf712rs-0",
  "task": "Write a one-sentence tagline for an edge compute platform.",
  "variant_a": {
    "prompt": "You are a concise marketing copywriter. Return just the tagline.",
    "response": "Real-time compute, right where your data lives."
  },
  "variant_b": {
    "prompt": "You are a technical explainer. Return just the tagline, focus on latency.",
    "response": "Execute code milliseconds from your users to eliminate network latency."
  },
  "votes_a": 0,
  "votes_b": 0,
  "status": "open",
  "created_at": "2026-08-04T21:51:55Z"
}
Enter fullscreen mode Exit fullscreen mode

Vote on a result 

curl -X POST https://edge-prompt-ab-tester-<id>.telnyxcompute.com/experiments/exp-msf712rs-0/vote \
  -H "Content-Type: application/json" \
  -d '{"variant":"a"}'
Enter fullscreen mode Exit fullscreen mode

Close the experiment:

curl -X POST https://edge-prompt-ab-tester-<id>.telnyxcompute.com/experiments/exp-msf712rs-0/close
Enter fullscreen mode Exit fullscreen mode

Check stats:

curl https://edge-prompt-ab-tester-<id>.telnyxcompute.com/stats
Enter fullscreen mode Exit fullscreen mode

Example:

{
  "total_experiments": 3,
  "open_experiments": 2,
  "closed_experiments": 1,
  "total_votes": 15,
  "leader": "variant_a",
  "leader_votes": 9
}
Enter fullscreen mode Exit fullscreen mode

The AI call

The app uses:

POST /v2/ai/chat/completions
Enter fullscreen mode Exit fullscreen mode

Default model:

zai-org/GLM-5.2
Enter fullscreen mode Exit fullscreen mode

Each prompt variant becomes the system message. The task becomes the user message. Both variants run against the same input, which makes the comparison easier to reason about.

Why Stateful Actors?

Prompt tests need a little bit of memory.

This example uses an ABTester Stateful Actor to store:

  • experiments
  • prompt variants
  • model responses
  • vote counts
  • open or closed status
  • aggregate stats

That means the API can remember experiment state without adding a separate database just for the demo.

Run it 

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/edge-prompt-ab-tester
Enter fullscreen mode Exit fullscreen mode

Store your API key as an Edge Compute secret:

telnyx-edge auth api-key set <YOUR_API_KEY>
telnyx-edge secrets add TELNYX_API_KEY "KEY0123..."
Enter fullscreen mode Exit fullscreen mode

Install and deploy:

npm install
telnyx-edge ship
Enter fullscreen mode Exit fullscreen mode

Then use the deployed URL to create experiments and collect votes.

Production notes

Before using this pattern for real product decisions, I would add:

  • authentication
  • evaluator identity
  • duplicate vote prevention
  • randomized or blinded variants
  • fixed evaluation sets
  • scoring rubrics
  • prompt version history
  • privacy controls for sensitive tasks

But even in this small form, it is useful. Prompt changes should not be judged only by vibes. They should leave a trail of examples, votes, and outcomes.

Resources:

Top comments (0)