DEV Community

Haley
Haley

Posted on

Make Your Agent's No Auditable: A Step-by-Step

#agents #ai #llm #tutorial

Your agent just said no to a user. No explanation. No alternative. Just cold words.

The user left. The ticket doubled. The AI team blamed the model.

The real bug was invisible: you had no idea why it refused. Policy? Tone? A hallucinated constraint?

You need a refusal card.

A refusal card is a structured record shown at the exact dead end. It turns a one-line no into a decision point.

Here is the flow:

User request -> Agent decision -> Refusal? -> Generate card -> Validate -> Show card

In this tutorial, I will build one using MonkeyCode. It is an open-source project. Its free tier currently offers 10 million tokens and a free server option. As of August 2026, that is enough for this experiment.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Step 1: Define the card schema

A card must answer five things:

  • What did the user ask?
  • What did the agent decide?
  • Why? With concrete evidence.
  • What alternatives exist?
  • Can a human override?

Here is a Python schema:

card = {
    'request': 'Cancel my subscription',
    'action_taken': 'declined',
    'reasons': [
        {'type': 'policy', 'detail': 'Only admins can cancel', 'evidence': 'user.role == viewer'}
    ],
    'alternatives': ['Ask an admin'],
    'human_override_possible': True,
    'stop_condition_triggered': True
}
Enter fullscreen mode Exit fullscreen mode

Validation is straightforward:

def validate(card):
    errors = []
    if 'request' not in card: errors.append('Missing request')
    if not card.get('reasons'): errors.append('Need at least one reason')
    for reason in card['reasons']:
        if 'evidence' not in reason: errors.append('Reason needs evidence')
    return errors
Enter fullscreen mode Exit fullscreen mode

Run this:

python -c 'from validate import validate; sample = {}; print(validate(sample))'
Enter fullscreen mode Exit fullscreen mode

Step 2: Generate the card with a free model

MonkeyCode exposes an OpenAI-compatible API. Set your key and base URL as environment variables.

import os, json
from openai import OpenAI

client = OpenAI(
    api_key=os.environ['MONKEYCODE_API_KEY'],
    base_url=os.environ.get('MONKEYCODE_BASE_URL')
)

def generate_card(user_request, agent_response):
    prompt = f'Convert this into a refusal card JSON: {user_request} -- {agent_response}'
    completion = client.chat.completions.create(
        model=os.environ.get('MONKEYCODE_MODEL', 'free'),
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0
    )
    return json.loads(completion.choices[0].message.content)
Enter fullscreen mode Exit fullscreen mode

The model name may differ. Check your dashboard.

Step 3: Test locally 

python test_generator.py
Enter fullscreen mode Exit fullscreen mode

My test printed a valid card. Then I ran the validator. It returned no errors.

Step 4: Deploy to a free server

MonkeyCode's free server lets you run a small HTTP service. Wrap the function in Flask:

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/refusal_card', methods=['POST'])
def endpoint():
    data = request.get_json()
    card = generate_card(data['user_request'], data['agent_response'])
    return jsonify(card), 200 if not validate(card) else 422
Enter fullscreen mode Exit fullscreen mode

Follow MonkeyCode's deployment guide for the free server. Then test with a small Python script:

import requests
response = requests.post(
    'https://your-free-server.monkeycode.example/refusal_card',
    json={'user_request': 'Cancel my subscription', 'agent_response': 'Sorry, I cannot cancel.'}
)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

You will get a structured refusal card.

Step 5: Wire it into your agent

Before your agent returns a refusal, call this service.

if should_refuse:
    card = generate_card(user_request, response)
    show(card)
Enter fullscreen mode Exit fullscreen mode

Now the user sees why, what else to try, and whether a human can override.

Limitations

  • Free tier rate limits may block high traffic.
  • The model can emit malformed JSON. Add a retry.
  • Evidence is only as good as the agent's response. If the agent lies, the card lies.
  • Model names and quotas change. Verify them before production.

Who should skip this

If your agent never makes high-stakes decisions, skip it. If you already have deterministic policy reasons, skip it.

But if your agent is a probabilistic black box, start here. The free tokens are enough to test a few hundred cases. That will expose the first lie.

Try it with your own logs. MonkeyCode's free tier is a good playground for this pattern.

Top comments (0)