How To Train Predictable AI Agents using XML+JSON

Prompting Predictable AI Agents: A Complete Guide

Instructions given to your agent greatly affects the consistency of its output. Once your agent receives suitable instructions, outputs are much more predictable and structured, resulting in less bugs and embarrassing mistakes.

Instruction File Formats

XML:

• Preferred format for training models due to less complexity for the system training on it
• Better for hierarchical instructions and complex nested structures
• More readable for humans when dealing with multi-layered prompts
• Natural way to separate different instruction components

JSON:

• Best for creating multiple outputs from a single execution of an AI agent
• More flexible with working on software contributions
• Easier to parse programmatically in most programming languages
• Better for structured data exchange between systems

How to Choose XML vs JSON

Choose XML when:

• You need clear separation of instruction sections
• Working with complex, nested prompt structures
• Human readability is a priority
• Building templates that will be reused frequently
• Creating prompts with multiple conditional branches

Choose JSON when:

• Building API integrations
• Need to programmatically generate or modify prompts
• Working with structured data inputs/outputs
• Building systems that need to parse responses automatically
• Creating dynamic prompts based on user input

Creating the Prompt

1. Define the Role

The role sets the context and expertise level for your AI agent. A well-defined role helps the AI understand what perspective to take and what knowledge to draw from.

Key principles for defining roles:

• Be specific about the expertise level (junior, senior, expert)
• Include relevant background context
• Specify the industry or domain if applicable
• Define the relationship to the user (assistant, advisor, reviewer, etc.)

Examples of good role definitions:

• "You are a senior software engineer with 10 years of experience in Python and web development"
• "You are a helpful customer service representative for a SaaS company"
• "You are an expert data analyst specializing in e-commerce metrics"

2. Define the Goal

The goal clarifies what you want the AI to accomplish. It should be specific, measurable, and aligned with your desired outcome.

Components of a good goal:

• Clear objective statement
• Expected deliverable format
• Success criteria
• Any constraints or limitations

3. Write Instructions

Instructions are the core of your prompt. They should be detailed, unambiguous, and actionable.

How to Write Effective Instructions

Be Specific and Detailed

Vague instructions lead to unpredictable outputs. Instead of "analyze this data," specify exactly what analysis you want, what metrics to focus on, and how to present the results.

Use Clear Structure

Break complex instructions into numbered steps or bullet points. This helps the AI follow a logical sequence and reduces the chance of missing important details.

Define Output Format

Specify exactly how you want the response formatted. Include details about:

• Length requirements
• Structure (headings, lists, paragraphs)
• Tone and style
• Any specific terminology to use or avoid

Include Constraints

Be explicit about what the AI should NOT do:

• Information it shouldn't include
• Approaches to avoid
• Length limitations
• Sensitive topics to handle carefully

Provide Context

Give the AI enough background information to understand the situation, but avoid information overload that might confuse the core task.

How to Provide Examples

Examples are one of the most powerful tools for ensuring consistent AI behavior. They show rather than tell the AI what you want.

Types of Examples to Include:

Input-Output Examples: Show the AI exactly what kind of input it will receive and what output you expect.

Format Examples: Demonstrate the exact structure, tone, and style you want in responses.

Edge Case Examples: Show how to handle unusual or problematic inputs.

Negative Examples: Include examples of what NOT to do, which can be as valuable as positive examples.

Best Practices for Examples:

• Include 2-3 examples minimum for consistency
• Make examples diverse to cover different scenarios
• Keep examples concise but comprehensive
• Update examples based on real-world performance

How to Write Good Prompts

The CLEAR Framework:

• Context: Provide necessary background information
• Length: Specify desired response length
• Examples: Include relevant examples
• Audience: Define who the response is for
• Role: Clearly define the AI's role and expertise

Additional Best Practices:

• Test your prompts with various inputs
• Iterate based on actual performance
• Keep prompts as concise as possible while maintaining clarity
• Use consistent terminology throughout
• Version control your prompts for tracking improvements

Examples of Good and Bad Prompts

Example 1: Content Creation

❌ Bad Prompt:

<prompt>
Write about dogs.
</prompt>

✅ Good Prompt:

<prompt>
<role>You are an expert pet care writer with 5 years of experience creating educational content for dog owners.</role>

<goal>Create an informative blog post about dog training basics that will help new dog owners establish good behaviors in their pets.</goal>

<instructions>
1. Write a 800-1000 word blog post about basic dog training
2. Include practical, actionable advice
3. Structure with clear headings and subheadings
4. Use a friendly, encouraging tone
5. Include 3-4 specific training techniques
6. End with a brief summary of key points
7. Avoid mentioning specific dog breeds unless necessary for context
</instructions>

<examples>
Good section example:
"## Positive Reinforcement Training
Positive reinforcement is the foundation of effective dog training. When your dog performs a desired behavior, immediately reward them with treats, praise, or play. This creates a positive association..."

Bad section example:
"Dogs are bad when they don't listen. You should punish them when they're bad."
</examples>

<output_format>
- Use markdown formatting
- Include H2 headings for main sections
- Write in second person (addressing the dog owner directly)
- Include actionable bullet points where appropriate
</output_format>
</prompt>

Example 2: Data Analysis

❌ Bad Prompt:

<prompt>
Look at this sales data and tell me what's happening.
</prompt>

✅ Good Prompt:

<prompt>
<role>You are a senior business analyst specializing in e-commerce sales performance with expertise in identifying trends and providing actionable insights.</role>

<goal>Analyze the provided quarterly sales data to identify key trends, opportunities, and areas of concern that require management attention.</goal>

<instructions>
1. Review the sales data for Q1-Q3 2024
2. Identify the top 3 positive trends and top 3 areas of concern
3. Calculate month-over-month and quarter-over-quarter growth rates
4. Compare performance against industry benchmarks (if available)
5. Provide specific, actionable recommendations for Q4
6. Focus on metrics that directly impact revenue and profitability
7. Do not speculate beyond what the data supports
</instructions>

<data_context>
The data includes: monthly revenue, number of transactions, average order value, customer acquisition cost, and return rate across three product categories (electronics, clothing, home goods).
</data_context>

<output_format>
## Executive Summary
[2-3 sentence overview]

## Key Findings
### Positive Trends
[Numbered list with specific data points]

### Areas of Concern  
[Numbered list with specific data points]

## Recommendations
[Actionable steps with expected impact]

## Supporting Data
[Relevant calculations and metrics]
</output_format>

<examples>
Good finding example:
"Electronics revenue increased 23% QoQ driven by a 15% increase in average order value ($127 to $146) while maintaining stable conversion rates at 3.2%."

Bad finding example:
"Sales went up which is good for business."
</examples>
</prompt>

Example 3: Code Review

❌ Bad Prompt:

<prompt>
Review this code and make it better.
</prompt>

✅ Good Prompt:

<prompt>
<role>You are a senior software engineer with expertise in Python, code quality, and best practices. You conduct thorough code reviews focusing on functionality, maintainability, and security.</role>

<goal>Provide a comprehensive code review that identifies issues, suggests improvements, and ensures the code meets professional standards.</goal>

<instructions>
1. Review the provided Python function for correctness and efficiency
2. Check for potential security vulnerabilities
3. Evaluate code readability and maintainability
4. Suggest specific improvements with code examples
5. Identify any missing error handling
6. Comment on naming conventions and documentation
7. Rate the overall code quality on a scale of 1-10 with justification
8. Prioritize suggestions by impact (high, medium, low)
</instructions>

<review_criteria>
- Functionality: Does the code work as intended?
- Security: Are there any security risks?
- Performance: Can efficiency be improved?
- Maintainability: Is the code easy to read and modify?
- Standards: Does it follow Python best practices?
</review_criteria>

<output_format>
## Summary
[Overall assessment and quality score]

## Critical Issues (Fix Immediately)
[High priority items with code examples]

## Improvements (Recommended)
[Medium priority suggestions]

## Minor Enhancements (Optional)
[Low priority, nice-to-have changes]

## Revised Code
[Improved version of the original code]
</output_format>

<examples>
Good feedback example:
"**Security Issue**: Line 23 uses string concatenation for SQL queries, creating a SQL injection vulnerability. Replace with parameterized queries:cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))

Conclusion

Creating predictable AI agents requires thoughtful prompt engineering. By defining clear roles, specific goals, and detailed instructions, you can significantly improve the consistency and quality of AI outputs. Remember to iterate on your prompts based on real-world performance and always include relevant examples to guide the AI's behavior.

The key to success is being explicit about your expectations while providing enough context for the AI to understand the task fully. With practice, you'll develop an intuition for crafting prompts that produce reliable, high-quality results.

Website: https://whoschris.dev
LinkedIn: https://www.linkedin.com/in/christopher-clemmons/