Day 9: Advanced Prompt Engineering — Few-Shot, Chain-of-Thought and Structured JSON

#ai #promptengineering #30dayaiseries

Prompt engineering is not just talking to a chatbot. It is the programmable interface that lets you get dependable, repeatable results from an AI, and in an automated system, dependable is the whole point.

In plain terms: Imagine briefing a new colleague. "Sort these tickets" gets you a guess. A short brief, a couple of worked examples, and a note on how you want the result written down gets you what you actually need. A good prompt is that brief.

Diagram: A strong prompt has clear parts: a role, a task, context, examples, an output format and rules. Each part removes guesswork for the model. See the animated version.

1. Few-shot prompting

With zero-shot prompting you give only an instruction. With few-shot prompting you add two or three perfectly formatted examples of the input and the output you want. The examples ground the model: it copies the pattern, so the answers come back in the same shape every time.

Diagram: Zero-shot gives only an instruction, so answers vary in wording and format. Few-shot adds two or three worked examples, which anchors the model to the exact format you want. See the animated version.

2. Chain-of-thought (CoT)

Chain-of-thought prompting asks the model to write out its reasoning step by step before it gives the final answer. If you ask an AI to analyse the dividend yield of a real-estate trust, making it do the maths in order, instead of jumping to a number, greatly reduces logic errors. A simple way to ask is: "Work through this step by step, then give the final answer on a new line."

Diagram: Chain-of-thought asks the model to show its reasoning before the final answer. On a multi-step calculation, working through the steps in order cuts down careless logic errors. See the animated version.

3. Structured JSON output

In automated workflows, free text is not much use. If the answer must go into a database or another program, you have to constrain the prompt so the model returns a strict JSON structure that code can parse. Many AI platforms now also offer a built-in structured-output mode that enforces a schema.

Diagram: Chatty free text cannot be parsed reliably. Constraining the model to a JSON schema lets the answer flow straight into databases and application code. See the animated version.

Here is a small, runnable example that builds a few-shot prompt and then checks the reply before trusting it. (It does not call a model: it shows what you send and how you validate what comes back.)

import json

# 1. A few-shot prompt: two worked examples, then the new ticket
examples = [
    ("Disk is full on the file server", {"team": "hardware", "priority": "medium"}),
    ("Cannot log in after the password reset", {"team": "identity", "priority": "high"}),
]
prompt = "Classify each IT ticket. Reply with JSON only, using keys team and priority.\n\n"
for text, answer in examples:
    prompt += f'Ticket: "{text}"\nAnswer: {json.dumps(answer)}\n\n'
prompt += 'Ticket: "VPN keeps dropping every few minutes"\nAnswer:'
print(prompt)

# 2. Whatever the model replies, check it before your code trusts it
def parse_ticket(reply: str) -> dict:
    data = json.loads(reply)                          # fails if the reply is not valid JSON
    assert set(data) == {"team", "priority"}, "unexpected keys"
    assert data["priority"] in {"low", "medium", "high"}, "bad priority"
    return data

print(parse_ticket('{"team": "network", "priority": "high"}'))     # accepted
try:
    parse_ticket("Sure! The ticket looks like a network issue.")   # chatty text is rejected
except Exception as err:
    print("rejected:", type(err).__name__)

Even with a good prompt, a model can occasionally break the format, so the checking step matters.

Choosing the technique

Diagram: Use few-shot examples for a consistent format, chain-of-thought for multi-step reasoning, and a structured JSON schema when the output feeds code. They can be combined. See the animated version.

Coming Up Next

Day 10: Tokenization, context windows and cost optimization.

PromptEngineering #LLMs #AIImplementation #SoftwareEngineering #Automation

Originally published at https://sureshpallapothu.in/blog/day-9-advanced-prompt-engineering, where this post includes animated diagrams.

DEV Community