Chain-of-Self-Questioning: How Agents Decide When to Abstain Instead of Hallucinate

Every agent builder hits the same production wall: your LLM answers confidently when it should refuse. In financial workflows, a wrong answer costs money. A refusal costs time. Chain-of-Self-Questioning (CoSQ) is a prompt-only framework that makes answer commitment conditional on an explicit information-sufficiency check. No model retraining or external fact-checking APIs required.

The core idea is simple. Before committing to an answer, the agent asks itself whether it has the information needed to answer correctly. If the self-assessment fails, it abstains. If it passes, it proceeds. This turns the answer-or-refuse decision into an observable, tunable control surface.

The Production Problem

LLMs produce fluent text even when their factual support is weak. Chain-of-thought prompting improves reasoning but does not prevent confident hallucinations. External verification tools add latency, cost, and integration surface area. Fine-tuning for abstention requires labeled data and model access.

CoSQ addresses this by embedding the abstention decision directly into the prompt structure. The agent evaluates its own information state before committing to an answer. This works across model families without changing weights or adding infrastructure.

How CoSQ Works

The framework introduces three variants, each with different risk-coverage trade-offs:

Grounded-CoSQ asks the model to list the information required to answer the question, then assess whether it has that information. If the assessment is positive, it answers. If negative, it abstains.

Critical-CoSQ adds a step: the model identifies which pieces of missing information are critical. It abstains only if critical information is missing. This increases coverage at the cost of slightly higher risk.

Adaptive-CoSQ adjusts the abstention threshold dynamically based on the question type. It uses a meta-prompt to classify questions by risk profile, then applies stricter or looser thresholds accordingly.

Each variant operates at a configurable threshold τ (tau), which controls the confidence level required to commit. Higher τ means fewer answers but higher accuracy on answered questions. Lower τ means more coverage but higher risk of wrong commitments.

Integration Pattern

CoSQ fits into existing agent loops without changing the orchestration layer. The pattern looks like this:

import re

def parse_confidence(assessment_text: str) -> float:
    """
    Extract confidence score from LLM self-assessment output.
    Expects format: "Assessment: YES (confidence: 0.92)" or "Assessment: NO (confidence: 0.45)"
    """
    match = re.search(r'confidence:\s*([0-9.]+)', assessment_text, re.IGNORECASE)
    if match:
        return float(match.group(1))
    # Fallback: binary YES/NO mapping
    if 'YES' in assessment_text.upper():
        return 0.95
    return 0.30

def agent_response(question: str, threshold: float = 0.90) -> dict:
    """
    CoSQ-wrapped agent response with explicit abstention logic.
    """
    # Step 1: Self-questioning prompt
    assessment_prompt = f"""Question: {question}

Before answering, list the information required to answer this question correctly.
Then assess: do you have all required information?

Required information:
- [list each requirement]

Assessment: [YES/NO with confidence score 0.0-1.0]
Format your final assessment as: "Assessment: YES (confidence: 0.92)" or "Assessment: NO (confidence: 0.45)"
"""

    assessment = llm.complete(assessment_prompt)
    confidence = parse_confidence(assessment)

    # Step 2: Conditional commitment
    if confidence >= threshold:
        answer_prompt = f"""Question: {question}

Based on your assessment, provide your answer.
"""
        return {
            "status": "answered",
            "response": llm.complete(answer_prompt),
            "confidence": confidence
        }
    else:
        return {
            "status": "abstained",
            "reason": assessment,
            "confidence": confidence
        }

This structure exposes three control points:

• Threshold tuning: Adjust τ based on business cost asymmetry
• Variant selection: Choose Grounded, Critical, or Adaptive based on domain
• Observability: Log confidence scores and abstention reasons for monitoring

Evaluation Results

The paper evaluates CoSQ on TruthfulQA (817 multiple-choice questions) across eleven model families. The balanced-option protocol tests the framework under realistic conditions where wrong answers and abstentions have different costs.

Metric	Chain-of-Thought Baseline	Grounded-CoSQ (τ=0.90)	Relative Change
Wrong commitment rate	13.1%	8.9%	-32.1%
Answered accuracy	86.9%	89.7%	+3.2%
Coverage (% answered)	100%	87.6%	-12.4%

The key result: Grounded-CoSQ reduces wrong commitments by 32% while improving accuracy on answered questions. The cost is 12.4% coverage reduction, meaning the agent refuses to answer about one in eight questions.

The improvements are consistent across all eleven tested models and hold at every evaluated threshold. This suggests the pattern is robust to model choice and tunable to business requirements.

Latency and Token Trade-offs

CoSQ adds one or two LLM calls per question:

1. Assessment call: Generates the self-questioning chain and confidence score
2. Answer call: Generates the final response (only if assessment passes threshold)

For a typical question, this adds:

• Latency: 1-2 seconds for the assessment call (depending on model and prompt length)
• Token cost: 200-500 tokens for assessment, 100-300 for answer
• Total overhead: Roughly 2x the baseline token cost when answering, 1x when abstaining

In high-frequency financial workflows, this overhead matters. You can optimize by:

• Batching assessments for multiple questions in a single call
• Caching assessment prompts to reduce repeated tokenization
• Using faster models for the assessment step (e.g., GPT-4o-mini instead of GPT-4)

The token cost is predictable and scales linearly with question volume, unlike external verification tools that may require database lookups or API calls.

Tuning the Abstention Threshold

The threshold τ controls the answer-or-refuse decision boundary. In production, you tune it based on the relative cost of false negatives (refusing good answers) versus false positives (committing to bad answers).

For financial agents:

• High-stakes decisions (e.g., trade execution, regulatory filings): Set τ high (0.95+) to minimize wrong commitments, accept lower coverage
• Advisory workflows (e.g., research summaries, trend analysis): Set τ moderate (0.85-0.90) to balance accuracy and coverage
• Low-risk retrieval (e.g., document search, FAQ lookup): Set τ low (0.75-0.80) to maximize coverage

You measure performance in production by tracking:

• Abstention rate: Percentage of questions refused
• Answered accuracy: Correctness of committed answers (requires ground truth or human review)
• Business impact: Downstream cost of wrong answers versus manual review cost

Start with τ=0.90 and adjust based on observed error rates and business feedback.

Failure Modes

CoSQ does not solve all hallucination problems. Known failure modes include:

Overconfident self-assessment: The model may incorrectly assess that it has sufficient information. This is the same calibration problem that affects all LLM confidence scores.

Prompt sensitivity: The self-questioning prompt structure affects results. Small wording changes can shift the abstention rate by 5-10%.

Domain mismatch: The framework was evaluated on TruthfulQA and Natural Questions. Performance on specialized financial or technical domains is unknown.

Adversarial questions: Deliberately misleading or trick questions may bypass the self-assessment logic.

To mitigate these:

• Monitor abstention patterns for sudden shifts that indicate prompt drift
• A/B test prompt variations to find stable formulations
• Combine with external verification for highest-stakes decisions
• Log and review abstained questions to identify systematic gaps

Deployment Shape

CoSQ fits naturally into agent orchestration layers that already use prompt chaining. The integration points are:

Orchestration layer: Add the assessment step before the answer step in your agent loop. Most frameworks (LangChain, LlamaIndex, AutoGen) support conditional branching based on LLM output.

Observability: Log confidence scores, abstention reasons, and threshold values to your telemetry pipeline. This data feeds threshold tuning and failure analysis.

Fallback routing: When the agent abstains, route to a fallback handler (human review queue, retrieval-augmented generation, or external API).

State management: Store abstention decisions in your agent state so downstream steps can react appropriately (e.g., escalate to human, retry with more context, or skip the question).

The framework requires no changes to model serving infrastructure. It works with any LLM API that supports prompt-based generation.

When to Use CoSQ

CoSQ makes sense when:

• Wrong answers are costly: Financial advice, medical triage, legal research, compliance checks
• You control the prompt but not the model: Using hosted APIs (OpenAI, Anthropic, Cohere) where fine-tuning is expensive or unavailable
• Coverage can flex: You can route abstained questions to human review or alternative workflows
• You need tunable risk: Business requirements change and you want to adjust the answer-or-refuse boundary without retraining

Avoid CoSQ when:

• Coverage is non-negotiable: Every question must receive an answer, even if uncertain
• Latency is critical: The extra LLM call for assessment is unacceptable
• You have labeled data and model access: Fine-tuning for abstention may be more efficient
• External verification is already in place: Adding self-assessment on top of fact-checking is redundant

Technical Verdict

CoSQ is a practical guardrail pattern for production agents where wrong answers have asymmetric costs. It requires no model retraining, integrates cleanly into existing prompt chains, and provides a tunable control surface for the answer-or-refuse decision.

The 32% reduction in wrong commitments is meaningful for financial workflows where a single hallucinated number can trigger compliance issues or trading losses. The 12% coverage cost is manageable if you have fallback routing for abstained questions.

The main limitation is prompt sensitivity. You will need to test and tune the self-questioning prompt for your domain, and monitor for drift over time. The framework also inherits the calibration problems of LLM confidence scores, so do not treat high-confidence assessments as ground truth.

While this paper is published in cs.CL (Computation and Language), we cover it in the financial category because agent abstention is critical infrastructure for financial workflows. The ability to refuse answers when factual support is weak directly addresses the production risk profile of trading systems, compliance agents, and advisory tools where wrong commitments have measurable business costs.

Use this when you need prompt-only risk control and can tolerate selective refusal. Skip it if you need guaranteed coverage or already have external verification infrastructure.

Source Links

• ArXiv Paper (2609.17516v1)
• PDF