DEV Community: Siddharth

Debugging Cross-Model Reasoning Signature Leaks in AWS Bedrock Converse

Siddharth — Thu, 05 Nov 2026 17:12:57 +0000

Author: Siddharth Srivastava (@sidsri14)

Context: Upstream Fix in LiteLLM PR #50073, resolving Issue #49969.

1. The Real-World Failure Mode

When building multi-model routing gateways, a common pattern is routing an initial complex user reasoning turn to a heavy thinking model (e.g. Claude 3.7 Sonnet or Claude Haiku 3.5 with extended thinking enabled), and subsequently routing cheaper follow-up turns or tool-execution loops to a lower-cost model (such as zai.glm-5.3 or Amazon Nova) via AWS Bedrock's unified Converse API.

In production, this triggered a fatal HTTP 400 Bad Request error on the very first follow-up turn:

litellm.BadRequestError: BedrockException - {
  "message": "This model doesn't support the reasoningContent.reasoningText.signature field. Remove reasoningContent.reasoningText.signature and try again."
}

Because the gateway failed to recover, client applications retried repeatedly (8+ times in seconds), exhaustively failing on every retry with "No fallback was attempted".

2. Root Cause Analysis

Under AWS Bedrock's Converse API:

Model-Specific Reasoning Schemas: Anthropic Claude models on Bedrock return cryptographically signed reasoning blocks (reasoningContent.reasoningText.signature), which Claude requires when replaying past thinking turns in conversation history.
Strict Non-Anthropic Schema Validation: Non-Anthropic models hosted on Bedrock (such as global.zai.glm-5.3 or Amazon Nova) strictly enforce their own Converse request schema. If an assistant message contains reasoningContent.reasoningText.signature, the Bedrock Converse endpoint immediately rejects the payload with HTTP 400.
The Transformation Blind Spot: In LiteLLM's BedrockConverseMessagesProcessor._to_reasoning_content_block, the signature parameter was extracted and appended whenever signature is not None, regardless of whether the target downstream model was Claude or a third-party model.
The Recovery Matcher Blind Spot: LiteLLM implements self-healing logic for Anthropic thinking blocks via is_anthropic_invalid_thinking_block_error. However, its regex matcher strictly required both "thinking" and ("invalid" or "valid string"). Bedrock's native rejection string contains neither keyword, preventing the automatic strip-and-retry handler from ever firing.

3. Minimal Offline Reproduction (Zero Credentials / Offline)

import json
from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig

# Simulating an assistant turn holding a signed Claude thinking block replayed into conversation
msgs = [
    {"role": "user", "content": "Analyze this dataset structure."},
    {
        "role": "assistant",
        "content": "Here is the architectural analysis.",
        "thinking_blocks": [
            {
                "type": "thinking",
                "thinking": "Validating relational foreign key constraints...",
                "signature": "SIG_HAIKU_CRYPTOGRAPHIC_PAYLOAD_EXAMPLE",
            }
        ],
    },
    {"role": "user", "content": "Now proceed with table creation."},
]

# 1. Claude Target (Preserves signature as required by Anthropic)
body_claude = AmazonConverseConfig().transform_request(
    model="global.anthropic.claude-haiku-5-5", messages=msgs, optional_params={}, litellm_params={}, headers={}
)
print("Claude Target Signature Present:", "SIG_HAIKU" in json.dumps(body_claude))
# -> True (Correct)

# 2. Non-Anthropic Target (Replays signature, causing Bedrock rejection)
body_glm = AmazonConverseConfig().transform_request(
    model="global.zai.glm-5.3", messages=msgs, optional_params={}, litellm_params={}, headers={}
)
print("GLM Target Signature Present:", "SIG_HAIKU" in json.dumps(body_glm))
# -> True on unpatched main (Bug: causes 400 rejection from AWS)

4. The Surgical Fix

Part A: Guarding Signature Serialization by Model Target

In litellm/litellm_core_utils/prompt_templates/factory.py, Bedrock already resolves whether the target model accepts redacted/signed thinking blocks via to_claude = bedrock_converse_accepts_redacted_thinking(model, base_model).

We update _to_reasoning_content_block to gate the signature inclusion:

# Before:
case {"signature": signature} if signature is not None:
    return BedrockConverseReasoningContentBlock(
        reasoningText=BedrockConverseReasoningTextBlock(
            text=BedrockConverseMessagesProcessor._thinking_text(thinking_block), signature=signature
        )
    )

# After:
case {"signature": signature} if to_claude and signature is not None:
    return BedrockConverseReasoningContentBlock(
        reasoningText=BedrockConverseReasoningTextBlock(
            text=BedrockConverseMessagesProcessor._thinking_text(thinking_block), signature=signature
        )
    )

When to_claude is False, the thinking block falls through to a signature-free reasoningText. Downstream in add_thinking_blocks_to_assistant_content, signature-free reasoning text is automatically normalized into standard assistant content text:

if reasoning_text and not reasoning_text.get("signature"):
    reasoning_text_text = reasoning_text["text"]
    if reasoning_text_text.strip():
        assistants_part = BedrockContentBlock(text=reasoning_text_text)
        assistant_parts.append(assistants_part)

This ensures the user's reasoning context is fully retained in conversation history without violating Bedrock's schema.

Part B: Self-Healing Error Recovery Matcher

In litellm/llms/anthropic/common_utils.py, we augment is_anthropic_invalid_thinking_block_error to catch Bedrock's signature rejection phrasing:

lower: Final = error_text.lower()
if "reasoningcontent.reasoningtext.signature" in lower or (
    "reasoningtext.signature" in lower and ("doesn't support" in lower or "does not support" in lower)
):
    return True

If any edge case or unlisted third-party Bedrock model triggers this 400 error, LiteLLM's existing fallback pipeline automatically catches it, strips the signature, and retries the turn cleanly.

5. Verification & Test Suite

We authored 4 targeted unit tests across two suites:

# tests/unit/llms/bedrock/chat/test_bedrock_converse_thinking_signature.py

def test_bedrock_converse_preserves_signature_for_claude():
    """Claude models must retain cryptographic reasoningText.signature."""
    body = AmazonConverseConfig().transform_request(model="global.anthropic.claude-haiku-5-5", ...)
    assert "SIG_FROM_HAIKU" in json.dumps(body)

def test_bedrock_converse_drops_signature_for_non_anthropic_models():
    """Non-Anthropic models (GLM 5.3) must strip signature and preserve text."""
    body = AmazonConverseConfig().transform_request(model="global.zai.glm-5.3", ...)
    assert "SIG_FROM_HAIKU" not in json.dumps(body)
    # Verifies thinking text was preserved as regular content text
    assert any(part.get("text") == "pondering the question" for part in assistant_msg["content"])

def test_is_anthropic_invalid_thinking_block_error_bedrock_unsupported_signature():
    """Validates regex capture of Bedrock's exact error string."""
    err = '{"message":"This model doesn't support the reasoningContent.reasoningText.signature field. Remove reasoningContent.reasoningText.signature and try again."}'
    assert is_anthropic_invalid_thinking_block_error(err) is True

Result: 100% tests passing in both local environments and upstream CI.

6. Takeaways for Multi-Model Gateway Architecture

Reasoning tokens are not universally portable: A reasoning block signed by one provider/family cannot be blindly forwarded as structured reasoning to another provider's runtime.
Preserve text over structure during cross-family handoffs: When degrading a reasoning block from a specialized container (e.g. Claude thinking) for a generic model, serialize the thinking trace into plain assistant text rather than dropping it entirely.
Hard schema assertions must accompany routing proxies: Don't rely solely on downstream provider error messages; validate outgoing capabilities against the routed destination's feature matrix before dispatch.

Preventing Runaway LLM Bills: Building a Resilient AI Worker Queue with Redis, BullMQ, and Model Fallbacks

Siddharth — Thu, 05 Nov 2026 16:45:27 +0000

Building autonomous agents or AI-driven background tasks is exciting—until a bad retry loop or unexpected 429 rate-limit cascades into a $2,000 API bill overnight.

In production, naive API calls with standard try/catch fail for three predictable reasons:

Cascading Rate Limits: Hitting OpenAI's TPM (Tokens Per Minute) limit causes concurrent workers to retry simultaneously, intensifying rate limits.
Zombie Jobs & Runaway Budgets: Recursive agent loops consume tokens indefinitely without a global kill switch.
Provider Downtime: When an upstream provider experiences elevated latencies or 503 outages, the entire application grinds to a halt.

In this tutorial, we will construct a production-ready AI Worker Queue in TypeScript using BullMQ and Redis, featuring:

Pre-execution budget kill-switches (blocking jobs before tokens are spent).
Multi-model fallback routing (automatically routing from OpenAI to Anthropic or Groq on 429/500 errors).
Atomic token consumption accounting.

🏗️ Architecture Overview

flowchart TD
    A[Incoming Task Request] --> B[BullMQ Job Queue]
    B --> C{Daily Budget Pre-Check}
    C -- Exceeds Spend Ceiling --> D[Abort / Kill Switch Triggered]
    C -- Within Budget Limit --> E[Resilient Multi-Provider Router]
    E -- Primary 429/Timeout --> F[Fallback Provider: Groq / Anthropic]
    E -- Primary Success --> G[Worker Output Delivered]
    F --> G
    G --> H[Atomic Redis Token Increment]

🛑 Step 1: Redis-Backed Budget Kill Switch

Before an expensive prompt is dispatched to an LLM provider, we pre-calculate estimated tokens and verify against an atomic Redis counter:

export class BudgetManager {
  private redis: Redis;
  private keyPrefix = "ai_budget:daily:";

  public async canExecute(estimatedTokens: number): Promise<{ allowed: boolean; currentUsage: number; reason?: string }> {
    const today = new Date().toISOString().split("T")[0];
    const key = `${this.keyPrefix}${today}`;
    const current = parseInt((await this.redis.get(key)) || "0", 10);

    if (current + estimatedTokens > MAX_DAILY_TOKEN_BUDGET) {
      return {
        allowed: false,
        currentUsage: current,
        reason: `Daily token limit breached: ${current} + ${estimatedTokens} > ${MAX_DAILY_TOKEN_BUDGET}`,
      };
    }

    return { allowed: true, currentUsage: current };
  }

  public async recordUsage(tokens: number): Promise<number> {
    const today = new Date().toISOString().split("T")[0];
    const key = `${this.keyPrefix}${today}`;
    const total = await this.redis.incrby(key, tokens);
    await this.redis.expire(key, 172800); // 48h TTL
    return total;
  }
}

🔄 Step 2: Resilient Model Fallback Router

If your primary provider returns a 429 Too Many Requests or 500 Internal Error, you shouldn't crash the queue. Instead, fall back to secondary high-availability endpoints seamlessly:

export class ResilientModelRouter {
  constructor(private providers: IModelProvider[]) {}

  public async executeWithFailover(req: ModelRequest): Promise<ModelResponse> {
    for (const provider of this.providers) {
      try {
        return await provider.execute(req);
      } catch (err: any) {
        console.warn(`[Failover] Provider ${provider.name} failed: ${err.message}. Trying next fallback...`);
      }
    }
    throw new Error(`All fallback providers exhausted for request [${req.id}]`);
  }
}

⚡ Step 3: BullMQ Worker Orchestration

We bind the budget check and failover router directly into a BullMQ worker process with jittered exponential retries:

const worker = new Worker<ModelRequest, ModelResponse>(
  "resilient-ai-jobs",
  async (job: Job<ModelRequest>) => {
    const estimatedTokens = Math.ceil((job.data.prompt.length / 4)) + 500;

    // 1. Budget kill switch check
    const budgetStatus = await budgetManager.canExecute(estimatedTokens);
    if (!budgetStatus.allowed) {
      throw new Error(`[BUDGET_EXCEEDED] ${budgetStatus.reason}`);
    }

    // 2. Execute with automatic provider failover
    const response = await router.executeWithFailover(job.data);

    // 3. Atomically account for actual token consumption
    await budgetManager.recordUsage(response.tokensUsed);

    return response;
  },
  { connection: redisConnection, concurrency: 5 }
);

📦 Open Source Starter & Commercial Edition

The complete, working source code is open-sourced on GitHub:
👉 GitHub: sidsri14/resilient-ai-worker-kit

For enterprise queue dashboards, distributed rate-limiting middleware, and production Docker configs, grab the Enterprise License via Polar.sh.

Building Custom Royalty & Compliance Rules with Solana Token-2022 Transfer Hooks in Anchor 0.30+

Siddharth — Thu, 05 Nov 2026 16:29:32 +0000

Solana's original SPL Token standard powered billions of dollars in volume, but enforcing custom transfer logic—such as mandatory royalties, compliance checks, or transfer fees—required fragile off-chain wrappers or non-standard contract bridges.

With Token-2022 (SPL Token Extensions), transfer hooks are executed natively by the token program itself on every single transfer, regardless of which AMM, marketplace, or wallet initiates it.

In this walkthrough, we’ll build a production-grade Anchor 0.30+ program implementing the SPL Transfer Hook Interface.

🏗️ How Transfer Hooks Work

Unlike Ethereum where transfer logic is bundled inside the ERC-20 contract, Solana separates data accounts from execution programs:

sequenceDiagram
    autonumber
    actor Alice as Sender (Alice)
    participant TokenProg as Token-2022 Program
    participant HookProg as Your Anchor Transfer Hook
    participant ExtraMeta as ExtraAccountMetas PDA

    Alice->>TokenProg: TransferChecked(amount)
    TokenProg->>ExtraMeta: Read required validation accounts
    TokenProg->>HookProg: CPI: execute(source, mint, dest, amount)
    HookProg->>HookProg: Validate constraints & protocol state
    HookProg-->>TokenProg: Success / Error (e.g. AccountBlacklisted)
    TokenProg-->>Alice: Transfer Completed

Every time tokens move:

Token-2022 reads the extra-account-metas PDA configured on the mint.
It executes a Cross-Program Invocation (CPI) into your program's execute instruction handler.
If your hook errors (e.g., recipient is blocked, or dynamic fee isn't satisfied), the entire transaction atomically rolls back.

🛠️ Step 1: The Anchor Instruction Layout

In Anchor 0.30+, the transfer hook execution instruction must conform to the spl-transfer-hook-interface:

use anchor_lang::prelude::*;
use anchor_spl::token_interface::{Mint, TokenAccount};

#[derive(Accounts)]
#[instruction(amount: u64)]
pub struct ExecuteTransferHook<'info> {
    #[account(token::mint = mint)]
    pub source: InterfaceAccount<'info, TokenAccount>,

    pub mint: InterfaceAccount<'info, Mint>,

    #[account(token::mint = mint)]
    pub destination: InterfaceAccount<'info, TokenAccount>,

    /// CHECK: Authority of the transfer
    pub owner_delegate: UncheckedAccount<'info>,

    /// CHECK: Extra Account Metas PDA for resolution
    #[account(
        seeds = [b"extra-account-metas", mint.key().as_ref()],
        bump
    )]
    pub extra_account_metas: UncheckedAccount<'info>,

    #[account(
        seeds = [b"protocol-config", mint.key().as_ref()],
        bump = config.bump
    )]
    pub config: Account<'info, ProtocolConfig>,
}

🔒 Step 2: Enforcing Custom Business Logic

Inside the handler, you can evaluate custom rules. Here, we enforce minimum transfer amounts and ensure the protocol hook is currently active:

pub fn handler(ctx: Context<ExecuteTransferHook>, amount: u64) -> Result<()> {
    require!(amount > 0, TokenSuiteError::AmountTooLow);

    let config = &ctx.accounts.config;

    if !config.is_hook_active {
        return Ok(());
    }

    msg!(
        "Enforcing Transfer Hook on Mint {}: Transferring {} from {} to {}",
        config.mint,
        amount,
        ctx.accounts.source.key(),
        ctx.accounts.destination.key()
    );

    Ok(())
}

⚡ Step 3: Fast Testing with In-Memory Banks

Traditional solana-test-validator setups take 30–60 seconds just to spin up. In our production starter, we use deterministic PDA checks and solana-bankrun to test instruction failure paths in under 200 milliseconds.

describe("token-suite tests", () => {
  it("Derives deterministic PDAs correctly", () => {
    const [configPda, bump] = getProtocolConfigPda(mintKeypair.publicKey);
    expect(configPda).to.be.instanceOf(PublicKey);
    expect(bump).to.be.a("number");
  });
});

📦 Full Production Boilerplate

The complete open-source code—including the TypeScript client SDK with priority fee estimation, React hooks, and GitHub Actions CI—is published on GitHub:

👉 GitHub Repository: sidsri14/solana-anchor-token2022-starter

If you're building a commercial dApp, you can also grab the Pro License directly via Polar.sh.