DEV Community

ZAKARIA KHCHICHE
ZAKARIA KHCHICHE

Posted on

Locating a signature in a scanned form with Jev 1.14 (TypeSafe) and Microsoft Agent Framework

#ai #python #opensource #agents

TL;DR: Jev 1.14 (TypeSafe AI, October 14, 2026) accepts images and a new Box question type that returns the bounding box of one target in an image. You can use it today with the TypeSafe Python SDK, and I proposed support in the Microsoft Agent Framework TypeSafe connector (PR #9411, awaiting review). TypeSafe flags both features as experimental ("Danger Zone"): prototypes only.

The use case: is the form signed, and where?

On document agents, scanned files cause a large share of failures: signed forms, stamps, handwritten fields. Before extracting anything, two simple questions often matter: is the document signed? and where is the signature?

Question Jev type Answer
Is the form signed? Noul probability from 0 to 1
Where is the signature? Box top, bottom, left, right as fractions of the image, plus confidence

Requirements: model jev-1.14.0 or jev-preview, header TypeSafe-Danger-Zone-Images: true, and an image in the request.

Today: with the TypeSafe SDK 

from pathlib import Path
from typesafe_sdk import AsyncTypeSafeClient, Box, Image, Noul

client = AsyncTypeSafeClient()
result = await client.system_one(
    state=Image(Path("form.png")),
    questions={
        "signed": Noul(instructions="Is the form signed?"),
        "signature": Box(instructions="the signature on the form"),
    },
    model="jev-1.14.0",
    extra_headers={"TypeSafe-Danger-Zone-Images": "true"},
)
signature = result.answers["signature"].box  # fractions of width and height
Enter fullscreen mode Exit fullscreen mode

Proposed: from a Microsoft Agent Framework agent

The connector currently rejects any non-text content. PR #9411 maps inline image content to TypeSafe image assets and adds the header only when the request contains an image or a Box question; text-only requests are unchanged.

response = await client.get_response(
    [Message("user", ["The scanned form:", Content.from_data(data=png_bytes, media_type="image/png")])],
    options={
        "model": "jev-1.14.0",
        "response_format": {
            "signed": Noul(instructions="Is the form signed?"),
            "signature": Box(instructions="the signature on the form"),
        },
    },
)
Enter fullscreen mode Exit fullscreen mode

This API is a proposal under review and may change before it is merged.

Three pitfalls

  1. No "not found" answer. Box always returns a box, even when the target is absent. Ask a Noul presence question in the same request and ignore the box when it is low.
  2. One box per question. Two signatures means two questions, each described so they cannot be confused ("the signature at the bottom right").
  3. Small targets in large images. Crop the region first, then ask again on the crop. TypeSafe reports that checking each edge with a Noul question caught 30 of 36 clipped boxes in its tests.

Takeaways

  • Jev 1.14 adds images and Box, still experimental: keep them out of production for now.
  • Always pair Box with a presence Noul.
  • Build a test set from your own scans before choosing a confidence threshold; TypeSafe recommends none.

Sources: TypeSafe blog, Jev 1.14 · Box documentation · Issue #9410

Zakaria Khchiche, freelance Tech Lead Data & AI, 200+ AI agents in production. LinkedIn · Website

Want your team to build agents like these? I run a hands-on Copilot Studio training (in French, with Spar-x, Qualiopi-certified, eligible for OPCO funding in France): https://zakariakhchiche.github.io/formation-copilot-studio/ and a free AI Act article 4 kit: https://zakariakhchiche.github.io/kit-ai-act/

Top comments (0)