DEV Community: Xiao Ling

How to Share a TWAIN Scanner Securely Over a LAN with Python FastAPI and Dynamic Web TWAIN

Xiao Ling — Tue, 02 Jun 2026 07:01:33 +0000

Shared TWAIN scanners are awkward to expose on a local network because direct LAN access does not solve identity, concurrency, or auditability. This tutorial shows how to share a TWAIN scanner securely with a Python FastAPI gateway and the Dynamic Web TWAIN Service REST API, so registered users can sign in, see shared scanner availability, and capture pages from a browser without direct access to the scanner host.

What you'll build: A Python FastAPI gateway that authenticates users, leases a shared TWAIN scanner to one operator at a time, and returns scanned pages to the browser with Dynamic Web TWAIN.

Demo Video: Secure Gateway Sign-In, Scanner Locking, and Page Preview

Prerequisites

Python 3.10 or later
Dynamic Web TWAIN Service installed on the gateway machine
A Dynamic Web TWAIN license with REST API access
FastAPI, Uvicorn, PyJWT, and python-multipart from the sample requirements
A current Dynamic Web TWAIN Service build. The repo does not pin a service version, so validate compatibility with the service once it is installed.

Get a 30-day free trial license for testing the gateway with Dynamic Web TWAIN.

Step 1: Install and Configure the Gateway

The sample keeps the web stack intentionally small, then reads the scanner host, JWT secret, admin bootstrap, and lock TTL from environment variables.

fastapi==0.115.12
uvicorn[standard]==0.30.6
PyJWT==2.9.0
python-multipart==0.0.9
twain-wia-sane-scanner==2.0.3

DWT_LICENSE_KEY=
DWT_SERVICE_HOST=http://127.0.0.1:18622
REMOTE_SCAN_JWT_SECRET=replace-with-a-long-random-secret
ACCESS_TOKEN_TTL_MINUTES=120
SCANNER_LOCK_TTL_SECONDS=600
REMOTE_SCAN_SCANNER_TYPES=0x50
DWT_SERVICE_VERIFY=true
# Optional: pre-create an admin account at startup
REMOTE_SCAN_ADMIN_USERNAME=
REMOTE_SCAN_ADMIN_PASSWORD=
REMOTE_SCAN_ADMIN_FULL_NAME=

Step 2: Store Users, Admin Roles, and Scanner Leases in SQLite

The gateway creates a WAL-mode SQLite database with an is_admin column on users, auto-promotes the earliest account when no admin exists, and enforces lock leases for scanner contention.

def init_database() -> None:
    DATABASE_PATH.parent.mkdir(parents=True, exist_ok=True)
    with get_connection() as connection:
        connection.execute('PRAGMA journal_mode=WAL')
        connection.execute(
            '''
            CREATE TABLE IF NOT EXISTS users (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                username TEXT NOT NULL UNIQUE,
                full_name TEXT NOT NULL,
                password_hash TEXT NOT NULL,
                created_at TEXT NOT NULL
            )
            '''
        )
        user_columns = {
            row['name']
            for row in connection.execute('PRAGMA table_info(users)').fetchall()
        }
        if 'is_admin' not in user_columns:
            connection.execute('ALTER TABLE users ADD COLUMN is_admin INTEGER NOT NULL DEFAULT 0')
        connection.execute(
            '''
            UPDATE users
            SET is_admin = 1
            WHERE id = (
                SELECT id
                FROM users
                ORDER BY created_at ASC, id ASC
                LIMIT 1
            )
            AND NOT EXISTS (
                SELECT 1
                FROM users
                WHERE is_admin = 1
            )
            '''
        )
        connection.execute(
            '''
            CREATE TABLE IF NOT EXISTS scanner_locks (
                scanner_id TEXT PRIMARY KEY,
                scanner_name TEXT NOT NULL,
                owner_username TEXT NOT NULL,
                lock_token TEXT NOT NULL,
                status TEXT NOT NULL,
                job_uid TEXT,
                acquired_at TEXT NOT NULL,
                expires_at TEXT NOT NULL
            )
            '''
        )
        connection.execute(
            '''
            CREATE TABLE IF NOT EXISTS audit_log (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                username TEXT NOT NULL,
                action TEXT NOT NULL,
                scanner_id TEXT,
                detail TEXT,
                created_at TEXT NOT NULL
            )
            '''
        )

Step 3: Register Users and Issue JWT Access Tokens

Registration stores a PBKDF2 hash in SQLite, auto-assigns the first registrant as administrator, and the login flow returns a short-lived bearer token that every protected route validates.

class RegistrationRequest(BaseModel):
    username: str = Field(min_length=3, max_length=50)
    full_name: str = Field(min_length=1, max_length=80)
    password: str = Field(min_length=8, max_length=128)


def create_user_account(payload: RegistrationRequest) -> Dict[str, Any]:
    username = payload.username.strip().lower()
    full_name = payload.full_name.strip()
    if not is_valid_username(username):
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail='Usernames may only include letters, numbers, dots, dashes, and underscores.',
        )

    with DB_LOCK:
        with get_connection() as connection:
            existing_user = connection.execute(
                'SELECT username FROM users WHERE username = ?',
                (username,),
            ).fetchone()
            if existing_user:
                raise HTTPException(
                    status_code=status.HTTP_409_CONFLICT,
                    detail='That username is already registered.',
                )

            created_at = utcnow_string()
            admin_exists = connection.execute(
                'SELECT 1 FROM users WHERE is_admin = 1 LIMIT 1'
            ).fetchone()
            is_admin = 0 if admin_exists else 1
            connection.execute(
                'INSERT INTO users (username, full_name, password_hash, created_at, is_admin) VALUES (?, ?, ?, ?, ?)',
                (username, full_name, hash_password(payload.password), created_at, is_admin),
            )

    write_audit_log(username, 'auth.registered')
    return {
        'username': username,
        'full_name': full_name,
        'created_at': created_at,
        'is_admin': bool(is_admin),
    }

The login endpoint validates credentials and returns a signed JWT. The token carries the username in the sub claim and an exp claim set to the configured TTL.

def create_access_token(username: str) -> TokenResponse:
    expires_at = utcnow() + timedelta(minutes=ACCESS_TOKEN_TTL_MINUTES)
    access_token = jwt.encode({'sub': username, 'exp': expires_at}, JWT_SECRET, algorithm=JWT_ALGORITHM)
    return TokenResponse(
        access_token=access_token,
        expires_in=ACCESS_TOKEN_TTL_MINUTES * 60,
    )

Every protected route extracts the token from the Authorization: Bearer header, decodes it with the shared secret, and looks up the user in SQLite. If the token is missing, expired, or maps to a deleted account, the gateway returns HTTP 401.

def get_current_user(token: str = Depends(oauth2_scheme)) -> Dict[str, Any]:
    authentication_error = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail='Could not validate credentials.',
        headers={'WWW-Authenticate': 'Bearer'},
    )
    try:
        payload = jwt.decode(token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
    except jwt.InvalidTokenError:
        raise authentication_error

    username = payload.get('sub')
    if not username:
        raise authentication_error

    user_record = get_user_record(username)
    if not user_record:
        raise authentication_error

    return {
        'username': user_record['username'],
        'full_name': user_record['full_name'],
        'created_at': user_record['created_at'],
        'is_admin': bool(user_record['is_admin']),
    }

Step 4: Lock Each Shared Scanner Before Starting a Job

The gateway never starts a scan until it has written a lease record for the selected scanner, which is what lets other users see status without stealing the device mid-job.

def acquire_lock(scanner_id: str, scanner_name: str, username: str) -> Optional[Dict[str, Any]]:
    with DB_LOCK:
        with get_connection() as connection:
            prune_expired_locks(connection)
            existing_lock = connection.execute(
                'SELECT scanner_id, scanner_name, owner_username, status, job_uid, acquired_at, expires_at FROM scanner_locks WHERE scanner_id = ?',
                (scanner_id,),
            ).fetchone()
            if existing_lock and existing_lock['owner_username'] != username:
                return {
                    'scanner_id': existing_lock['scanner_id'],
                    'scanner_name': existing_lock['scanner_name'],
                    'owner_username': existing_lock['owner_username'],
                    'status': existing_lock['status'],
                    'job_uid': existing_lock['job_uid'],
                    'acquired_at': existing_lock['acquired_at'],
                    'expires_at': existing_lock['expires_at'],
                }

            now = utcnow_string()
            expires_at = (utcnow() + timedelta(seconds=SCANNER_LOCK_TTL_SECONDS)).strftime('%Y-%m-%dT%H:%M:%SZ')
            connection.execute(
                '''
                INSERT INTO scanner_locks (
                    scanner_id, scanner_name, owner_username, lock_token, status, job_uid, acquired_at, expires_at
                ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
                ON CONFLICT(scanner_id) DO UPDATE SET
                    scanner_name = excluded.scanner_name,
                    owner_username = excluded.owner_username,
                    lock_token = excluded.lock_token,
                    status = excluded.status,
                    job_uid = excluded.job_uid,
                    acquired_at = excluded.acquired_at,
                    expires_at = excluded.expires_at
                ''',
                (scanner_id, scanner_name, username, uuid.uuid4().hex, 'pending', '', now, expires_at),
            )
    return None

conflicting_lock = acquire_lock(scanner_id, scanner.get('name', 'Unknown scanner'), current_user['username'])
if conflicting_lock:
    raise HTTPException(
        status_code=status.HTTP_423_LOCKED,
        detail={
            'message': 'Scanner is currently locked by another user.',
            'locked_by': conflicting_lock['owner_username'],
            'status': conflicting_lock['status'],
            'expires_at': conflicting_lock['expires_at'],
        },
    )

job = scanner_controller.createJob(
    SERVICE_HOST,
    {
        'license': LICENSE_KEY,
        'device': scanner['device'],
        'autoRun': False,
        'jobTimeout': payload.job_timeout,
        'scannerFailureTimeout': payload.scanner_failure_timeout,
        'requestFocusForScanningUI': False,
        'checkFeederLoaded': payload.feeder_enabled,
        'config': {
            'IfShowUI': payload.show_ui,
            'PixelType': payload.pixel_type,
            'Resolution': payload.resolution,
            'IfFeederEnabled': payload.feeder_enabled,
            'IfDuplexEnabled': payload.duplex_enabled,
            'IfCloseSourceAfterAcquire': True,
        },
    },
)
job_uid = job.get('jobuid', '')
if not job_uid:
    raise HTTPException(
        status_code=status.HTTP_502_BAD_GATEWAY,
        detail=job or scanner_controller.last_error or {'message': 'Failed to create scan job.'},
    )

update_lock(scanner_id, current_user['username'], 'pending', job_uid=job_uid)

start_result = scanner_controller.updateJob(SERVICE_HOST, job_uid, {'status': JobStatus.RUNNING})
if start_result.get('status') not in (JobStatus.RUNNING, JobStatus.COMPLETED):
    raise HTTPException(
        status_code=status.HTTP_502_BAD_GATEWAY,
        detail=start_result or scanner_controller.last_error or {'message': 'Failed to start scan job.'},
    )

update_lock(scanner_id, current_user['username'], 'scanning', job_uid=job_uid)

Step 5: Stream Scanned Pages Back to the Browser Dashboard

Once the lock is held and the job is running, the backend drains the scan job into base64 strings and the frontend renders them as data URLs.

images = scanner_controller.getImageStreams(SERVICE_HOST, job_uid, imageType=payload.image_type)
job_info = scanner_controller.checkJob(SERVICE_HOST, job_uid)
if job_info.get('status') == JobStatus.FAULTED:
    raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=job_info)

encoded_images = [base64.b64encode(image).decode('utf-8') for image in images]
write_audit_log(
    current_user['username'],
    'scan.completed',
    scanner_id=scanner_id,
    detail='{count} page(s)'.format(count=len(encoded_images)),
)
return {
    'scanner': {'id': scanner_id, 'name': scanner['name']},
    'page_count': len(encoded_images),
    'image_type': payload.image_type,
    'job_status': job_info.get('status', JobStatus.COMPLETED),
    'images': encoded_images,
}

Step 6: Build the Browser Dashboard for Scanner Selection and Live Status

The frontend stores the bearer token in sessionStorage, polls scanner status every five seconds, and disables the scan button when another user owns the lease.

async function api(path, options = {}) {
  const headers = new Headers(options.headers || {});
  if (state.token) {
    headers.set("Authorization", `Bearer ${state.token}`);
  }
  if (options.json) {
    headers.set("Content-Type", "application/json");
  }

  const response = await fetch(path, {
    method: options.method || "GET",
    headers,
    body: options.json ? JSON.stringify(options.json) : options.body,
  });

  const contentType = response.headers.get("content-type") || "";
  const payload = contentType.includes("application/json")
    ? await response.json()
    : await response.text();

  if (!response.ok) {
    throw payload;
  }
  return payload;
}

function renderScanners(scanners) {
  state.scanners = scanners;
  syncSelectedScanner(scanners);

  if (!scanners.length) {
    scannerSelect.innerHTML = '<option value="">No scanners available</option>';
    scannerSelect.disabled = true;
    renderSelectedScanner();
    return;
  }

  scannerSelect.disabled = false;
  scannerSelect.innerHTML = scanners.map((scanner) => {
    const selected = scanner.id === state.selectedScannerId ? " selected" : "";
    const lockLabel = scanner.locked ? ` (${scanner.locked_by || "locked"})` : "";
    return `<option value="${scanner.id}"${selected}>${scanner.name}${lockLabel}</option>`;
  }).join("");
  renderSelectedScanner();
}

function renderSelectedScanner() {
  const scanner = getSelectedScanner();
  if (!scanner) {
    scannerDetail.innerHTML = '<div class="empty-state">No shared TWAIN scanners are available right now.</div>';
    scanSelectedButton.disabled = true;
    return;
  }

  const isLockedByOther = scanner.locked && scanner.locked_by !== state.currentUser.username;
  const availabilityClass = scanner.locked ? "locked" : "free";
  const stateClass = scanner.lock_status || "idle";
  const ownerLine = scanner.locked
    ? `Locked by <strong>${scanner.locked_by}</strong> until ${scanner.lock_expires_at}`
    : "Available for the next authenticated user.";

  scannerDetail.innerHTML = `
    <article class="scanner-summary-card">
      <div class="scanner-card-header">
        <div>
          <h3>${scanner.name}</h3>
          <p class="scanner-meta">ID: ${scanner.id}</p>
        </div>
        <span class="lock-chip ${availabilityClass}">${scanner.locked ? "Locked" : "Ready"}</span>
      </div>
      <p class="scanner-meta">${ownerLine}</p>
      <div class="scanner-card-footer">
        <span class="state-chip ${stateClass}">${scanner.lock_status}</span>
        <span class="scanner-meta">Selected scanner</span>
      </div>
    </article>
  `;
  scanSelectedButton.disabled = isLockedByOther;
  scanSelectedButton.textContent = isLockedByOther
    ? `Locked by ${scanner.locked_by}`
    : "Scan Selected Scanner";
}

async function runScan(scannerId, scannerName) {
  const payload = {
    resolution: Number(document.getElementById("resolutionInput").value),
    pixel_type: Number(document.getElementById("pixelTypeInput").value),
    image_type: document.getElementById("imageTypeInput").value,
    feeder_enabled: document.getElementById("feederInput").checked,
    duplex_enabled: document.getElementById("duplexInput").checked,
    show_ui: document.getElementById("showUiInput").checked,
  };

  try {
    setBanner(`Locking ${scannerName} and waiting for scanned pages...`);
    const result = await api(`/api/scanners/${scannerId}/scan`, {
      method: "POST",
      json: payload,
    });
    renderResults(result);
    setBanner(`Completed ${result.page_count} page(s) on ${scannerName}.`, "success");
  } catch (error) {
    setBanner(formatError(error), "error");
  } finally {
    await loadScanners().catch(() => {});
  }
}

Step 7: Add an Administrator Workspace for User Management

Administrator accounts see a dedicated User Management panel instead of scanner controls. They can delete accounts, toggle admin roles, and the first registered user is automatically granted admin rights.

function setSignedInState(user) {
  state.currentUser = user;
  sessionState.innerHTML = `<span class="status-dot"></span><span>Signed in as ${user.username}</span>`;
  workspaceRolePill.textContent = user.is_admin ? "Administrator Workspace" : "Operator Workspace";

  // Role-appropriate sections
  scanControlsSection.hidden = user.is_admin;
  scannerStatusSection.hidden = user.is_admin;
  usersPanel.hidden = !user.is_admin;
  resultPanel.hidden = user.is_admin;
  workspaceSection.classList.toggle("admin-view", user.is_admin);

  // Switch panel title
  workspaceTitle.textContent = user.is_admin ? "User Management" : "Scanner Operations";
  workspaceCopy.textContent = user.is_admin
    ? "Delete accounts or change a user's administrator permissions."
    : "Controls and status change based on your account role.";

  // Admin badge is redundant — "Signed in as ..." already shows username
  userBadge.hidden = user.is_admin;

  setAuthMode("hidden");
  syncLayout();
}

Source Code

https://github.com/yushulx/python-twain-wia-sane-scanner/tree/main/webexample

Build a Python QR Code and Page Number OCR Scanner with PySide6 and Dynamsoft Capture Vision

Xiao Ling — Wed, 27 May 2026 02:27:59 +0000

Answer sheets, notebooks, and worksheet pages often carry two identifiers at once: a QR code for machine lookup and a printed page number for human ordering. This Python desktop app reads both from the same fixed-layout page image by combining barcode decoding and OCR inside Dynamsoft Capture Vision, then shows the results in a PySide6 desktop viewer with drag-and-drop, a file list, and auto-detection on selection. The Python implementation uses dynamsoft-capture-vision-bundle>=3.4.2001 and keeps barcode decoding plus OCR in one file-first capture flow.

What you'll build: A PySide6 desktop app that scans a QR code and printed page number from the same page image with Dynamsoft Capture Vision and overlays both results during review.

Demo Video: Python Page QR + OCR Scanner in Action

Step 1: Review the Prerequisites and Install the Python Dependencies

Python 3.9 or newer
A working desktop Python environment for PySide6
A Dynamsoft license key for Capture Vision
The project dependencies from page_qr_ocr/requirements.txt

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

The project keeps the GUI runtime and the synthetic test-set generator in one requirements file.

PySide6>=6.5
opencv-python>=4.8
numpy>=1.24
dynamsoft-capture-vision-bundle>=3.4.2001
qrcode>=8.2
Pillow>=10.0

Step 2: Configure the Template So OCR Follows the QR Code

The key template idea is that OCR does not use a fixed page crop. Instead, TargetROIDefOptions defines an ROI relative to the detected barcode, and NumberCharRecognition keeps the OCR model numeric while the Python code avoids hard-coded page-number length heuristics.

{
  "Name": "roi-recognize-text-barcode",
  "TaskSettingNameArray": [
    "task-recognize-text"
  ],
  "Location": {
    "ReferenceObjectFilter": {
      "AtomicResultTypeArray": ["ART_BARCODE"]
    },
    "Offset": {
      "ReferenceObjectOriginIndex": 0,
      "ReferenceObjectType": "ROT_ATOMIC_OBJECT",
      "MeasuredByPercentage": 1,
      "FirstPoint": [ -300, -100 ],
      "SecondPoint": [ -100, -100 ],
      "ThirdPoint": [ -100, 0 ],
      "FourthPoint": [ -300, 0 ]
    }
  }
}

"LabelRecognizerTaskSettingOptions": [
  {
    "Name": "task-recognize-text",
    "TextLineSpecificationNameArray": [
      "tls-textlines"
    ],
    "SectionArray": [
      {
        "Section": "ST_REGION_PREDETECTION",
        "ImageParameterName": "ip-recognize-textlines"
      },
      {
        "Section": "ST_TEXT_LINE_LOCALIZATION",
        "ImageParameterName": "ip-recognize-textlines"
      },
      {
        "Section": "ST_TEXT_LINE_RECOGNITION",
        "ImageParameterName": "ip-recognize-textlines",
        "StageArray": [
          {
            "Stage": "SST_RECOGNIZE_RAW_TEXT_LINES"
          },
          {
            "Stage": "SST_ASSEMBLE_TEXT_LINES",
            "StringLengthRange": [ 1, 64 ]
          }
        ]
      }
    ]
  }
],
"TextLineSpecificationOptions": [
  {
    "Name": "tls-textlines",
    "CharacterModelName": "NumberCharRecognition",
    "OutputResults": 1,
    "StringLengthRange": [ 1, 64 ]
  }
]

Step 3: Initialize Capture Vision and Load the Template File

The scanner initializes the license, resolves the active template name from the JSON, and loads the settings into CaptureVisionRouter.

from dynamsoft_capture_vision_bundle import (
    CaptureVisionRouter,
    EnumImagePixelFormat,
    LicenseManager,
)


class CaptureVisionPageScanner:
    def __init__(self, template_path: Path) -> None:
        self._template_path = template_path
        self._init_license()
        self._template_name = self._resolve_template_name(template_path)

        self._template_router = CaptureVisionRouter()
        err, msg = self._template_router.init_settings_from_file(str(template_path))
        if err != 0:
            raise RuntimeError(f"Failed to load template file: {msg}")

    @staticmethod
    def _init_license() -> None:
        err, msg = LicenseManager.init_license(LICENSE_KEY)
        if err != 0:
            print(f"[DCV] License warning ({err}): {msg}")

Step 4: Capture the Page Once and Fan Out Barcode and OCR Results

The important runtime choice is to use file-based capture when the image path is available. After that, the code extracts barcode items and recognized text lines from the same capture result and forwards them through scanner-layer callbacks.

def detect(
    self,
    image_bgr: np.ndarray,
    image_path: Optional[Path] = None,
    on_barcodes: Optional[BarcodeResultCallback] = None,
    on_text_lines: Optional[TextResultCallback] = None,
) -> ScanResult:
    logs: List[str] = []

    captured = None
    scale = 1.0

    if image_path is not None and image_path.exists():
        logs.append(
            f"[CAPTURE] template={self._template_name}, source=file"
        )
        captured = self._capture_with_template_file(image_path, self._template_name)
    else:
        logs.append(
            f"[CAPTURE] template={self._template_name}, source=array"
        )
        captured = self._capture_with_template(image_bgr, self._template_name)

    if captured is None:
        logs.append("[CAPTURE] file/array capture returned no result, fallback=array")
        captured = self._capture_with_template(image_bgr, self._template_name)

    err_code = int(captured.get_error_code())
    err_msg = captured.get_error_string() or ""
    logs.append(f"[CAPTURE] err={err_code}, msg={err_msg}")

    variant = "oneshot/file" if image_path is not None and image_path.exists() else "oneshot/array"

    barcodes = self._extract_barcodes(captured, variant)
    for hit in barcodes:
        hit.points = self._rescale_points(hit.points, scale)
    barcodes = self._dedupe_barcodes(barcodes)

    text_lines = self._extract_text_lines(captured, variant)
    for hit in text_lines:
        hit.points = self._rescale_points(hit.points, scale)
    text_lines = self._dedupe_text_lines(text_lines)

    if on_barcodes is not None:
        on_barcodes(barcodes)
        logs.append(f"[CALLBACK] on_barcodes: {len(barcodes)}")
    if on_text_lines is not None:
        on_text_lines(text_lines)
        logs.append(f"[CALLBACK] on_text_lines: {len(text_lines)}")

    page_number = self._pick_page_number(text_lines, barcodes)
    logs.append(
        f"[SUMMARY] barcodes={len(barcodes)}, text_lines={len(text_lines)}, page_number={page_number}"
    )
    return ScanResult(barcodes=barcodes, text_lines=text_lines, page_number=page_number, logs=logs)

Step 5: Score the OCR Hits and Pick the Best Page Number

With NumberCharRecognition and the barcode-referenced ROI in the template, the recognized text lines are already page-number candidates. The Python code therefore uses the returned text directly and only scores each OCR hit against the QR code geometry and the OCR confidence.

@staticmethod
def _pick_page_number(
    text_hits: Sequence[TextHit],
    barcode_hits: Sequence[BarcodeHit],
) -> Optional[str]:
    anchor: Optional[Tuple[float, float, float, float]] = None
    if barcode_hits:
        primary = max(barcode_hits, key=lambda hit: hit.confidence)
        if primary.points:
            xs = [p[0] for p in primary.points]
            ys = [p[1] for p in primary.points]
            cx = (min(xs) + max(xs)) * 0.5
            cy = (min(ys) + max(ys)) * 0.5
            bw = max(max(xs) - min(xs), 1.0)
            bh = max(max(ys) - min(ys), 1.0)
            anchor = (cx, cy, bw, bh)

    candidates: List[Tuple[float, str]] = []
    for hit in text_hits:
        raw = hit.text.strip()
      if not raw:
            continue

        if hit.points:
            xs = [p[0] for p in hit.points]
            ys = [p[1] for p in hit.points]
            box_w = max(xs) - min(xs)
            box_h = max(ys) - min(ys)
            if box_w < 6.0 or box_h < 6.0:
                continue

        score = float(hit.confidence) * 10.0

        if anchor and hit.points:
            ax, ay, aw, ah = anchor
            hx = (min(xs) + max(xs)) * 0.5
            hy = (min(ys) + max(ys)) * 0.5
            dx = ax - hx
            dy = ay - hy

            expected_dx = 2.0 * aw
            expected_dy = 1.0 * ah
            score -= (abs(dx - expected_dx) / aw) * 8.0
            score -= (abs(dy - expected_dy) / ah) * 4.0

            if dx <= 0:
                score -= 25.0

        candidates.append((score, raw))

    if not candidates:
        return None
    candidates.sort(key=lambda item: item[0], reverse=True)
    return candidates[0][1]

Step 6: Build a Drag-and-Drop File Browser for Multi-Page Review

The current PySide6 window is not a single-image viewer anymore. It keeps a file list on the left, accepts dropped files from both the image view and the list, and exposes Prev, Next, and Clear Images in the top bar.

self._view = ImageView(self._scene, self)
self._view.files_dropped.connect(self._add_paths)

self._file_list = FileListWidget(self)
self._file_list.setAlternatingRowColors(True)
self._file_list.setSelectionMode(QAbstractItemView.SingleSelection)
self._file_list.currentRowChanged.connect(self._on_file_selected)
self._file_list.files_dropped.connect(self._add_paths)

self._prev_btn = QPushButton("< Prev")
self._prev_btn.clicked.connect(self._prev_image)
self._next_btn = QPushButton("Next >")
self._next_btn.clicked.connect(self._next_image)

def _build_ui(self) -> None:
    load_btn = QPushButton("Load Images...")
    load_btn.clicked.connect(self._on_load_images)

    clear_btn = QPushButton("Clear Images")
    clear_btn.clicked.connect(self._clear_images)

    top_bar = QHBoxLayout()
    top_bar.addWidget(load_btn)
    top_bar.addWidget(clear_btn)
    top_bar.addSpacing(12)
    top_bar.addWidget(self._prev_btn)
    top_bar.addWidget(self._nav_label)
    top_bar.addWidget(self._next_btn)
    top_bar.addSpacing(12)
    top_bar.addWidget(self._toggle_log_btn)
    top_bar.addStretch(1)

Step 7: Auto-Trigger Detection When the Selected Image Changes

The auto-detect behavior now lives in the image-loading path, not in a manual detect button. When the user selects a file from the list or navigates with Prev and Next, _load_image_at_index() loads the image, updates the status bar, redraws the scene, and immediately calls _on_detect().

def _on_file_selected(self, row: int) -> None:
    if row < 0 or row >= len(self._file_paths):
        return
    self._load_image_at_index(row)

def _load_image_at_index(self, index: int) -> None:
    if index < 0 or index >= len(self._file_paths):
        return

    image_path = self._file_paths[index]
    image_bgr = cv2.imread(str(image_path))
    if image_bgr is None:
        QMessageBox.warning(self, "Load Failed", f"Cannot open image: {image_path}")
        return

    self._current_index = index

    self._image_bgr = image_bgr
    self._image_path = image_path
    self._scan_result = None
    self._image_rect = None

    self._status_label.setText(f"Image {index + 1}/{len(self._file_paths)}: {image_path}")
    self._barcode_label.setText("Barcodes: 0")
    self._page_number_label.setText("Page number: -")
    self._log_box.setPlainText("")

    self._redraw_scene()
    self._update_navigation()
    self._on_detect()

Step 8: Render Results and Reset the Session Cleanly

The overlay drawing is still isolated in _redraw_scene(), and the new _clear_images() method resets the entire session in one step: file list, selected image, overlays, logs, and navigation state.

def _clear_images(self) -> None:
    self._file_paths.clear()
    self._current_index = -1
    self._image_bgr = None
    self._image_path = None
    self._scan_result = None
    self._image_rect = None

    self._file_list.clear()
    self._scene.clear()
    self._scene.setSceneRect(QRectF())
    self._view.resetTransform()

    self._status_label.setText("Load images to start.")
    self._barcode_label.setText("Barcodes: 0")
    self._page_number_label.setText("Page number: -")
    self._log_box.setPlainText("")
    self._update_navigation()

def _redraw_scene(self) -> None:
    self._scene.clear()
    self._image_rect = None
    if self._image_bgr is None:
        return

    pixmap = self._to_qpixmap(self._image_bgr)
    pixmap_item = self._scene.addPixmap(pixmap)
    self._image_rect = pixmap_item.boundingRect()
    self._scene.setSceneRect(self._image_rect)

    if self._scan_result is not None:
        for hit in self._scan_result.barcodes:
            label = f"{hit.fmt}: {hit.text}"
            self._add_polygon(hit.points, Qt.blue, label)

        for hit in self._scan_result.text_lines:
            if not hit.text:
                continue
            label = f"OCR: {hit.text}"
            self._add_polygon(hit.points, Qt.red, label)

    self._view.resetTransform()
    if self._image_rect is not None and not self._image_rect.isNull():
        self._view.fitInView(self._image_rect, Qt.KeepAspectRatio)

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/page_qr_ocr

How to Build an iOS MRZ Scanner with SwiftUI: Real-Time Passport and ID Card Recognition

Xiao Ling — Fri, 22 May 2026 06:59:25 +0000

Passports and national ID cards carry Machine Readable Zones (MRZ) — standardized text lines that encode holder identity, document numbers, and expiry dates. Reading these zones reliably from a mobile camera is critical for border control, hotel check-in, and KYC workflows. This tutorial walks through building a SwiftUI iOS app that captures, verifies, and parses MRZ data in real-time using the Dynamsoft Capture Vision SDK (v3.4).

What you'll build: A SwiftUI iOS app that detects document boundaries, reads MRZ text lines from passports and ID cards, extracts portrait photos, and displays structured results — all from a live camera feed.

Demo Video

Prerequisites

Xcode 15+ on macOS
iOS 16.0+ deployment target
A physical iOS device with a camera (simulators do not support camera input)
A Dynamsoft Capture Vision license key

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Create the Xcode Project and Add SDK Dependencies

Create a new SwiftUI app project in Xcode. Add the two required SPM packages via File → Add Package Dependencies:

Package	Repository
`capture-vision-spm`	`https://github.com/Dynamsoft/capture-vision-spm`
`mrz-scanner-spm`	`https://github.com/Dynamsoft/mrz-scanner-spm`

capture-vision-spm provides the core engine (CameraEnhancer, CaptureVisionRouter, ImageProcessor). mrz-scanner-spm provides the MRZ-specific pipeline (IdentityProcessor, code parser, and MRZ templates).

Add camera permission to Info.plist by setting the following build setting in your Xcode target:

INFOPLIST_KEY_NSCameraUsageDescription = Camera is required for scanning ID documents and reading MRZ data.

Step 2: Initialize the License and Configure the Capture Vision Pipeline

Create a ScannerController class that conforms to both CapturedResultReceiver and IntermediateResultReceiver. This controller owns the entire Dynamsoft pipeline.

import SwiftUI
import UIKit
import Combine
import DynamsoftCaptureVisionBundle

@MainActor
class ScannerController: NSObject, ObservableObject,
    CapturedResultReceiver, IntermediateResultReceiver
{
    @Published var statusText: String = ""
    @Published var mrzResultText: String = ""
    @Published var isCapturing: Bool = true
    @Published var pendingLabelMap: [String: String]?
    @Published var pendingPortraitImage: UIImage?
    @Published var showResult: Bool = false
    @Published var errorMessage: String?

    let cameraView = CameraView()
    private let dce = CameraEnhancer()
    private let cvr = CaptureVisionRouter()
    private let idProcessor = IdentityProcessor()
}

In the setup() method, initialize the license, configure the camera, set up the cross-verification filter, and load the MRZ template:

func setup() {
    // Initialize license
    MrzParser.initLicense()

    // Camera setup
    dce.cameraView = cameraView
    dce.enableEnhancedFeatures(.frameFilter)

    // Cross-verification filter for stable multi-frame results
    let filter = MultiFrameResultCrossFilter()
    filter.enableResultCrossVerification([.textLine, .detectedQuad], isEnabled: true)
    let criteria = CrossVerificationCriteria()
    criteria.frameWindow = 5
    criteria.minConsistentFrames = 2
    filter.setResultCrossVerificationCriteria(criteria, resultItemTypes: .detectedQuad)
    cvr.addResultFilter(filter)

    // Load MRZ template configuration
    if let templatePath = Bundle.main.path(forResource: "mrz-mobile", ofType: "json") {
        try? cvr.initSettingsFromFile(templatePath)
    }

    // Set camera as input and configure pipeline
    try? cvr.setInput(dce)

    if let settings = try? cvr.getSimplifiedSettings("ReadPassportAndId") {
        settings.documentSettings?.minQuadrilateralAreaRatio = 2
        try? cvr.updateSettings("ReadPassportAndId", settings: settings)
    }

    // Register result receivers
    cvr.getIntermediateResultManager().addResultReceiver(self)
    cvr.addResultReceiver(self)

    configureDrawingLayers()
}

The MultiFrameResultCrossFilter is essential — it requires at least 2 consistent frames within a 5-frame window before accepting a document quad, which eliminates flickering and false detections.

Step 3: Bridge the Camera View to SwiftUI

Dynamsoft's CameraView is a UIKit UIView. Wrap it with UIViewRepresentable for SwiftUI:

import SwiftUI
import DynamsoftCaptureVisionBundle

struct CameraPreview: UIViewRepresentable {
    let cameraView: CameraView

    func makeUIView(context: Context) -> UIView {
        let container = UIView()
        container.backgroundColor = .black
        cameraView.translatesAutoresizingMaskIntoConstraints = false
        container.addSubview(cameraView)
        NSLayoutConstraint.activate([
            cameraView.topAnchor.constraint(equalTo: container.topAnchor),
            cameraView.bottomAnchor.constraint(equalTo: container.bottomAnchor),
            cameraView.leadingAnchor.constraint(equalTo: container.leadingAnchor),
            cameraView.trailingAnchor.constraint(equalTo: container.trailingAnchor),
        ])
        return container
    }

    func updateUIView(_ uiView: UIView, context: Context) {}
}

Use this in your ContentView with .ignoresSafeArea() for a full-screen camera preview.

Step 4: Configure Drawing Layers for Live Document Overlays

The CameraView exposes a DrawingLayer system for rendering overlays on the camera feed. Configure three layers during setup:

private var portraitLayerId: UInt = 100

private func configureDrawingLayers() {
    // Make preset layers visible for auto-drawing
    cameraView.getDrawingLayer(1)?.visible = true   // DDN layer (document quad)
    cameraView.getDrawingLayer(3)?.visible = true   // DLR layer (MRZ text lines)

    // Create a custom portrait layer with cyan stroke style
    let portraitStyleId = DrawingStyleManager.createDrawingStyle(
        .cyan, strokeWidth: 3,
        fill: UIColor.cyan.withAlphaComponent(0.1),
        textColor: .white, font: .systemFont(ofSize: 12)
    )
    let portraitLayer = cameraView.createDrawingLayer()
    portraitLayer.visible = true
    portraitLayer.setDefaultStyle(portraitStyleId)
    portraitLayerId = portraitLayer.layerId
}

Layer 1 (DDN) draws the detected document boundary as a quadrilateral
Layer 3 (DLR) draws rectangles around recognized MRZ text lines
Custom layer (100+) draws the portrait zone with a distinct cyan style

When detection results arrive, update the overlays:

private func drawOverlays(result: CapturedResult, portraitZone: Quadrilateral?) {
    let ddnLayer = cameraView.getDrawingLayer(1)
    if let quadItem = result.processedDocumentResult?.detectedQuadResultItems?.first {
        ddnLayer?.clearDrawingItems()
        ddnLayer?.addDrawingItems([QuadDrawingItem(quadrilateral: quadItem.location)])
    } else {
        ddnLayer?.clearDrawingItems()
    }

    let portraitLayer = cameraView.getDrawingLayer(portraitLayerId)
    if let pz = portraitZone {
        portraitLayer?.clearDrawingItems()
        portraitLayer?.addDrawingItems([QuadDrawingItem(quadrilateral: pz)])
    } else {
        portraitLayer?.clearDrawingItems()
    }
}

Each overlay is cleared when its source data disappears, keeping the UI in sync with what the camera actually sees.

Step 5: Receive and Process MRZ Recognition Results

Implement CapturedResultReceiver to handle final pipeline output — parsed MRZ fields and document quads:

public func onCapturedResultReceived(_ result: CapturedResult) {
    guard let parsedItem = result.parsedResult?.items?.first else { return }
    let map = MrzParser.parse(parsedItem)
    guard !map.isEmpty else { return }

    let quadItem = result.processedDocumentResult?.detectedQuadResultItems?.first
    var portraitZone = findPortraitZone()

    // Validate portrait is inside document and proportionally reasonable
    if let pz = portraitZone, let quad = quadItem {
        let docRegion = quad.location
        let allInside = pz.points.allSatisfy { docRegion.contains($0.cgPointValue) }
        let areaRatioOk = docRegion.area / pz.area >= 3
        if !allInside || !areaRatioOk {
            portraitZone = nil
        }
    }

    // Crop portrait from original image
    var portrait: UIImage?
    if let pz = portraitZone,
       let imageData = cvr.getIntermediateResultManager().getOriginalImage(result.originalImageHashId) {
        portrait = try? ImageProcessor().cropAndDeskewImage(imageData, quad: pz).toUIImage()
    }

    DispatchQueue.main.async { [weak self] in
        self?.drawOverlays(result: result, portraitZone: portraitZone)
    }

    DispatchQueue.main.async { [weak self] in
        self?.pendingLabelMap = map
        self?.pendingPortraitImage = portrait
        self?.isCapturing = false
    }
}

The portrait validation step is important: it checks that the detected portrait zone lies entirely within the document boundary and that the document is at least 3× larger than the portrait, preventing false positives.

Step 6: Extract Portrait Zone from Intermediate Pipeline Results

Portrait detection uses intermediate pipeline data rather than a separate model. Implement IntermediateResultReceiver to collect the required units:

public func onScaledColourImageUnitReceived(_ unit: ScaledColourImageUnit, info: IntermediateResultExtraInfo) {
    DispatchQueue.main.async { self.scaledColourImageUnit = unit }
}

public func onLocalizedTextLinesReceived(_ unit: LocalizedTextLinesUnit, info: IntermediateResultExtraInfo) {
    DispatchQueue.main.async { self.localizedTextLinesUnit = unit }
}

public func onDetectedQuadsReceived(_ unit: DetectedQuadsUnit, info: IntermediateResultExtraInfo) {
    DispatchQueue.main.async { self.detectedQuadsUnit = unit }
}

public func onDeskewedImageReceived(_ unit: DeskewedImageUnit, info: IntermediateResultExtraInfo) {
    DispatchQueue.main.async { self.deskewedImageUnit = unit }
}

Then use IdentityProcessor to compute the portrait zone, filtering by confidence:

private func findPortraitZone() -> Quadrilateral? {
    guard let scaledUnit = scaledColourImageUnit,
          let localizedUnit = localizedTextLinesUnit,
          let textLinesUnit = recognizedTextLinesUnit,
          let quadsUnit = detectedQuadsUnit,
          quadsUnit.getCount() > 0,
          let imageUnit = deskewedImageUnit,
          let elements = localizedUnit.getAuxiliaryRegionElements() else {
        return nil
    }

    var hasHighConfidence = false
    for element in elements {
        if element.getName() == "PortraitZone" && element.getConfidence() > 60 {
            hasHighConfidence = true
            break
        }
    }
    guard hasHighConfidence else { return nil }

    return idProcessor.findPortraitZone(
        scaledUnit,
        localizedTextLinesUnit: localizedUnit,
        recognizedTextLinesUnit: textLinesUnit,
        detectedQuadsUnit: quadsUnit,
        deskewedImageUnit: imageUnit
    )
}

The confidence threshold of 60 filters out low-quality portrait detections from cluttered backgrounds.

Step 7: Parse MRZ Fields into Structured Data

The ParsedResultItem from Capture Vision contains raw MRZ fields. Create an MrzParser that extracts and formats them:

struct MrzParser {
    private static let licenseKey = "YOUR_LICENSE_KEY"

    static func initLicense() {
        LicenseManager.initLicense(licenseKey, verificationDelegate: nil)
    }

    static func parse(_ item: ParsedResultItem) -> [String: String] {
        let entry = item.parsedFields
        var properties: [String: String] = [:]

        let codeType = item.codeType
        var docType = "PASSPORT"
        if codeType.contains("TD1") || codeType.contains("ID") {
            docType = "ID"
        } else if codeType.contains("VISA") {
            docType = "VISA"
        }

        let number = getFirstNonNull(entry, keys: ["passportNumber", "documentNumber", "idNumber"])
        let firstName = getFirstNonNull(entry, keys: ["secondaryIdentifier", "givenNames"])
        let lastName = getFirstNonNull(entry, keys: ["primaryIdentifier", "lastName"])

        properties["Document Type"] = docType
        properties["Name"] = lastName.isEmpty ? firstName : "\(lastName), \(firstName)"
        properties["Document Number"] = number.isEmpty ? "—" : number
        properties["Nationality"] = entry["nationality"] ?? "Unknown"
        properties["Issuing State"] = entry["issuingState"] ?? "Unknown"
        // ... additional fields: sex, age, dates
        return properties
    }
}

The getFirstNonNull helper tries multiple field name variants because different MRZ formats use different key names. TD3 passports use passportNumber, while TD1/TD2 ID cards use documentNumber or idNumber.

Step 8: Display Structured Results in a Result View

Present the parsed data in a SwiftUI result view with profile header, document info card, and personal info card:

struct ScanResultView: View {
    let labelMap: [String: String]
    let portraitImage: UIImage?
    @Environment(\.dismiss) private var dismiss

    var body: some View {
        ScrollView {
            VStack(spacing: 20) {
                profileHeader
                documentInfoCard
                personalInfoCard
                Button(action: { dismiss() }) {
                    HStack {
                        Image(systemName: "camera.rotate")
                        Text("Scan Again")
                    }
                    .font(.headline)
                    .foregroundColor(.white)
                    .frame(maxWidth: .infinity)
                    .padding()
                    .background(Color.blue)
                    .cornerRadius(12)
                }
                .padding(.horizontal)
            }
        }
        .navigationTitle("Scan Result")
    }
}

The result view shows the cropped portrait image (or a placeholder), the holder's name, document type badge, and structured field cards. Dismissing the view triggers .onAppear on ContentView, which restarts scanning automatically.

Step 9: Add the MRZ Template Configuration File

The Capture Vision pipeline is configured via a JSON template file. Add mrz-mobile.json to your Xcode project target. This template defines three capture modes:

ReadPassportAndId — The default mode that handles both passports and ID cards
ReadPassport — Optimized for TD3 passports (2 lines × 44 characters)
ReadId — Optimized for TD1 (3 lines × 30 characters) and TD2 (2 lines × 36 characters) ID cards

The template specifies text line recognition parameters including character set, line count, and expected character length per format, plus the code parser configuration that converts raw MRZ text into structured fields.

Step 10: Build the Full-Screen Camera UI with Overlays

Wire up the full-screen camera view with status text, MRZ result overlay, and capture button:

struct ContentView: View {
    @StateObject private var controller = ScannerController()

    var body: some View {
        ZStack {
            CameraPreview(cameraView: controller.cameraView)
                .ignoresSafeArea()

            VStack {
                HStack {
                    if !controller.statusText.isEmpty {
                        Text(controller.statusText)
                            .font(.subheadline)
                            .foregroundColor(.white)
                            .padding(.horizontal, 12)
                            .padding(.vertical, 6)
                            .background(Color.black.opacity(0.6))
                            .cornerRadius(8)
                    }
                    Spacer()
                }
                .padding(.horizontal)

                Spacer()

                if !controller.mrzResultText.isEmpty {
                    Text(controller.mrzResultText)
                        .font(.system(.caption, design: .monospaced))
                        .foregroundColor(.green)
                        .padding(8)
                        .background(Color.black.opacity(0.7))
                        .cornerRadius(8)
                        .padding(.horizontal)
                }

                // Capture button
                HStack {
                    Spacer()
                    Button(action: { controller.onCapture() }) {
                        ZStack {
                            Circle().stroke(Color.white, lineWidth: 3)
                                .frame(width: 68, height: 68)
                            Circle().fill(controller.isCapturing ? Color.gray : Color.white)
                                .frame(width: 58, height: 58)
                            if controller.isCapturing {
                                ProgressView()
                                    .progressViewStyle(CircularProgressViewStyle(tint: .white))
                            }
                        }
                    }
                    .disabled(controller.isCapturing || controller.pendingLabelMap == nil)
                    Spacer()
                }
                .padding(.bottom, 32)
            }
        }
        .navigationBarHidden(true)
        .navigationDestination(isPresented: $controller.showResult) {
            ScanResultView(
                labelMap: controller.pendingLabelMap ?? [:],
                portraitImage: controller.pendingPortraitImage
            )
        }
        .onAppear {
            controller.setup()
            controller.startScanning()
        }
        .onDisappear {
            controller.stopScanning()
        }
    }
}

The capture button is disabled until MRZ data is available (pendingLabelMap != nil) and shows a spinner while the pipeline is still capturing.

Source Code

https://github.com/yushulx/ios-swiftui-barcode-mrz-document-scanner/tree/main/examples/MrzScanner

Dynamsoft Barcode SDK: Build Mobile Barcode Scanners with React Native, Flutter, and .NET MAUI

Xiao Ling — Wed, 20 May 2026 06:36:42 +0000

Choosing the right cross-platform framework for a mobile barcode scanning app means weighing API ergonomics against rendering performance and developer experience. This article compares React Native, Flutter, and .NET MAUI side by side on Android and iOS, each integrated with the Dynamsoft Capture Vision SDK, to show how they differ in setup, camera control, barcode decoding, and overlay rendering when building real-time scanning apps.

What you'll build: A real-time mobile barcode scanner that decodes barcodes from a live camera feed and draws bounding-box overlays on each detected code, implemented in three frameworks using the Dynamsoft Capture Vision SDK.

Prerequisites

Framework	SDK	Version	Dev Environment
Flutter	`dynamsoft_capture_vision_flutter`	3.4.1200	Flutter 3.x, Dart SDK ≥3.3.0
React Native	`dynamsoft-capture-vision-react-native`	3.4.1200	Node ≥18, React Native 0.79
.NET MAUI	`Dynamsoft.CaptureVisionBundle.Maui`	3.4.1200	.NET 10 SDK, Visual Studio 2022

Get a 30-day free trial license for Dynamsoft Barcode Reader.

Initialize the Dynamsoft Barcode SDK License

All three frameworks require license initialization before any scanning can proceed, but the placement and error handling differ.

Flutter — called in the root widget's initState():

Future<void> _initLicense() async {
  try {
    final licenseResult = await LicenseManager.initLicense(
        'LICENSE-KEY');
    if (kDebugMode) print('License init: $licenseResult');
  } catch (e) {
    if (kDebugMode) print(e);
  }
}

React Native — called at module level before component render:

const License = 'LICENSE-KEY';
LicenseManager.initLicense(License).catch(e => {
  Alert.alert('License error', e.message);
});

.NET MAUI — called in MauiProgram.cs during app startup. The mobile SDK uses the same LicenseManager from the Capture Vision Bundle:

string license = "LICENSE-KEY";
string errorMsg;
int errorCode = LicenseManager.InitLicense(license, out errorMsg);
if (errorCode != (int)Dynamsoft.Core.EnumErrorCode.EC_OK)
    Debug.WriteLine("License initialization error: " + errorMsg);

Comparison: Flutter and React Native both use async/await patterns with exceptions; .NET MAUI uses a synchronous call with error codes. React Native's module-level initialization is the most concise, while Flutter's widget-lifecycle approach gives more control over timing. .NET MAUI's synchronous approach is straightforward but requires platform-specific placement.

Install and Configure the SDK

Flutter — a single package covers all Dynamsoft functionality:

# pubspec.yaml
dependencies:
  dynamsoft_capture_vision_flutter: ^3.4.1200
  provider: ^6.1.5+1
  url_launcher: ^6.3.2
  share_plus: ^12.0.1
  image_picker: ^1.2.1
  flutter_exif_rotation: ^0.5.2

React Native — two packages: the core router and the barcode reader bundle:

{
  "dependencies": {
    "dynamsoft-capture-vision-react-native": "^3.4.1200",
    "dynamsoft-barcode-reader-bundle-react-native": "^11.4.1200",
    "react": "19.0.0",
    "react-native": "0.79.0"
  }
}

.NET MAUI — the Dynamsoft.CaptureVisionBundle.Maui package covers Android and iOS:

<ItemGroup Condition="'$(TargetFramework)' == 'net10.0-android36.0' Or '$(TargetFramework)' == 'net10.0-ios26.0'">
    <PackageReference Include="Dynamsoft.CaptureVisionBundle.Maui" Version="3.4.1200" />
</ItemGroup>

<ItemGroup>
    <PackageReference Include="SkiaSharp.Views.Maui.Controls" Version="3.116.1" />
    <PackageReference Include="CommunityToolkit.Maui" Version="9.0.1" />
</ItemGroup>

Comparison: Flutter wins on simplicity — one package, one import. React Native's two-package split is still straightforward. .NET MAUI sits in the middle with a single Capture Vision Bundle but needs additional dependencies like SkiaSharp for overlay drawing.

Set Up the Camera and Start Capturing

All three frameworks follow the same Capture Vision architecture: create a CameraEnhancer, create a CaptureVisionRouter, bind them together, and start capturing with a preset template.

Flutter:

Future<void> _sdkInit() async {
  _cvr = CaptureVisionRouter.instance;
  _cameraEnhancer = CameraEnhancer.instance;

  SimplifiedCaptureVisionSettings? currentSettings =
      await _cvr.getSimplifiedSettings(EnumPresetTemplate.readBarcodes);
  currentSettings!.barcodeSettings!.barcodeFormatIds =
      EnumBarcodeFormat.all;
  currentSettings.barcodeSettings!.expectedBarcodesCount = 0;
  await _cvr.updateSettings(EnumPresetTemplate.readBarcodes, currentSettings);

  _cvr.setInput(_cameraEnhancer);

  final CapturedResultReceiver receiver = CapturedResultReceiver()
    ..onDecodedBarcodesReceived = (DecodedBarcodesResult result) async {
      List<BarcodeResultItem>? res = result.items;
      if (mounted) {
        decodeRes = res ?? [];
        setState(() {});
      }
    };
  _cvr.addResultReceiver(receiver);

  await _cvr.startCapturing(EnumPresetTemplate.readBarcodes);
  await _cameraEnhancer.open();
}

React Native:

const cvr = CaptureVisionRouter.getInstance();
const camera = CameraEnhancer.getInstance();

useEffect(() => {
  CameraEnhancer.requestCameraPermission();
  cvr.setInput(camera);
  camera.setCameraView(cameraView.current);

  const receiver = cvr.addResultReceiver({
    onDecodedBarcodesReceived: ({ items }) => {
      if (items?.length) {
        const overlays = items.map((item) => ({
          id: `${item.text}-${item.formatString}`,
          text: item.text,
          formatString: item.formatString,
          corners: item.location.points.map(point => ({
            x: point.x * scaleFactor - cropOffsetX,
            y: point.y * scaleFactor - cropOffsetY
          })),
        }));
        setBarcodeOverlays(overlays);
      }
    },
  });

  camera.open();
  cvr.startCapturing(EnumPresetTemplate.PT_READ_BARCODES)
    .catch(e => Alert.alert('Start error', e.message));

  return () => {
    cvr.stopCapturing();
    camera.close();
    cvr.removeResultReceiver(receiver);
  };
}, [cvr, camera]);

.NET MAUI (Android/iOS):

public AndroidCameraPage()
{
    InitializeComponent();

    CameraPreview = new Dynamsoft.CameraEnhancer.Maui.CameraView();
    CameraPreview.SizeChanged += OnImageSizeChanged;
    MainGrid.Children.Insert(0, CameraPreview);

    enhancer = new CameraEnhancer();
    router = new CaptureVisionRouter();
    router.SetInput(enhancer);
    router.AddResultReceiver(this);
}

protected override void OnHandlerChanged()
{
    base.OnHandlerChanged();
    if (this.Handler != null && enhancer != null)
    {
        enhancer.SetCameraView(CameraPreview);
        enhancer.Open();
    }
}

protected override async void OnAppearing()
{
    base.OnAppearing();
    await Permissions.RequestAsync<Permissions.Camera>();
    router?.StartCapturing(EnumPresetTemplate.PT_READ_BARCODES, this);
}

Comparison: Flutter and React Native use singleton instances (CaptureVisionRouter.instance, CaptureVisionRouter.getInstance()) with callback-based result receivers. .NET MAUI uses instance construction (new CaptureVisionRouter()) and C# interface-based callbacks (ICapturedResultReceiver). Flutter and React Native initialize the camera in the same call chain as the router; .NET MAUI requires the camera view to be set in OnHandlerChanged() after the native handler is ready.

Render Barcode Overlays on the Camera Feed

Overlay rendering is where the frameworks diverge most significantly. Each uses a fundamentally different drawing approach.

Flutter uses CustomPaint with a CustomPainter that draws directly on a Canvas:

class OverlayPainter extends CustomPainter {
  final List<BarcodeResultItem> results;

  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint()
      ..color = Colors.blue
      ..strokeWidth = 30
      ..style = PaintingStyle.stroke;

    for (var result in results) {
      List<Point> points = result.location.points;

      canvas.drawLine(
          Offset(points[0].x.toDouble(), points[0].y.toDouble()),
          Offset(points[1].x.toDouble(), points[1].y.toDouble()), paint);
      canvas.drawLine(
          Offset(points[1].x.toDouble(), points[1].y.toDouble()),
          Offset(points[2].x.toDouble(), points[2].y.toDouble()), paint);
      canvas.drawLine(
          Offset(points[2].x.toDouble(), points[2].y.toDouble()),
          Offset(points[3].x.toDouble(), points[3].y.toDouble()), paint);
      canvas.drawLine(
          Offset(points[3].x.toDouble(), points[3].y.toDouble()),
          Offset(points[0].x.toDouble(), points[0].y.toDouble()), paint);

      TextPainter textPainter = TextPainter(
        text: TextSpan(
          text: result.text,
          style: const TextStyle(color: Colors.yellow, fontSize: 100.0),
        ),
        textAlign: TextAlign.center,
        textDirection: TextDirection.ltr,
      );
      textPainter.layout(minWidth: 0, maxWidth: size.width);
      textPainter.paint(canvas, Offset(minX, minY));
    }
  }
}

The camera view is wrapped in a FittedBox with BoxFit.cover, and the overlay is positioned on top:

SizedBox(
  width: screenWidth,
  height: screenHeight,
  child: FittedBox(
    fit: BoxFit.cover,
    child: Stack(
      children: [
        SizedBox(
          width: isPortrait ? _previewHeight : _previewWidth,
          height: isPortrait ? _previewWidth : _previewHeight,
          child: CameraView(cameraEnhancer: _cameraEnhancer),
        ),
        Positioned(
            left: 0, top: 0, right: 0, bottom: 0,
            child: createOverlay(decodeRes))
      ],
    ),
  ),
);

React Native draws overlays using absolutely-positioned View components — no canvas API is needed:

const BarcodeContour: React.FC<{ corners: Array<{ x: number, y: number }> }> = ({ corners }) => {
  if (corners.length !== 4) return null;

  return (
    <>
      {corners.map((corner, index) => {
        const nextCorner = corners[(index + 1) % 4];
        const deltaX = nextCorner.x - corner.x;
        const deltaY = nextCorner.y - corner.y;
        const length = Math.sqrt(deltaX * deltaX + deltaY * deltaY);
        const angle = Math.atan2(deltaY, deltaX) * (180 / Math.PI);

        return (
          <View
            key={index}
            style={[
              styles.contourLine,
              {
                left: corner.x,
                top: corner.y - 1.5,
                width: length,
                height: 3,
                transform: [{ rotate: `${angle}deg` }],
              },
            ]}
          />
        );
      })}
      {corners.map((corner, index) => (
        <View
          key={`dot-${index}`}
          style={[styles.cornerDot, { left: corner.x - 4, top: corner.y - 4 }]}
        />
      ))}
    </>
  );
};

Coordinate mapping from camera space to screen space is handled manually:

const scaleFactorX = cameraViewWidth / cameraResolutionWidth;
const scaleFactorY = cameraViewHeight / cameraResolutionHeight;
const scaleFactor = Math.max(scaleFactorX, scaleFactorY);

const scaledCameraWidth = cameraResolutionWidth * scaleFactor;
const scaledCameraHeight = cameraResolutionHeight * scaleFactor;
const cropOffsetX = (scaledCameraWidth - viewWidth) / 2;
const cropOffsetY = (scaledCameraHeight - viewHeight) / 2;

const scaledCorners = item.location.points.map(point => ({
  x: (point.x * scaleFactor) - cropOffsetX,
  y: (point.y * scaleFactor) - cropOffsetY
}));

.NET MAUI (iOS) uses SkiaSharp (SKCanvasView) for overlay rendering. The Android implementation uses a GraphicsView with a custom Drawable, while iOS uses SkiaSharp for precise drawing:

public void OnDecodedBarcodesReceived(DecodedBarcodesResult result)
{
    if (imageWidth == 0 && imageHeight == 0)
    {
        IntermediateResultManager manager = router.GetIntermediateResultManager();
        ImageData data = manager.GetOriginalImage(result.OriginalImageHashId);
        imageWidth = data.Width;
        imageHeight = data.Height;
    }

    lock (_lockObject)
    {
        _barcodeResult = result;
        CameraPreview.GetDrawingLayer(EnumDrawingLayerId.DLI_DBR).Visible = false;
        MainThread.BeginInvokeOnMainThread(() =>
        {
            canvasView.InvalidateSurface();
        });
    }
}

void OnCanvasViewPaintSurface(object sender, SKPaintSurfaceEventArgs args)
{
    SKCanvas canvas = args.Surface.Canvas;
    canvas.Clear();

    SKPaint skPaint = new SKPaint
    {
        Style = SKPaintStyle.Stroke,
        Color = SKColors.Blue,
        StrokeWidth = 4,
    };

    SKPaint textPaint = new SKPaint
    {
        Style = SKPaintStyle.Stroke,
        Color = SKColors.Red,
        StrokeWidth = 4,
    };

    float textSize = 18;
    SKFont font = new SKFont() { Size = textSize };

    lock (_lockObject)
    {
        if (_barcodeResult != null)
        {
            var items = _barcodeResult.Items;
            if (items != null)
            {
                foreach (var barcodeItem in items)
                {
                    Microsoft.Maui.Graphics.Point[] points = barcodeItem.Location.Points;
                    float x1 = (float)(points[0].X / scale);
                    float y1 = (float)(points[0].Y / scale);
                    float x2 = (float)(points[1].X / scale);
                    float y2 = (float)(points[1].Y / scale);
                    float x3 = (float)(points[2].X / scale);
                    float y3 = (float)(points[2].Y / scale);
                    float x4 = (float)(points[3].X / scale);
                    float y4 = (float)(points[3].Y / scale);

                    canvas.DrawText(barcodeItem.Text, x1, y1 - 10, SKTextAlign.Left, font, textPaint);
                    canvas.DrawLine(x1, y1, x2, y2, skPaint);
                    canvas.DrawLine(x2, y2, x3, y3, skPaint);
                    canvas.DrawLine(x3, y3, x4, y4, skPaint);
                    canvas.DrawLine(x4, y4, x1, y1, skPaint);
                }
            }
        }
    }
}

Comparison:

Aspect	Flutter	React Native	.NET MAUI
Drawing engine	CustomPainter / Canvas	Absolute-positioned Views	SkiaSharp / GraphicsView Drawable
Coordinate mapping	`FittedBox.cover` handles it automatically	Manual scale + crop offset calculation	Manual scale calculation with orientation handling
Built-in overlay option	No	No	No
Thread safety	Single-threaded (UI thread)	Single-threaded (JS thread)	Requires `lock` for cross-thread result access
Overlay complexity	Low (Canvas API)	Medium (CSS-style transforms for lines)	Medium (SkiaSharp, clean API, orientation logic)

Handle App Lifecycle and Camera State

Managing camera start/stop when the app goes to background is critical for resource management and battery life.

Flutter uses WidgetsBindingObserver:

@override
void didChangeAppLifecycleState(AppLifecycleState state) {
  super.didChangeAppLifecycleState(state);
  switch (state) {
    case AppLifecycleState.resumed:
      start();
      break;
    case AppLifecycleState.inactive:
      stop();
      break;
    default:
      break;
  }
}

React Native uses AppState:

const sub = AppState.addEventListener('change', nextState => {
  if (nextState === 'active') {
    camera.open();
    cvr.startCapturing(EnumPresetTemplate.PT_READ_BARCODES);
  } else if (nextState.match(/inactive|background/)) {
    cvr.stopCapturing();
    camera.close();
  }
});

.NET MAUI (Android) uses WeakReferenceMessenger with lifecycle events registered in MauiProgram.cs:

.ConfigureLifecycleEvents(events =>
{
#if ANDROID
    events.AddAndroid(android => android
        .OnResume((activity) => NotifyPage("Resume"))
        .OnStop((activity) => NotifyPage("Stop")));
#endif
})

Received in the page:

WeakReferenceMessenger.Default.Register<LifecycleEventMessage>(this, (r, message) =>
{
    if (message.EventName == "Resume") enhancer?.Open();
    else if (message.EventName == "Stop") enhancer?.Close();
});

Comparison: Flutter and React Native handle lifecycle directly in the scanning component. .NET MAUI's messenger pattern decouples lifecycle detection from the page, which is cleaner for larger apps but adds indirection for simple use cases.

Compare SDK Architecture and API Patterns

Feature	Flutter	React Native	.NET MAUI
Router creation	`CaptureVisionRouter.instance` (singleton)	`CaptureVisionRouter.getInstance()` (singleton)	`new CaptureVisionRouter()` (instance)
Camera creation	`CameraEnhancer.instance` (singleton)	`CameraEnhancer.getInstance()` (singleton)	`new CameraEnhancer()` (instance)
Result receiver	Callback object	Callback object	`ICapturedResultReceiver` interface
Camera view	`CameraView(cameraEnhancer: ...)`	`<CameraView ref={...} />`	`Dynamsoft.CameraEnhancer.Maui.CameraView()`
Start capturing	`startCapturing(template)` async	`startCapturing(template)` async	`StartCapturing(template, listener)`
Overlay library	Built-in Canvas	Built-in Views	SkiaSharp (3rd-party)

Flutter and React Native use singleton patterns, which simplifies access across widgets but can complicate testing. .NET MAUI uses instance construction, which is more testable and explicit but requires managing object lifetimes.

Compare Framework Metrics Side by Side

Metric	Flutter	React Native	.NET MAUI
SDK packages	1	2	1 + SkiaSharp
Camera setup	3 lines	3 lines	5 lines
Overlay approach	CustomPainter	Absolutely-positioned Views	SkiaSharp
Coordinate mapping	Automatic (FittedBox)	Manual	Manual (with orientation handling)
Router pattern	Singleton	Singleton	Instance
State management	Provider (built in)	useState hooks	CommunityToolkit.Mvvm
Thread safety	UI thread only	JS thread only	Requires explicit locking

Source Code

Build a SwiftUI iOS Document Scanner with Stable Auto Capture and PDF Export

Xiao Ling — Thu, 14 May 2026 08:01:00 +0000

This project builds a SwiftUI document scanner for iPhone and iPad that captures pages from the camera, stabilizes the detected document quad, and lets users review, edit, and export the final scan set. It uses Dynamsoft Capture Vision on iOS, so the same camera feed powers auto capture, manual fallback capture, gallery import, and deskewed export.

What you'll build: A SwiftUI iOS document scanner with live camera capture, page review, crop adjustment, and PDF or JPEG export using Dynamsoft Capture Vision.

Demo Video: Live document capture, page review, crop editing, and export on iPhone

Prerequisites

Xcode 16 or later
iOS 16 or later
A valid Dynamsoft Capture Vision license key
A Mac with SwiftPM package resolution enabled in Xcode

Get a 30-day free trial license

Step 1: Install and Configure the SDK

The project depends on the Swift Package Manager package capture-vision-spm, which provides the Capture Vision products used by the scanner. The package is pinned in the Xcode project, and the app initializes the SDK during launch.

/* Begin XCRemoteSwiftPackageReference section */
        C30000000000000000000001 /* XCRemoteSwiftPackageReference "capture-vision-spm" */ = {
            isa = XCRemoteSwiftPackageReference;
            repositoryURL = "https://github.com/Dynamsoft/capture-vision-spm";
            requirement = {
                kind = upToNextMajorVersion;
                minimumVersion = 3.4.1200;
            };
        };
/* End XCRemoteSwiftPackageReference section */

import SwiftUI
import DynamsoftCaptureVisionBundle

@main
struct DynamsoftDocumentScannerApp: App {
    @StateObject private var store = DocumentScannerStore()

    init() {
        LicenseManager.initLicense(
            "LICENSE-KEY",
            verificationDelegate: nil
        )
    }

    var body: some Scene {
        WindowGroup {
            ContentView()
                .environmentObject(store)
        }
    }
}

Step 2: Stabilize Document Detection from the Camera Feed

The scanner view creates a CameraView, attaches a CameraEnhancer, and starts the DetectAndNormalizeDocument_Default template. It also keeps a short cooldown window so the same document is not captured repeatedly.

import SwiftUI
import UIKit
import DynamsoftCaptureVisionBundle

struct CameraScannerView: UIViewControllerRepresentable {
    @EnvironmentObject private var store: DocumentScannerStore

    let manualCaptureToken: Int
    let settings: AutoCaptureSettings

    func makeCoordinator() -> Coordinator {
        Coordinator(store: store, settings: settings)
    }

    func makeUIViewController(context: Context) -> UIViewController {
        context.coordinator.makeViewController()
    }

    func updateUIViewController(_ uiViewController: UIViewController, context: Context) {
        context.coordinator.update(settings: settings, manualCaptureToken: manualCaptureToken)
    }

    static func dismantleUIViewController(_ uiViewController: UIViewController, coordinator: Coordinator) {
        coordinator.stop()
    }

    final class Coordinator: NSObject, CapturedResultReceiver {
        private struct CaptureCandidate {
            let originalImageData: ImageData?
            let normalizedImageData: ImageData
            let quad: Quadrilateral?
            let crossVerified: Bool
        }

        private unowned let store: DocumentScannerStore
        private let router = CaptureVisionRouter()
        private let stabilizer: QuadStabilizer

        private var cameraEnhancer: CameraEnhancer?
        private var latestCandidate: CaptureCandidate?
        private var cooldown = false
        private var awaitingManualCapture = false
        private var lastManualCaptureToken = 0
        private var fallbackWorkItem: DispatchWorkItem?

        init(store: DocumentScannerStore, settings: AutoCaptureSettings) {
            self.store = store
            self.stabilizer = QuadStabilizer(settings: settings)
            super.init()
        }

        func makeViewController() -> UIViewController {
            let controller = UIViewController()
            controller.view.backgroundColor = .black

            let cameraView = CameraView(frame: .zero)
            cameraView.translatesAutoresizingMaskIntoConstraints = false
            cameraView.scanRegionMaskVisible = false
            cameraView.torchButtonVisible = false
            controller.view.addSubview(cameraView)

            NSLayoutConstraint.activate([
                cameraView.leadingAnchor.constraint(equalTo: controller.view.leadingAnchor),
                cameraView.trailingAnchor.constraint(equalTo: controller.view.trailingAnchor),
                cameraView.topAnchor.constraint(equalTo: controller.view.topAnchor),
                cameraView.bottomAnchor.constraint(equalTo: controller.view.bottomAnchor)
            ])

            cameraEnhancer = CameraEnhancer(view: cameraView)

            if let cameraEnhancer {
                cameraEnhancer.setResolution(.resolution1080P)
                try? router.setInput(cameraEnhancer)
            }

            configureTemplate()
            router.addResultReceiver(self)
            start()

            return controller
        }

        func update(settings: AutoCaptureSettings, manualCaptureToken: Int) {
            stabilizer.settings = settings
            if lastManualCaptureToken != manualCaptureToken {
                lastManualCaptureToken = manualCaptureToken
                requestManualCapture()
            }
        }

        func start() {
            cameraEnhancer?.open()
            router.startCapturing(detectAndNormalizeTemplateName) { [weak self] isSuccess, error in
                guard let self, !isSuccess else { return }
                Task { @MainActor in
                    self.store.errorMessage = error?.localizedDescription ?? "Failed to start capturing."
                }
            }
        }

        func stop() {
            fallbackWorkItem?.cancel()
            fallbackWorkItem = nil
            router.stopCapturing()
            router.removeAllResultReceivers()
            cameraEnhancer?.close()
        }

        private func configureTemplate() {
            guard let settings = try? router.getSimplifiedSettings(detectAndNormalizeTemplateName) else {
                return
            }

            settings.outputOriginalImage = true
            settings.minImageCaptureInterval = 0
            settings.timeout = 3000
            settings.maxParallelTasks = 1
            settings.documentSettings?.expectedDocumentsCount = 1
            _ = try? router.updateSettings(detectAndNormalizeTemplateName, settings: settings)
        }

        private func requestManualCapture() {
            guard !cooldown else { return }

            fallbackWorkItem?.cancel()
            latestCandidate = nil
            awaitingManualCapture = true

            let workItem = DispatchWorkItem { [weak self] in
                self?.captureFallbackFrameIfNeeded()
            }
            fallbackWorkItem = workItem
            DispatchQueue.main.asyncAfter(deadline: .now() + 0.5, execute: workItem)
        }

        func onProcessedDocumentResultReceived(_ result: ProcessedDocumentResult) {
            guard let item = result.deskewedImageResultItems?.first,
                  let normalizedImageData = item.imageData else {
                return
            }

            let originalImageData = router.getIntermediateResultManager().getOriginalImage(result.originalImageHashId)
            let candidate = CaptureCandidate(
                originalImageData: originalImageData,
                normalizedImageData: normalizedImageData,
                quad: cloneQuadrilateral(item.sourceDeskewQuad),
                crossVerified: item.crossVerificationStatus.rawValue == 1
            )

            latestCandidate = candidate

            if awaitingManualCapture {
                awaitingManualCapture = false
                fallbackWorkItem?.cancel()
                fallbackWorkItem = nil
                commit(candidate, autoCaptured: false)
                return
            }

            guard candidate.crossVerified, let quad = candidate.quad else { return }
            if stabilizer.feed(quad) {
                commit(candidate, autoCaptured: true)
            }
        }
    }
}

Step 3: Review, Adjust, and Export Scans

The store keeps the page list, selected page, color mode, and export payloads in one place. It also handles rotation, retake, reordering, PDF generation, and image export for the final scan batch.

import Foundation
import SwiftUI
import UIKit
import DynamsoftCaptureVisionBundle

@MainActor
final class DocumentScannerStore: ObservableObject {
    @Published var route: ScannerRoute = .scanner
    @Published var pages: [ScannedPage] = []
    @Published var selectedPageIndex: Int = 0
    @Published var retakePageIndex: Int? = nil
    @Published var autoCaptureSettings = AutoCaptureSettings.load() {
        didSet {
            autoCaptureSettings.persist()
        }
    }
    @Published var autoCaptureFlashVisible = false
    @Published var showSettings = false
    @Published var editorTarget: EditorTarget? = nil
    @Published var sharePayload: SharePayload? = nil
    @Published var errorMessage: String? = nil

    func integrateCapturedPage(_ page: ScannedPage, autoCaptured: Bool) {
        if let retakePageIndex, pages.indices.contains(retakePageIndex) {
            pages[retakePageIndex] = page
            selectedPageIndex = retakePageIndex
            self.retakePageIndex = nil
            route = .results
        } else {
            pages.append(page)
            selectedPageIndex = max(0, pages.count - 1)
        }

        if autoCaptured {
            autoCaptureFlashVisible = true
            Task { @MainActor in
                try? await Task.sleep(nanoseconds: 1_200_000_000)
                self.autoCaptureFlashVisible = false
            }
        }
    }

    func rotateSelectedPage() {
        guard pages.indices.contains(selectedPageIndex) else { return }
        pages[selectedPageIndex].rotationQuarterTurns = (pages[selectedPageIndex].rotationQuarterTurns + 1) % 4
    }

    func exportPDF() {
        let images = pages.compactMap { $0.renderedImage() }
        guard !images.isEmpty else { return }

        let exportURL = FileManager.default.temporaryDirectory
            .appendingPathComponent("dynamsoft-scan-\(UUID().uuidString)")
            .appendingPathExtension("pdf")

        let firstBounds = CGRect(origin: .zero, size: images[0].size)
        let renderer = UIGraphicsPDFRenderer(bounds: firstBounds)

        do {
            try renderer.writePDF(to: exportURL) { context in
                for image in images {
                    let bounds = CGRect(origin: .zero, size: image.size)
                    context.beginPage(withBounds: bounds, pageInfo: [:])
                    image.draw(in: bounds)
                }
            }
            sharePayload = SharePayload(items: [exportURL])
        } catch {
            errorMessage = error.localizedDescription
        }
    }
}

import SwiftUI
import UIKit
import DynamsoftCaptureVisionBundle

struct AutoCaptureSettingsSheet: View {
    @EnvironmentObject private var store: DocumentScannerStore

    var body: some View {
        NavigationStack {
            Form {
                Toggle("Enable auto capture", isOn: $store.autoCaptureSettings.autoCaptureEnabled)

                VStack(alignment: .leading, spacing: 8) {
                    HStack {
                        Text("IoU threshold")
                        Spacer()
                        Text(store.autoCaptureSettings.iouThreshold.formatted(.number.precision(.fractionLength(2))))
                            .foregroundStyle(.secondary)
                    }
                    Slider(value: $store.autoCaptureSettings.iouThreshold, in: 0.5...0.98)
                }

                VStack(alignment: .leading, spacing: 8) {
                    HStack {
                        Text("Area delta threshold")
                        Spacer()
                        Text(store.autoCaptureSettings.areaDeltaThreshold.formatted(.number.precision(.fractionLength(2))))
                            .foregroundStyle(.secondary)
                    }
                    Slider(value: $store.autoCaptureSettings.areaDeltaThreshold, in: 0.02...0.30)
                }

                VStack(alignment: .leading, spacing: 8) {
                    HStack {
                        Text("Stable frame count")
                        Spacer()
                        Text("\(store.autoCaptureSettings.stableFrameCount)")
                            .foregroundStyle(.secondary)
                    }
                    Stepper(value: $store.autoCaptureSettings.stableFrameCount, in: 1...8) {
                        EmptyView()
                    }
                }
            }
            .navigationTitle("Stabilization")
            .toolbar {
                ToolbarItem(placement: .confirmationAction) {
                    Button("Done") {
                        store.showSettings = false
                    }
                }
            }
        }
    }
}

struct QuadEditorSheet: View {
    let page: ScannedPage
    let onApply: (Quadrilateral) -> Void
    let onCancel: () -> Void

    @StateObject private var session = QuadEditorSession()

    var body: some View {
        VStack(spacing: 0) {
            HStack {
                Button("Cancel") {
                    onCancel()
                }

                Spacer()

                Text("Adjust Crop")
                    .font(.system(size: 18, weight: .bold, design: .rounded))

                Spacer()

                Button("Apply") {
                    if let quad = session.currentQuad() {
                        onApply(quad)
                    } else {
                        onCancel()
                    }
                }
                .fontWeight(.semibold)
            }
            .padding(.horizontal, 18)
            .padding(.vertical, 14)
            .background(.thinMaterial)

            if page.originalImageData != nil {
                QuadEditorRepresentable(page: page, session: session)
                    .ignoresSafeArea(edges: .bottom)
            } else {
                Spacer()
                Text("This page does not have an editable source image.")
                Spacer()
            }
        }
    }
}

Step 4: Import Photos and Deskew Existing Images

The app also supports gallery imports. If Capture Vision finds a document in the imported image, the store saves the deskewed image and its quad; otherwise it keeps the original image as a fallback page.

    func importPhotoData(_ data: Data) {
        guard let uiImage = UIImage(data: data)?.normalizedOrientationImage() else {
            errorMessage = "Unable to decode the selected image."
            return
        }

        let originalImageData = try? ImageIO().read(fromMemory: data)
        let capturedResult = importRouter.captureFromImage(uiImage, templateName: detectAndNormalizeTemplateName)

        if let item = capturedResult.processedDocumentResult?.deskewedImageResultItems?.first,
           let normalizedImageData = item.imageData {
            let page = ScannedPage(
                originalImageData: originalImageData,
                normalizedImageData: normalizedImageData,
                quad: cloneQuadrilateral(item.sourceDeskewQuad),
                fallbackImage: nil
            )
            integrateCapturedPage(page, autoCaptured: false)
            return
        }

        let page = ScannedPage(
            originalImageData: nil,
            normalizedImageData: nil,
            quad: nil,
            fallbackImage: uiImage
        )
        integrateCapturedPage(page, autoCaptured: false)
    }

Step 5: Trigger Auto Capture Only After the Document Quad Stabilizes

The scanner does not commit a page on the first detection. Instead, it compares each new quadrilateral against the previous one and only auto-captures when the overlap and area change stay within the configured thresholds for several consecutive frames.

import Foundation
import DynamsoftCaptureVisionBundle

final class QuadStabilizer {
    var settings: AutoCaptureSettings

    private var previousQuad: Quadrilateral?
    private var consecutiveStableFrames = 0

    init(settings: AutoCaptureSettings) {
        self.settings = settings
    }

    func reset() {
        previousQuad = nil
        consecutiveStableFrames = 0
    }

    func feed(_ quad: Quadrilateral) -> Bool {
        guard settings.autoCaptureEnabled else { return false }

        guard let previousQuad else {
            self.previousQuad = cloneQuadrilateral(quad)
            consecutiveStableFrames = 0
            return false
        }

        let iou = quadrilateralIoU(previousQuad, quad)
        let previousArea = quadrilateralArea(previousQuad)
        let currentArea = quadrilateralArea(quad)
        let areaDelta = previousArea > 0 ? abs(currentArea - previousArea) / previousArea : 1

        if iou >= settings.iouThreshold && areaDelta <= settings.areaDeltaThreshold {
            consecutiveStableFrames += 1
            if consecutiveStableFrames >= settings.stableFrameCount {
                reset()
                return true
            }
        } else {
            consecutiveStableFrames = 0
        }

        self.previousQuad = cloneQuadrilateral(quad)
        return false
    }
}

import CoreGraphics
import DynamsoftCaptureVisionBundle

func quadrilateralIoU(_ lhs: Quadrilateral, _ rhs: Quadrilateral) -> Double {
    let a = quadrilateralBounds(lhs)
    let b = quadrilateralBounds(rhs)

    let intersection = a.intersection(b)
    guard !intersection.isNull && intersection.width > 0 && intersection.height > 0 else {
        return 0
    }

    let intersectionArea = intersection.width * intersection.height
    let unionArea = (a.width * a.height) + (b.width * b.height) - intersectionArea
    guard unionArea > 0 else { return 0 }
    return intersectionArea / unionArea
}

func quadrilateralArea(_ quad: Quadrilateral) -> Double {
    let points = quadrilateralPoints(quad)
    guard points.count >= 4 else { return 0 }

    var area: Double = 0
    for index in points.indices {
        let nextIndex = (index + 1) % points.count
        area += Double(points[index].x * points[nextIndex].y)
        area -= Double(points[nextIndex].x * points[index].y)
    }
    return abs(area) / 2
}

Step 6: Build the Capture-to-Review Workflow in SwiftUI

The app keeps capture, review, and page ordering inside one SwiftUI state machine. ScannerScreen handles import, manual capture, and thumbnails, while ResultsScreen exposes export, retake, edit, rotate, and sort actions.

import SwiftUI
import PhotosUI

struct ScannerScreen: View {
    @EnvironmentObject private var store: DocumentScannerStore
    @State private var manualCaptureToken = 0
    @State private var selectedPhotoItem: PhotosPickerItem?

    var body: some View {
        VStack(spacing: 18) {
            ZStack(alignment: .topLeading) {
                CameraScannerView(manualCaptureToken: manualCaptureToken, settings: store.autoCaptureSettings)
                    .frame(maxWidth: .infinity)
                    .aspectRatio(3 / 4, contentMode: .fit)

                if store.autoCaptureFlashVisible {
                    Text("Auto captured")
                }
            }

            HStack(spacing: 18) {
                PhotosPicker(selection: $selectedPhotoItem, matching: .images) {
                    Label("Import", systemImage: "photo.on.rectangle.angled")
                }

                Spacer()

                Button {
                    manualCaptureToken += 1
                } label: {
                    Circle()
                        .fill(Color.white)
                        .frame(width: 86, height: 86)
                }

                Spacer()

                Button {
                    store.openResults()
                } label: {
                    Label("Next", systemImage: "arrow.right")
                }
                .disabled(store.pages.isEmpty)
            }

            VStack(alignment: .leading, spacing: 10) {
                if store.pages.isEmpty || store.retakePageIndex != nil {
                    RoundedRectangle(cornerRadius: 22, style: .continuous)
                        .frame(height: 84)
                } else {
                    ScrollView(.horizontal, showsIndicators: false) {
                        HStack(spacing: 12) {
                            ForEach(Array(store.pages.enumerated()), id: \.element.id) { index, page in
                                ThumbnailCard(page: page, index: index + 1) {
                                    store.openResults(from: index)
                                } onRemove: {
                                    store.removePage(id: page.id)
                                }
                            }
                        }
                    }
                }
            }
        }
        .onChange(of: selectedPhotoItem) { newValue in
            guard let newValue else { return }
            Task {
                if let data = try? await newValue.loadTransferable(type: Data.self) {
                    await MainActor.run {
                        store.importPhotoData(data)
                    }
                } else {
                    await MainActor.run {
                        store.errorMessage = "Unable to load the selected photo."
                    }
                }
                await MainActor.run {
                    selectedPhotoItem = nil
                }
            }
        }
    }
}

struct ResultsScreen: View {
    @EnvironmentObject private var store: DocumentScannerStore

    var body: some View {
        VStack(spacing: 18) {
            HStack(alignment: .top) {
                Spacer()

                Menu {
                    Button("Export PDF") {
                        store.exportPDF()
                    }
                    Button("Export Images") {
                        store.exportImages()
                    }
                } label: {
                    Image(systemName: "square.and.arrow.up")
                }
            }

            if store.pages.isEmpty {
                Text("Capture a page before reviewing results.")
            } else {
                TabView(selection: $store.selectedPageIndex) {
                    ForEach(Array(store.pages.enumerated()), id: \.element.id) { index, page in
                        ZoomablePageView(page: page)
                            .tag(index)
                    }
                }

                HStack(spacing: 12) {
                    ActionChip(title: "Continue", systemImage: "plus.viewfinder") {
                        store.continueScanning()
                    }
                    ActionChip(title: "Retake", systemImage: "camera.rotate") {
                        store.startRetake()
                    }
                    ActionChip(title: "Edit", systemImage: "crop") {
                        store.presentEditor()
                    }
                    .disabled(!(store.currentPage?.canEdit ?? false))
                    ActionChip(title: "Rotate", systemImage: "rotate.right") {
                        store.rotateSelectedPage()
                    }
                    ActionChip(title: "Sort", systemImage: "square.grid.2x2") {
                        store.route = .sort
                    }
                }
            }
        }
    }
}

Step 7: Render Final Pages in Color, Grayscale, or Binary

Each page keeps either a normalized document image from Capture Vision or a fallback UIImage. The rendering pipeline converts grayscale and binary variants on demand, then rotates the preview or export output without mutating the original source.

struct ScannedPage: Identifiable {
    let id = UUID()
    var originalImageData: ImageData?
    var normalizedImageData: ImageData?
    var quad: Quadrilateral?
    var fallbackImage: UIImage?
    var colorMode: DocumentColorMode = .color
    var rotationQuarterTurns: Int = 0

    var canEdit: Bool {
        originalImageData != nil && quad != nil
    }

    func renderedImage(processor: ImageProcessor = ImageProcessor()) -> UIImage? {
        let rendered: UIImage?

        if let normalizedImageData {
            var workingImage = normalizedImageData
            switch colorMode {
            case .color:
                break
            case .grayscale:
                workingImage = processor.convert(toGray: workingImage)
            case .binary:
                let grayscaleImage = processor.convert(toGray: workingImage)
                workingImage = processor.convert(toBinaryLocal: grayscaleImage)
            }
            rendered = try? workingImage.toUIImage()
        } else if let fallbackImage {
            rendered = fallbackImage.processed(for: colorMode)
        } else {
            rendered = nil
        }

        return rendered?.rotated(quarterTurns: rotationQuarterTurns)
    }
}

import UIKit

extension UIImage {
    func rotated(quarterTurns: Int) -> UIImage {
        let normalizedQuarterTurns = ((quarterTurns % 4) + 4) % 4
        guard normalizedQuarterTurns != 0 else { return self }

        let angle = CGFloat(normalizedQuarterTurns) * (.pi / 2)
        let rotatedSize = normalizedQuarterTurns.isMultiple(of: 2) ? size : CGSize(width: size.height, height: size.width)

        let renderer = UIGraphicsImageRenderer(size: rotatedSize)
        return renderer.image { context in
            context.cgContext.translateBy(x: rotatedSize.width / 2, y: rotatedSize.height / 2)
            context.cgContext.rotate(by: angle)
            draw(in: CGRect(x: -size.width / 2, y: -size.height / 2, width: size.width, height: size.height))
        }
    }

    func processed(for mode: DocumentColorMode) -> UIImage {
        switch mode {
        case .color:
            return self
        case .grayscale:
            return pixelProcessed(binary: false)
        case .binary:
            return pixelProcessed(binary: true)
        }
    }
}

Source Code

https://github.com/yushulx/ios-swiftui-barcode-mrz-document-scanner/tree/main/examples/DynamsoftDocumentScanner

Build an Android Passport Scanner with MRZ and Portrait Detection

Xiao Ling — Wed, 13 May 2026 06:06:38 +0000

Identity verification workflows — from airport check-in to hotel registration — still rely heavily on manual data entry from physical travel documents. Automating MRZ (Machine-Readable Zone) scanning eliminates typos, speeds up onboarding, and keeps sensitive data on-device. The Dynamsoft Capture Vision SDK bundles document detection, text recognition, code parsing, and identity processing into a single pipeline, making it practical to ship a production-grade scanner in an Android app.

What you'll build: A real-time Android MRZ scanner app that captures live camera frames, parses travel document fields, and validates portrait zones with the Dynamsoft Capture Vision SDK.

Watch the MRZ Scanner Demo

Prerequisites

Android Studio Hedgehog (2023.1.1) or newer
A device or emulator running Android 5.0 (API 21) or later
The Dynamsoft MRZ Scanner Bundle (com.dynamsoft:mrzscannerbundle:3.4.1200)

Get a 30-day free trial license

Step 1: Set Up the Android Project with the Dynamsoft SDK

Add the Dynamsoft Maven repository to your top-level build.gradle:

// build.gradle (project level)
allprojects {
    repositories {
        maven { url "https://download2.dynamsoft.com/maven/aar" }
        google()
        mavenCentral()
    }
}

Declare the MRZ Scanner Bundle dependency in app/build.gradle. The bundle pulls in Camera Enhancer, Capture Vision Router, Document Normalizer, Label Recognizer, Code Parser, and Identity Processor — no need to add each module individually:

// app/build.gradle
plugins {
    id 'com.android.application'
}

android {
    namespace 'com.test.mrzscanner'
    compileSdk 34

    defaultConfig {
        applicationId "com.test.mrzscanner"
        minSdk 21
        targetSdk 34
        versionCode 2
        versionName "2.0"
    }

    buildFeatures {
        viewBinding true
    }

    packagingOptions {
        doNotStrip "**/*.so"
    }

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

dependencies {
    implementation 'com.dynamsoft:mrzscannerbundle:3.4.1200'
    implementation 'com.google.android.material:material:1.11.0'
    implementation 'androidx.appcompat:appcompat:1.6.1'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
    implementation 'com.github.bumptech.glide:glide:4.16.0'
}

The packagingOptions.doNotStrip block is important — the Dynamsoft SDK ships native .so libraries, and stripping them would cause a runtime crash.

Step 2: Initialize the Dynamsoft Capture Vision License

Before calling any SDK API, initialize your license. In this project the license key lives in MrzParser.java:

public class MrzParser {
    private static final String LICENSE_KEY = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

    public static void initLicense() {
        com.dynamsoft.license.LicenseManager.initLicense(LICENSE_KEY, (isSuccess, error) -> {
            if (!isSuccess) {
                android.util.Log.e("MrzParser", "License initialization failed: " + error.getMessage());
            }
        });
    }
}

Call MrzParser.initLicense() early in MainActivity.onCreate() before constructing any SDK components.

Replace the placeholder key with your own trial license from the Dynamsoft Customer Portal.

Step 3: Set Up the Camera Stream with Dynamsoft Camera Enhancer

MainActivity uses CameraEnhancer backed by a CameraView declared in XML:

<com.dynamsoft.dce.CameraView
    android:id="@+id/dce_camera_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"/>

In code, request the camera permission, create the CameraEnhancer, and set it as the input source for the CaptureVisionRouter:

public class MainActivity extends AppCompatActivity {
    private CameraEnhancer mCamera;
    private CameraView mCameraView;
    private final CaptureVisionRouter mRouter = new CaptureVisionRouter();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_scan);

        mCameraView = findViewById(R.id.dce_camera_view);
        PermissionUtil.requestCameraPermission(this);

        mCamera = new CameraEnhancer(mCameraView, this);
        mCamera.setColourChannelUsageType(EnumColourChannelUsageType.CCUT_FULL_CHANNEL);
        try {
            mCamera.enableEnhancedFeatures(EnumEnhancerFeatures.EF_FRAME_FILTER);
        } catch (CameraEnhancerException ignore) {}

        // ...router setup goes here
        mRouter.setInput(mCamera);
    }

    @Override
    protected void onResume() {
        super.onResume();
        mCamera.open();
        mRouter.startCapturing("ReadPassportAndId", new CompletionListener() {
            @Override
            public void onSuccess() { /* ready */ }
            @Override
            public void onFailure(int code, String msg) { /* error */ }
        });
    }

    @Override
    protected void onPause() {
        super.onPause();
        mCamera.close();
        mRouter.stopCapturing();
    }
}

The camera lifecycle is tied to onResume/onPause — open it when the activity gains focus and close it when focus is lost.

Step 4: Configure the MRZ Recognition Pipeline

The CaptureVisionRouter orchestrates the detection→recognition→parsing pipeline. Configuration is driven by a JSON template file bundled with the SDK:

MultiFrameResultCrossFilter filter = new MultiFrameResultCrossFilter();
filter.enableResultCrossVerification(
    EnumCapturedResultItemType.CRIT_TEXT_LINE
    | EnumCapturedResultItemType.CRIT_DESKEWED_IMAGE
    | EnumCapturedResultItemType.CRIT_DETECTED_QUAD,
    true
);
filter.setResultCrossVerificationCriteria(
    EnumCapturedResultItemType.CRIT_DESKEWED_IMAGE
    | EnumCapturedResultItemType.CRIT_DETECTED_QUAD,
    new CrossVerificationCriteria(5, 2)
);
mRouter.addResultFilter(filter);

mRouter.initSettingsFromFile("mrzscanner-mobile-templates.json");
mRouter.setInput(mCamera);

SimplifiedCaptureVisionSettings st = mRouter.getSimplifiedSettings("ReadPassportAndId");
if (st.documentSettings != null) {
    st.documentSettings.minQuadrilateralAreaRatio = 2;
}
mRouter.updateSettings("ReadPassportAndId", st);

The MultiFrameResultCrossFilter requires the same result to be confirmed across multiple frames before it's emitted, suppressing flickering false positives. The template name "ReadPassportAndId" references a pre-built pipeline inside the JSON template that targets travel documents.

Step 5: Parse MRZ Output and Cache Results

Two receivers listen for output. The IntermediateResultReceiver caches intermediate units (scaled color image, localized text lines, recognized text, detected quads, deskewed image) that the IdentityProcessor needs later. The CapturedResultReceiver handles the parsed MRZ result:

private final CapturedResultReceiver mResultReceiver = new CapturedResultReceiver() {
    @Override
    public void onCapturedResultReceived(@NonNull CapturedResult result) {
        ParsedResult pr = result.getParsedResult();
        if (pr == null || pr.getItems().length == 0) return;

        HashMap<String, String> map = (HashMap<String, String>)
            MrzParser.parse(pr.getItems()[0]);
        if (map.isEmpty()) return;

        // ... portrait detection logic ...

        mPendingLabelMap = map;
        runOnUiThread(() -> {
            btnCapture.setEnabled(true);
            tvStatus.setText("Ready");
        });
    }
};

The MrzParser.parse() helper uses the SDK's built-in ParsedResultItem.getParsedFields() to extract structured key-value pairs — no regex needed for the primary path:

public static Map<String, String> parse(com.dynamsoft.dcp.ParsedResultItem item) {
    HashMap<String, String> entry = item.getParsedFields();
    HashMap<String, String> properties = new HashMap<>();

    String number = getFirstNonNull(entry, "passportNumber", "documentNumber", "idNumber");
    String firstName = getFirstNonNull(entry, "secondaryIdentifier", "givenNames");
    String lastName = getFirstNonNull(entry, "primaryIdentifier", "lastName");
    String nationality = entry.get("nationality") != null ? entry.get("nationality") : "Unknown";
    String issuingState = entry.get("issuingState") != null ? entry.get("issuingState") : "Unknown";
    String sex = entry.get("sex") != null ? entry.get("sex") : "Unknown";

    String birthDate = formatDate(
        entry.get("birthYear"), entry.get("birthMonth"), entry.get("birthDay"));
    String expiryDate = formatDate(
        entry.get("expiryYear"), entry.get("expiryMonth"), entry.get("expiryDay"));

    properties.put("Document Type", docType);
    properties.put("Name", fullName);
    properties.put("Sex", formatSex(sex));
    properties.put("Age", age >= 0 ? String.valueOf(age) : "—");
    properties.put("Document Number", number.isEmpty() ? "—" : number);
    properties.put("Issuing State", issuingState);
    properties.put("Nationality", nationality);
    properties.put("Date of Birth(YYYY-MM-DD)", birthDate.isEmpty() ? "—" : birthDate);
    properties.put("Date of Expiry(YYYY-MM-DD)", expiryDate.isEmpty() ? "—" : expiryDate);

    return properties;
}

Step 6: Validate the Portrait Zone and Show the Scan Results

The IdentityProcessor can detect a portrait zone from the intermediate results. Before using it, the code checks for a high-confidence auxiliary region tagged "PortraitZone" and validates the quadrilateral against document bounds:

// Check auxiliary regions for PortraitZone with confidence > 60
int highConfIdx = -1;
if (mLocalizedTextLinesUnit.getAuxiliaryRegionElementsCount() > 0) {
    for (int i = 0; i < mLocalizedTextLinesUnit.getAuxiliaryRegionElementsCount(); i++) {
        if ("PortraitZone".equals(mLocalizedTextLinesUnit.getAuxiliaryRegionElement(i).getName())) {
            if (mLocalizedTextLinesUnit.getAuxiliaryRegionElement(i).getConfidence() > 60) {
                highConfIdx = i;
                break;
            }
        }
    }
}

// Find the portrait zone
Quadrilateral pz = null;
if (highConfIdx != -1 && mDetectedQuadsUnit != null && mDetectedQuadsUnit.getCount() > 0) {
    pz = mIdProcessor.findPortraitZone(
        mScaledColourImageUnit, mLocalizedTextLinesUnit,
        mRecognizedTextLinesUnit, mDetectedQuadsUnit, mDeskewedImageUnit);
}

// Validate portrait is inside document and proportionally reasonable
if (pz != null && quadItem != null) {
    Quadrilateral docRegion = quadItem.getLocation();
    boolean valid = docRegion.isPointInQuadrilateral(pz.points[0])
        && docRegion.isPointInQuadrilateral(pz.points[1])
        && docRegion.isPointInQuadrilateral(pz.points[2])
        && docRegion.isPointInQuadrilateral(pz.points[3])
        && docRegion.getArea() / pz.getArea() >= 3;
    if (!valid) pz = null;
}

Once validated, the portrait region is deskewed (warped to a rectangle) and passed to ScanResultActivity as a JPEG byte array alongside the parsed field map.

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/IdScanner

How to Extract Structured Data and Photos from Passports in JavaScript with Dynamsoft Capture Vision

Xiao Ling — Mon, 11 May 2026 02:36:58 +0000

Passport onboarding, hotel check-in, and KYC flows usually need more than plain OCR. You need MRZ parsing, document localization, and a reliable portrait crop from the same capture pipeline, ideally in the browser without shipping data to a server. This sample uses Dynamsoft Capture Vision on the web to read passport and ID card MRZ data, draw document boundaries, and extract the portrait photo from uploaded images or a captured camera frame.

What you'll build: A plain JavaScript passport and ID scanner that uses Dynamsoft Capture Vision 3.4.2001 to extract MRZ text, structured fields, document borders, and portrait photos from uploaded images or a frozen camera frame.

Live demo

https://yushulx.me/javascript-barcode-qr-code-scanner/examples/mrz_scanner/

Watch the JavaScript Passport Scanner Demo

Prerequisites

A modern browser with camera and clipboard support
A local web server such as Python's built-in HTTP server
Dynamsoft Capture Vision Bundle 3.4.2001
A valid Dynamsoft license key for MRZ features

Get a 30-day free trial license

Step 1: Load the Dynamsoft Bundle and Build the Page Shell

The page uses a plain HTML layout with one image view, one camera view, an overlay canvas, and a face crop canvas.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI-Powered MRZ & Passport Scanner</title>
    <link rel="stylesheet" href="style.css">
    <!-- Dynamsoft Capture Vision Bundle for MRZ -->
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-bundle@3.4.2001/dist/dcv.bundle.min.js"></script>
    <!-- Face/Border detection powered by Dynamsoft IdentityProcessor (ReadPassportAndId template) -->
</head>

<body>
    <div class="app-container">
        <!-- Header / License Config -->
        <header>
            <h1>🛂 MRZ & Passport Scanner</h1>
            <div class="license-bar">
                <input type="text" id="licenseKey"
                    placeholder="LICENSE-KEY">
                <button id="initBtn">Initialize MRZ</button>
                <a href="https://www.dynamsoft.com/customer/license/trialLicense/?product=dcv&package=cross-platform"
                    target="_blank" rel="noopener noreferrer" class="trial-link">Get MRZ License</a>
            </div>
        </header>


        <!-- Main Content -->
        <main>
            <!-- Left Panel: Image Source & Display -->
            <section class="panel left-panel">
                <div class="toolbar">
                    <button id="btnLoad" title="Load Image" disabled>📂 Load</button>
                    <button id="btnCamera" title="Start Camera" disabled>📷 Camera</button>
                    <button id="btnPaste" title="Paste from Clipboard" disabled>📋 Paste</button>
                    <span id="status">Ready</span>
                </div>

                <div class="viewer-container" id="dropZone">
                    <div id="cameraView" class="hidden">
                        <canvas id="cameraOverlay"></canvas>
                    </div>
                    <div id="imageView" class="hidden">
                        <img id="displayImage" src="" alt="Input">
                        <canvas id="overlayCanvas"></canvas>
                    </div>
                    <div class="placeholder-text" id="placeholderText">
                        Drag & Drop Image Here<br>or Select Source
                    </div>
                </div>
            </section>

Step 2: Initialize Capture Vision and Load the MRZ Template

The initialization step loads the license, WASM modules, MRZ specifications, and the custom findPrecisePortraitZone.json template used by the ReadPassportAndId workflow.

els.initBtn.addEventListener('click', async () => {
    let key = els.licenseKey.value.trim();
    if (!key) {
        key = "LICENSE-KEY";
    }

    try {
        els.status.textContent = "Initializing MRZ SDK...";
        els.initBtn.disabled = true;

        // Initialize License
        await Dynamsoft.License.LicenseManager.initLicense(key, true);

        // Load WASM modules
        els.status.textContent = "Loading WASM modules...";
        await Dynamsoft.Core.CoreModule.loadWasm(["DLR", "DDN"]);

        // Create Code Parser for MRZ parsing
        parser = await Dynamsoft.DCP.CodeParser.createInstance();

        // Load MRZ specs
        els.status.textContent = "Loading MRZ specs...";
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD1_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_VISA");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_PASSPORT");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_VISA");

        // Preload the custom MRZ recognition models referenced by the sample settings file.
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer([
            "MRZCharRecognition",
            "MRZTextLineRecognition"
        ]);

        // Create Capture Vision Router
        cvr = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();

        // Load MRZ template
        const settingsResult = await cvr.initSettings("./findPrecisePortraitZone.json?t=" + new Date().getTime());
        if (settingsResult?.errorCode && settingsResult.errorCode !== 0) {
            throw new Error(`Template load failed: ${settingsResult.errorString || settingsResult.errorCode}`);
        }

        const hasReadPassportAndId = await cvr.checkTemplateNameValidity("ReadPassportAndId");
        if (!hasReadPassportAndId) {
            const templateNames = await cvr.getTemplateNames();
            throw new Error(
                `Template ReadPassportAndId is not available after loading findPrecisePortraitZone.json. Available templates: ${templateNames.join(", ")}`
            );
        }

        isMRZReady = true;

        if (typeof faceProcessor !== 'undefined') {
            await faceProcessor.setCVR(cvr);
        }

The JSON template is where the sample ties MRZ reading to document detection and normalization.

{
  "CaptureVisionTemplates": [
    {
      "Name": "ReadPassportAndId",
      "ImageROIProcessingNameArray": [
        "roi_passport_and_id"
      ],
      "SemanticProcessingNameArray": [
        "sp_passport_and_id"
      ],
      "OutputOriginalImage": 1,
      "MaxParallelTasks": 0,
      "Timeout": 20000
    }
  ],
  "TargetROIDefOptions": [
    {
      "Name": "roi_passport_and_id",
      "TaskSettingNameArray": [
        "task_passport_and_id",
        "task-detect-and-normalize-document"
      ]
    }
  ]
}

Step 3: Process Uploaded Images and Parse MRZ Text

For still images, the app loads the image into the UI first, clears the previous intermediate results, runs cvr.capture(), and then parses the concatenated MRZ text.

function loadImage(base64Image, captureSource = null) {
    currentImageCaptureSource = captureSource ?? base64Image;
    els.displayImage.src = base64Image;
    els.placeholderText.classList.add('hidden');
    els.imageView.classList.remove('hidden');
    els.cameraView.classList.add('hidden');

    els.displayImage.onload = async () => {
        resizeCanvas();
        els.status.textContent = "Processing...";
        clearResults();

        // Show spinner
        els.loadingSpinner.classList.add('visible');

        const ctx = els.overlayCanvas.getContext('2d', { willReadFrequently: true });
        ctx.clearRect(0, 0, els.overlayCanvas.width, els.overlayCanvas.height);

        try {
            // Run MRZ detection (only if license is initialized)
            if (isMRZReady && cvr) {
                try {
                    // Clear previous intermediate results so IdentityProcessor
                    // only sees data from this capture call
                    if (typeof faceProcessor !== 'undefined') {
                        faceProcessor.clearIntermediateResults();
                    }

                    // Use ReadPassportAndId template: one capture returns MRZ text,
                    // document border (DetectedQuad), and intermediate units for
                    // IdentityProcessor.findPortraitZone()
                    const captureSource = currentImageCaptureSource ?? els.displayImage;
                    const result = await cvr.capture(captureSource, "ReadPassportAndId");
                    const items = result.items || [];

                    let mrzTexts = [];
                    for (const item of items) {
                        if (item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_TEXT_LINE) {
                            mrzTexts.push(item.text);

                            // Draw overlay and capture MRZ zone
                            const location = item.location;
                            if (location && location.points) {
                                drawOverlay(location);
                            }
                        }
                    }

                    if (mrzTexts.length > 0) {
                        // Display raw MRZ text with newlines for readability
                        els.mrzRawText.textContent = mrzTexts.join('\n');

                        // Parse MRZ - join all lines into single string (no separators)
                        // TD3 passport has 2 lines of 44 chars each = 88 chars total
                        const mrzForParsing = mrzTexts.map(t => t.trim()).join('');

                        const parseResults = await parser.parse(mrzForParsing);
                        displayParsedMrz(parseResults);
                    } else {
                        els.mrzRawText.textContent = "No MRZ detected.";
                        els.mrzResults.textContent = "No MRZ detected.";
                    }

                    // Process face/border from the same captured result
                    if (typeof faceProcessor !== 'undefined') {
                        await faceProcessor.processCapturedResult(
                            result, els.displayImage, els.faceCropCanvas, els.overlayCanvas
                        );
                    }

Step 4: Draw Document Borders and Portrait Crops from One Capture Result

The helper class uses the same CapturedResult to locate the document quadrilateral, call IdentityProcessor.findPortraitZone(), and draw the cropped portrait.

async processCapturedResult(capturedResult, imageElement, faceCanvas, canvasOverlay) {
    if (!this.isInitialized) {
        this._clearFaceCanvas(faceCanvas);
        return;
    }

    let detectedQuad = null;
    let portraitZone = null;

    try {
        // 1. Extract detected document border from captured result items
        if (capturedResult && capturedResult.items) {
            for (const item of capturedResult.items) {
                if (item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_DETECTED_QUAD) {
                    detectedQuad = item;
                    break;
                }
            }
        }

        // 2. Find portrait zone using the same unit group assembly as the Python reference.
        portraitZone = await this.findPortraitZoneForCapturedResult(capturedResult);

        // 3. Draw document border
        if (detectedQuad) {
            const { width: srcW, height: srcH } = this._getSourceDimensions(imageElement);
            this.drawDocumentBorder(detectedQuad, canvasOverlay, srcW, srcH);
        }

        // 4. Draw portrait zone and crop
        if (portraitZone) {
            this.drawAndCropPortrait(portraitZone, imageElement, faceCanvas, canvasOverlay);
        } else {
            this._clearFaceCanvas(faceCanvas);
        }

    } catch (e) {
        console.error('FaceProcessor error:', e);
        this._clearFaceCanvas(faceCanvas);
    }
}

drawAndCropPortrait(portraitZone, sourceImage, faceCanvas, canvasOverlay) {
    const pts = portraitZone.points;
    if (!pts || pts.length < 4) return;

    const { width: imgW, height: imgH } = this._getSourceDimensions(sourceImage);

    const clamped = this._clampPoints(pts, imgW, imgH);

    this._drawQuad(clamped, canvasOverlay, 'Portrait (Identity)', false, '#0070f3');
    if (!this._drawPortraitQuadCrop(sourceImage, clamped, faceCanvas)) {
        this._clearFaceCanvas(faceCanvas);
        return;
    }

Step 5: Freeze the Camera Frame Before Scanning

Instead of scanning every live frame, this sample freezes the current camera image first, shows it immediately, and then runs MRZ and portrait detection on that frozen canvas.

async function captureAndFreezeCamera() {
    const video = els.cameraView.querySelector('video');
    if (!video || video.readyState < 2 || !video.videoWidth || !video.videoHeight) {
        els.status.textContent = "Camera frame is not ready yet.";
        return;
    }

    const frameCanvas = document.createElement('canvas');
    frameCanvas.width = video.videoWidth;
    frameCanvas.height = video.videoHeight;
    const frameCtx = frameCanvas.getContext('2d', { willReadFrequently: true });
    frameCtx.drawImage(video, 0, 0, frameCanvas.width, frameCanvas.height);
    const frozenFrameDataUrl = frameCanvas.toDataURL('image/jpeg', 0.95);

    isCameraRunning = false;

    if (els.cameraOverlay) {
        const ctx = els.cameraOverlay.getContext('2d', { willReadFrequently: true });
        ctx.clearRect(0, 0, els.cameraOverlay.width, els.cameraOverlay.height);
    }

    if (videoStream) {
        videoStream.getTracks().forEach(track => track.stop());
        videoStream = null;
    }
    video.srcObject = null;

    document.getElementById('btnCamera').textContent = "📷 Camera";
    els.cameraView.classList.add('hidden');
    els.imageView.classList.remove('hidden');
    els.status.textContent = "Processing captured frame...";
    clearResults();
    els.loadingSpinner.classList.add('visible');

    lastCameraResult = {
        base64Image: frozenFrameDataUrl,
        mrzTexts: [],
        mrzLocations: [],
        detectedQuad: null,
        portraitZone: null
    };

    try {
        await showCapturedCameraFrame(frozenFrameDataUrl);

        if (isMRZReady && cvr) {
            try {
                if (typeof faceProcessor !== 'undefined') {
                    faceProcessor.clearIntermediateResults();
                }

                const result = await cvr.capture(frameCanvas, "ReadPassportAndId");

Source Code

https://github.com/yushulx/javascript-barcode-qr-code-scanner/tree/main/examples/mrz_scanner

How to Generate Custom Dynamsoft Barcode Reader Templates in Python for Hard Barcode Images

Xiao Ling — Wed, 06 May 2026 09:13:49 +0000

Default barcode presets are designed for broad coverage, not for every extreme image you will see in production. When a symbol is tiny, low-contrast, tightly cropped, or surrounded by distracting texture, the right fix is usually not to keep randomizing parameters. A better approach is to let AI follow a strict workflow: inspect the image, validate the current template, probe controlled variants, classify the failure mode, and then generate the smallest focused JSON template that matches the evidence.

What you'll build: A Python workflow that uses AI, OpenCV, and Dynamsoft Barcode Reader helper scripts to diagnose hard barcode images, generate focused JSON templates, and validate them before import.

Follow the Workflow from Diagnosis to Focused Template Generation

Why hard barcode images often need custom DBR templates instead of more blind retries.
How to turn barcode-tuning knowledge into an AI skill and prompt that produce consistent results.
How AI visual analysis and OpenCV-based validation work together in the same tuning loop.
How to validate and probe DBR templates with small Python helpers before changing JSON.
How to generate a focused single-barcode DataMatrix template for a real tiny-symbol image.

Why Focused DBR Templates Matter for Hard Barcode Images

Dynamsoft Barcode Reader is flexible because it lets you tune scope, localization, decode, and acceptance behavior independently. That flexibility is exactly why custom templates matter. A hard image might fail because the scene contains too much background, because localization is too conservative, because the decoder needs a different grayscale or binarization path, or because the acceptance rules reject a symbol that is otherwise readable.

The problem is not capability. The problem is undirected tuning. Without a disciplined workflow, it is easy to change DeblurModes, LocalizationModes, and MinQuietZoneWidth all at once and learn nothing from the result. In this project, AI is useful because it does not just generate JSON. It follows a falsifiable decision tree encoded in the skill and prompt. It also knows when to stop. For example, the same repo contains an Aztec case where raw plus basic variants still prove that missing context pixels are the real blocker, which means preprocessing is more justified than more template churn.

Prepare the Inputs for Repeatable Template Tuning

Use the default DBR preset as a baseline rather than a guarantee for extreme images.
Let AI classify whether scope, localization, decode, or acceptance settings deserve attention first.
Treat raw plus basic-variant failure as a stop signal for template-only tuning.
Keep focused JSON templates small so they are easier to validate and import into the desktop app.

Prerequisites

Python 3.9+
Get a 30-day free trial license for Dynamsoft Barcode Reader.
The dependencies in template-optimizer/requirements.txt if you only want the optimizer package, or requirements.txt if you also want the full GUI app
A difficult barcode image
A baseline DBR template file if you already have one

Install the standalone optimizer dependencies:

pip install -r template-optimizer/requirements.txt

Step 1: Encode the Tuning Rules into a Reusable AI Workflow

The first step is to stop asking AI for ad hoc parameter guesses. In this project, the canonical workflow now lives inside the self-contained template-optimizer/ directory: template-optimizer/SKILL.md captures the decision logic, template-optimizer/PROMPT.md provides a portable user-facing entrypoint, and template-optimizer/KNOWLEDGE.md stores the tuning reference. The .github/ files are still useful in VS Code Copilot, but only as thin wrappers around that portable package.

This is the copyable prompt contract from template-optimizer/PROMPT.md:

Read `template-optimizer/SKILL.md` and `template-optimizer/KNOWLEDGE.md` first. Follow the Python-only template-optimizer workflow and use the bundled helper scripts under `template-optimizer/tools/` for single-image triage before broad benchmarking.

Task: tune this barcode image.

- Image path: `<image-path>`
- Current template path: `<template-path-or-none>`
- Format hint: `<format-or-none>`

And this discipline now lives in the canonical template-optimizer/SKILL.md workflow, with .github/skills/template-optimizer/SKILL.md acting only as an optional Copilot wrapper:

## Procedure

1. Observe the image before touching the template.
Use the image tool on the provided barcode image and describe visible symptoms in concrete terms: quiet-zone loss, border clipping, blur, low contrast, warping, wrinkles, inversion, reflection, background texture, finder-pattern damage, or perspective skew.

2. Validate the current template directly through the simplest DBR path.
The validator must initialize the DBR license before creating `CaptureVisionRouter`; otherwise treat `NO_RESULT` as invalid evidence.
Run `python template-optimizer/tools/validate_dbr_template.py <image> --template-file <template>`.

3. Establish a broader baseline only after the direct validation result is known.
Run `python template-optimizer/tools/probe_dbr_templates.py <image>`.

That structure matters because AI now has a repeatable way to reason about DBR flexibility. It must classify the problem before it edits JSON. That is what turns AI from a template randomizer into a tuning assistant.

Step 2: Run the Portable Skill and Prompt in Practice

This part is critical. If an article shows the generated skill and prompt but never explains how to use them, many readers still will not know how to make the workflow run in a real tool.

The simplest packaging rule is this: treat template-optimizer/ as the product, and treat .github/ as optional integration glue for VS Code Copilot. In practice, a developer can copy just the template-optimizer/ directory into another workspace, install template-optimizer/requirements.txt, provide a barcode image plus any current template path, and ask the agent to read template-optimizer/PROMPT.md. That is enough to run the same workflow in Copilot, Claude, or another coding agent that can read files and execute local Python commands.

Inside VS Code Copilot, the flow becomes:

Open the workspace that contains template-optimizer/.
Optionally keep .github/prompts/ and .github/skills/ if you want native Copilot discovery.
Provide the barcode image, the current template path, and any known format constraints in Copilot Chat.
Use template-optimizer/PROMPT.md directly, or ask Copilot to use the template-optimizer skill if the optional wrapper is present.
Let the agent call the bundled Python scripts under template-optimizer/tools/, such as validate_dbr_template.py, probe_dbr_templates.py, and compare_dbr_template_profiles.py, to turn the visual diagnosis into evidence.

In other words, the portable prompt decides where the workflow starts, the canonical skill decides how it should reason, and the local scripts decide how the conclusions get validated. The .github/ files just make that package easier for Copilot to discover.

What about other tools? They can use the same method, but you need to separate the workflow itself from the file format that stores it.

Tools such as OpenCode and Claude Code can usually reuse the same methodology, but they may not natively understand GitHub Copilot-style files such as .github/prompts/*.prompt.md and .github/skills/*/SKILL.md. That is exactly why the prompt was moved into template-optimizer/PROMPT.md: the portable package no longer depends on .github/ to be usable. In those tools, you usually just tell the agent to read template-optimizer/PROMPT.md, template-optimizer/SKILL.md, and template-optimizer/KNOWLEDGE.md.

That does not mean the logic has to be rewritten. The portable parts are these three:

the workflow itself: inspect the image, validate, probe, classify, and then generate a focused template
the local script capability: template-optimizer/tools/validate_dbr_template.py, template-optimizer/tools/probe_dbr_templates.py, and JSON template editing
the image input capability: the AI must be able to actually inspect the image, not just read a text description of it

So if you want to use this approach in OpenCode or Claude Code, at least three conditions should be true:

The tool supports image input or multimodal visual analysis.
The tool can access the workspace files and run local terminal commands.
You are willing to translate the VS Code-style prompt and skill content into whatever instruction format that tool expects.

Once those conditions are met, the approach is not limited to VS Code. GPT-5.4 can be used, and other multimodal AI systems with visual capability can also be used. VS Code Copilot Chat can consume this file structure more directly, while OpenCode and Claude Code are better treated as environments where the same rules are adapted and reused. What needs to travel is not one specific file format, but the full loop of visual diagnosis, script-based validation, and focused template generation.

Step 3: Combine AI Vision with OpenCV-Based Validation

This workflow uses both AI and OpenCV, but they do different jobs.

In the first step, the model inspects the image itself and produces a visual diagnosis. That part is not driven by OpenCV code. It comes from the model's own multimodal reasoning: looking at the image and describing visible symptoms such as a tiny symbol, a noisy background, missing quiet zone, clipping, inversion, blur, or perspective distortion. That matters because template tuning is rarely just a brute-force search problem. Someone, or something, has to decide which parameter family deserves attention first.

GPT-5.4 is one option for that role, but it is not the only one. Other multimodal AI systems can also be used to generate focused templates, as long as they can actually inspect images and reason about visible symptoms before touching the JSON. The key requirement is not a specific vendor or model name. The key requirement is visual capability plus the ability to participate in a disciplined validation loop.

After that, OpenCV and DBR take over for the verification step. OpenCV loads the image, generates controlled variants when needed, and feeds those pixels into CaptureVisionRouter. DBR then gives the hard evidence: NO_RESULT, a successful decode, or a change in behavior after a specific template edit. In other words, AI proposes the diagnosis, while OpenCV and DBR test whether that diagnosis is actually true.

That combination is important because each side covers the other's weakness. AI alone is good at spotting visual patterns quickly, but it still needs a deterministic validation path. OpenCV alone is good at repeatable pixel operations, but it has no built-in understanding of which image symptom is most likely tied to scope, localization, decode, or acceptance settings. Put together, the workflow becomes much stronger: AI chooses a plausible next move, and OpenCV plus DBR turn that move into evidence.

Step 4: Validate the Current Template Before Editing JSON

The smallest trustworthy DBR check in this repo is template-optimizer/tools/validate_dbr_template.py. It loads one image, resolves one template name, initializes the license, and calls capture() directly. That makes it the right first step before you change anything.

The README already exposes the direct validation command:

python template-optimizer/tools/validate_dbr_template.py path/to/image.png --template-file candidate-template.json

The implementation is deliberately minimal:

def decode(image_path, template_file=None, template_name=None, fallback_profile="none"):
    ensure_dbr_license()
    router = CaptureVisionRouter()
    if template_file:
        err, msg = router.init_settings_from_file(str(template_file))
        if err != 0:
            print(f"[DBR] Template load failed ({err}): {msg}")

    cv_img = cv2.imread(str(image_path))
    if cv_img is None:
        raise RuntimeError(f"Failed to load image with OpenCV: {image_path}")

    result = router.capture(cv_img, template_name)
    return result.get_decoded_barcodes_result()

Example input image used throughout Steps 4-6: a tiny DataMatrix sits on a pink object within a large dark textured scene.

On the example image used in this project, that discipline immediately pays off. The raw image is dominated by dark textured background, while the DataMatrix occupies only a tiny region on a pink object. Running the default preset and the built-in templates on the original image produced only NO_RESULT, which is strong evidence that the first problem is not acceptance tuning. It is scene scope and localization.

Step 5: Probe Variants to Classify the Failure Before Editing JSON

Once the direct check fails, the next step is not to guess. It is to probe the image systematically. The helper script does exactly that by trying the preset, every bundled example template, and optional preprocessing variants.

The README exposes the broad evidence pass like this:

python template-optimizer/tools/probe_dbr_templates.py path/to/image.png --variant-set basic --report-json tuning-report.json

The variant generator is intentionally small and transparent:

def build_variants(image_path, variant_set):
    variants = [("original", image_path)]
    if variant_set == "none":
        return variants

    source = cv2.imread(str(image_path), cv2.IMREAD_COLOR)
    if source is None:
        return variants

    variant_dir = Path.cwd() / "_probe_variants" / image_path.stem
    variant_dir.mkdir(parents=True, exist_ok=True)

    def save_variant(name, image):
        path = variant_dir / f"{name}.png"
        cv2.imwrite(str(path), image)
        variants.append((name, path))

    padded = cv2.copyMakeBorder(
        source,
        40,
        40,
        40,
        40,
        cv2.BORDER_CONSTANT,
        value=(255, 255, 255),
    )
    save_variant("padded_40", padded)

    for scale in (2, 4):
        resized = cv2.resize(source, None, fx=scale, fy=scale, interpolation=cv2.INTER_NEAREST)
        save_variant(f"nearest_{scale}x", resized)

For the example image in this project, that probe gave the kind of evidence you want before generating a template:

The original image stayed NO_RESULT across the default preset and all built-in templates.
Padding alone did not fix the issue.
nearest_4x plus ReadBarcodes_ReadRateFirst decoded one DATAMATRIX result: DM-18Batch.
The variant results show that the main blocker is tiny effective symbol size in a large noisy scene, not an unrecoverable pixel failure.

That pattern is important. It means the pixels are recoverable, but the raw scene is too hostile for the broad default path. In other words, this is a good candidate for AI-generated focused tuning. In the final focused template, the key fix was to narrow the ROI to the centered symbol region so DBR spends its effort on the relevant area instead of the full textured background.

Step 6: Generate a Focused Single-Barcode DataMatrix Template

After the probe showed a single tiny DataMatrix and a scale-sensitive recovery path, AI can generate a much smaller template that matches the actual image. In this case, the key decisions are straightforward:

limit the format to BF_DATAMATRIX
limit the expected count to 1
add stronger localization modes for a tiny centered symbol
narrow the ROI to the center of the image instead of searching the whole frame
keep both normal and mirrored format specs available

AI can generate a focused template like the following when the image evidence clearly points to a single tiny DataMatrix:

{
    "BarcodeFormatSpecificationOptions" :
    [
        {
            "BarcodeFormatIds" :
            [
                "BF_DATAMATRIX"
            ],
            "DataMatrixSizeOptions" :
            [
                "DMS_DEFAULT"
            ],
            "ExpectedBarcodesCount" : 1,
            "FindUnevenModuleBarcode" : 1,
            "HasVerticalQuietZone" : 1,
            "MinQuietZoneWidth" : 4,
            "MinResultConfidence" : 30,
            "MirrorMode" : "MM_BOTH",
            "Name" : "bfs-single-tiny-datamatrix-both",
            "PartitionModes" :
            [
                "PM_WHOLE_BARCODE",
                "PM_ALIGNMENT_PARTITION"
            ],
            "ReturnPartialBarcodeValue" : 1
        }
    ],
    "TargetROIDefOptions" :
    [
        {
            "Name" : "roi-read-single-tiny-datamatrix",
            "Location" :
            {
                "Offset" :
                {
                    "FirstPoint" : [40, 30, 1, 1],
                    "SecondPoint" : [65, 30, 1, 1],
                    "ThirdPoint" : [65, 65, 1, 1],
                    "FourthPoint" : [40, 65, 1, 1],
                    "MeasuredByPercentage" : 1
                }
            }
        }
    ],
    "BarcodeReaderTaskSettingOptions" :
    [
        {
            "BarcodeFormatIds" :
            [
                "BF_DATAMATRIX"
            ],
            "ExpectedBarcodesCount" : 1,
            "Name" : "task-read-single-tiny-datamatrix"
        }
    ]
}

That generated template can then be validated against the target image with the same helper script:

python template-optimizer/tools/validate_dbr_template.py path/to/image.png --template-file path/to/generated-template.json

In this workspace, the custom template decoded the DataMatrix value DM-18Batch directly on the original image. That is the practical payoff of AI-generated template work: not a giant exported preset, but a small file that matches the failure mode you actually observed and solves it through template configuration alone.

Step 7: Load the Focused Template Back into the App

Once the focused template works, the desktop app can import it directly. This is why focused single-template files are especially useful in this project: the GUI explicitly uses the first template name in the JSON file.

Here is the import logic from main.py:

def _import_dbr_template(self):
    """Load a custom DBR template JSON file for all subsequent captures."""
    path, _ = QFileDialog.getOpenFileName(
        self, "Import DBR Template",
        self._active_template_path or "",
        "JSON files (*.json)"
    )
    if not path:
        return
    try:
        # Validate by trying to load into a temporary router
        router = CaptureVisionRouter()
        err, msg = router.init_settings_from_file(path)
        if err != 0:
            QMessageBox.warning(self, "Import Failed",
                                f"DBR error [{err}]: {msg}")
            return

        # Find the first template name inside the JSON so we use the right one
        with open(path, "r", encoding="utf-8") as f:
            data = json.load(f)
        templates = data.get("CaptureVisionTemplates", [])
        if templates:
            template_name = templates[0]["Name"]
        else:
            # Fallback: keep the default name
            template_name = EnumPresetTemplate.PT_READ_BARCODES.value

That small detail has a big architectural consequence: if you want predictable behavior in the GUI, generate a focused template whose first CaptureVisionTemplate is the one you actually want to run. AI-generated templates are much easier to keep clean than manually edited all-in-one exports.

Watch the Desktop Demo

The sample project already includes a short demo of the Python desktop tool that can import and test custom DBR templates:

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/annotation_tool

Build a Dual-Engine Barcode Annotation Tool in Python to Create Ground-Truth Datasets

Xiao Ling — Wed, 15 Apr 2026 08:04:29 +0000

Evaluating the accuracy of a barcode SDK requires clean, human-verified ground-truth data — but building that dataset by hand is slow and error-prone. Running two independent engines in parallel, comparing their outputs automatically, and letting a developer resolve discrepancies is a far more efficient approach. This tutorial shows how to build exactly that: a PySide6 desktop annotation tool driven by Dynamsoft Barcode Reader Bundle and ZXing-C++, capable of producing a barcode-benchmark/1.0 JSON dataset in minutes.

What you'll build: A Python desktop annotation application that loads image folders, auto-detects barcodes with dual engines (Dynamsoft DBR + ZXing-C++), highlights cross-engine mismatches for human review, supports manual quad drawing and crop-and-detect, and exports a portable ground-truth JSON dataset.

Demo Video: Dual-Engine Barcode Annotation Tool in Action

Prerequisites

Python 3.9 or later
pip install PySide6>=6.5 opencv-python>=4.8 numpy>=1.24 zxing-cpp>=3.0.0 dynamsoft-barcode-reader-bundle>=11.4.2000
A Dynamsoft Barcode Reader license key (a 30-day trial key is bundled in the sample; replace it for production use)

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Install all dependencies at once:

pip install -r requirements.txt

Then launch the annotation tool:

python main.py

Step 1: Initialise the Dynamsoft License

The SDK must be activated once at module level before any CaptureVisionRouter instance is created. The call is synchronous and caches the result, so subsequent launches work even without a network connection.

from dynamsoft_barcode_reader_bundle import (
    LicenseManager, CaptureVisionRouter, EnumPresetTemplate, EnumErrorCode
)

LICENSE_KEY = "LICENSE-KEY"

err, msg = LicenseManager.init_license(LICENSE_KEY)
if err not in (EnumErrorCode.EC_OK, EnumErrorCode.EC_LICENSE_CACHE_USED):
    print(f"[DBR] License warning: {msg}")

EC_LICENSE_CACHE_USED is a non-fatal status returned when the SDK uses its locally cached license after a successful prior activation, so both codes are treated as success.

Step 2: Run Dual-Engine Detection in a Background Thread

A QThread subclass runs both Dynamsoft DBR and ZXing-C++ on every image. The engines operate independently; their results are then merged by matching decoded text values. When both agree on the same text, the annotation is flagged match=True and routed to the Verified list automatically.

import zxingcpp
from PySide6.QtCore import QThread, Signal

class DetectionWorker(QThread):
    progress = Signal(int, int)    # done, total
    file_done = Signal(str, list)  # abs_path, annotations

    def __init__(self, file_paths, template_path=None, template_name=None, parent=None):
        super().__init__(parent)
        self._files = list(file_paths)
        self._template_path = template_path
        self._template_name = template_name or EnumPresetTemplate.PT_READ_BARCODES.value

    def run(self):
        router = CaptureVisionRouter()
        if self._template_path:
            err, msg = router.init_settings_from_file(self._template_path)
            if err != 0:
                print(f"[DBR] Template load failed ({err}): {msg}")
        total = len(self._files)
        for i, path in enumerate(self._files):
            anns = self._detect(router, path)
            self.file_done.emit(path, anns)
            self.progress.emit(i + 1, total)

    def _detect(self, router, path):
        # --- Dynamsoft DBR ---
        dbr_items = []
        try:
            result = router.capture(path, self._template_name)
            if result:
                dbr_r = result.get_decoded_barcodes_result()
                if dbr_r:
                    for item in (dbr_r.get_items() or []):
                        pts = item.get_location().points
                        dbr_items.append({
                            "text":   item.get_text(),
                            "format": item.get_format_string(),
                            "points": [(p.x, p.y) for p in pts],
                        })
        except Exception as exc:
            print(f"[DBR] {exc}")

        # --- ZXing-C++ ---
        zxing_items = []
        try:
            img = cv2.imread(path)
            if img is not None:
                for zx in zxingcpp.read_barcodes(img):
                    pos = zx.position
                    zxing_items.append({
                        "text":   zx.text,
                        "format": zx.format.name,
                        "points": [
                            (pos.top_left.x,     pos.top_left.y),
                            (pos.top_right.x,    pos.top_right.y),
                            (pos.bottom_right.x, pos.bottom_right.y),
                            (pos.bottom_left.x,  pos.bottom_left.y),
                        ],
                    })
        except Exception as exc:
            print(f"[ZXing] {exc}")

        # Merge: pair results by decoded text value
        zxing_text_set = {r["text"] for r in zxing_items}
        zxing_by_text  = {}
        for r in zxing_items:
            zxing_by_text.setdefault(r["text"], []).append(r)

        anns = []
        for r in dbr_items:
            match = r["text"] in zxing_text_set
            zxing_pair = zxing_by_text[r["text"]].pop(0) if match and zxing_by_text.get(r["text"]) else None
            anns.append({
                **r,
                "match":  match,
                "source": "dbr",
                "dbr":    {"text": r["text"], "format": r["format"], "points": r["points"]},
                "zxing":  {"text": zxing_pair["text"], "format": zxing_pair["format"],
                           "points": zxing_pair["points"]} if zxing_pair else None,
            })
        for r in zxing_items:
            if r["text"] not in {i["text"] for i in dbr_items}:
                anns.append({**r, "match": False, "source": "zxing", "dbr": None,
                             "zxing": {"text": r["text"], "format": r["format"], "points": r["points"]}})
        return anns

The worker is started from the main thread and its file_done signal is connected to the UI:

self._worker = DetectionWorker(
    paths,
    template_path=self._active_template_path,
    template_name=self._active_template_name,
)
self._worker.file_done.connect(self._on_file_detected)
self._worker.progress.connect(self._on_progress)
self._worker.finished.connect(self._on_detection_done)
self._worker.start()

Step 3: Classify and Display Results in Three Lists

Each image is classified as soon as its detection completes, without waiting for the full batch to finish. Images where every barcode was agreed on by both engines go straight to Verified (green), those with any mismatch go to Needs Review (red), and images where nothing was detected go to No Result (grey).

def _on_file_detected(self, path, annotations):
    self._files[path]["annotations"] = annotations
    filename = os.path.basename(path)

    item = QListWidgetItem(filename)
    item.setData(Qt.UserRole, path)
    if len(annotations) == 0:
        item.setForeground(QColor(120, 120, 120))
        self._list_noresult.addItem(item)
    elif all(a.get("match") for a in annotations):
        item.setForeground(QColor(0, 130, 0))
        self._list_verified.addItem(item)
    else:
        item.setForeground(QColor(180, 0, 0))
        self._list_review.addItem(item)

    self._grp_verified.setTitle(f"Verified ({self._list_verified.count()})")
    self._grp_review.setTitle(f"Needs Review ({self._list_review.count()})")
    self._grp_noresult.setTitle(f"No Result ({self._list_noresult.count()})")

    if self._current_path is None:
        self._select_file(path)

Step 4: Add and Edit Annotations Manually

Two complementary annotation modes handle cases where automatic detection fails.

Crop Mode — drag a rectangle over a region and both engines attempt to decode only that cropped area. Coordinates are translated back to the full image space before being stored:

def _decode_quad_region(self, path, quad_points):
    cv_img = load_image_cv(path)
    xs = [p[0] for p in quad_points]
    ys = [p[1] for p in quad_points]
    x1, y1 = max(0, int(min(xs))), max(0, int(min(ys)))
    x2, y2 = min(cv_img.shape[1], int(max(xs)) + 1), min(cv_img.shape[0], int(max(ys)) + 1)
    crop = cv_img[y1:y2, x1:x2]

    # Run DBR on the cropped region via a temp file
    import tempfile
    with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
        tmp_path = tmp.name
    cv2.imwrite(tmp_path, crop)
    router = CaptureVisionRouter()
    result = router.capture(tmp_path, self._active_template_name)
    os.unlink(tmp_path)
    # ... ZXing pass and coordinate offset back to full image omitted for brevity

Draw Mode — press D, click four corners of the barcode quad to draw a custom polygon, then enter the text and format in the BarcodeEditDialog. The annotation is stored with source="manual":

def _on_quad_drawn(self, quad_points):
    dlg = BarcodeEditDialog(text="", fmt="Unknown")
    if dlg.exec() != QDialog.Accepted:
        return
    ann = {
        "text":   dlg.text_edit.text(),
        "format": dlg.fmt_combo.currentText(),
        "points": quad_points,
        "match":  False,
        "source": "manual",
        "dbr":    None,
        "zxing":  None,
    }
    self._files[self._current_path]["annotations"].append(ann)
    self._redraw(self._current_path)

Every polygon corner is also a draggable DraggableVertex handle, so fine-tuning positions requires no mode switching — just click and drag.

Step 5: Export and Re-Import the Ground-Truth Dataset

Click Export JSON to save all loaded images and their annotations to a barcode-benchmark/1.0 JSON file. The schema is self-describing:

def _export_json(self):
    images_out = []
    total_barcodes = 0
    for path in self._all_paths:
        filename = os.path.basename(path)
        anns = (self._files.get(path) or {}).get("annotations") or []
        barcodes = [
            {"text": a["text"], "format": a["format"], "points": a["points"]}
            for a in anns
        ]
        total_barcodes += len(barcodes)
        images_out.append({"file": filename, "barcodes": barcodes})

    out = {
        "format":        "barcode-benchmark/1.0",
        "dataset":       "Annotated Collection",
        "total_images":  len(images_out),
        "total_barcodes": total_barcodes,
        "images":         images_out,
    }
    save_path, _ = QFileDialog.getSaveFileName(
        self, "Save Annotations", "annotations.json", "JSON (*.json)"
    )
    if save_path:
        with open(save_path, "w", encoding="utf-8") as f:
            json.dump(out, f, indent=2, ensure_ascii=False)

In a later session, click Import JSON to reload exported annotations directly onto the same image set — no re-detection is needed. Every imported barcode is immediately flagged as verified and appears in the Verified list:

annotations = [
    {
        "text":   bc.get("text", ""),
        "format": bc.get("format", "Unknown"),
        "points": [tuple(p) for p in bc.get("points", [])],
        "match":  True,
        "source": "imported",
        "dbr":    None,
        "zxing":  None,
    }
    for bc in raw_barcodes
]

Step 6: Use the Lightweight Read-Only Viewer

The companion viewer.py depends only on tkinter (Python built-in) and Pillow — no PySide6 or OpenCV required — making it easy to pass to teammates or testers who don't need to run the full annotation tool.

pip install Pillow
python viewer.py

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/annotation_tool

iOS Barcode SDK Benchmark: Dynamsoft vs ML Kit, Apple Vision, and ZXing-CPP in SwiftUI

Xiao Ling — Mon, 13 Apr 2026 06:41:47 +0000

Developers integrating barcode scanning into iOS apps have several viable options — Dynamsoft Barcode Reader (commercial), Google ML Kit (free), Apple Vision (native), and ZXing-CPP (open source) — each with different accuracy, speed, and format coverage characteristics. This article walks through building a SwiftUI benchmark app that runs all four SDKs side-by-side against identical image and video inputs. The results show Dynamsoft Barcode Reader as the clear leader: it achieved the highest detection count on a real-world test set while remaining the only SDK to combine that accuracy with hardware-accelerated, production-viable speed.

What you'll build: An iOS 16+ SwiftUI app that benchmarks four barcode scanning SDKs (Dynamsoft, ML Kit, Apple Vision, ZXing-CPP) across image, video, and live camera modes with colored bounding-box overlays and an HTML report exportable from a built-in HTTP server.

Demo Video: iOS Barcode Scanner Benchmark in Action

Prerequisites

Xcode 15.0+ with Swift 5.0 support
iOS 16.0+ device or simulator
CocoaPods installed (sudo gem install cocoapods) — required for Google ML Kit
A Dynamsoft Barcode Reader trial license key (the other three SDKs are free)

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Install and Configure the SDKs

ML Kit is not available through Swift Package Manager, so it is installed via CocoaPods. From the project root:

cd BarcodeBenchmarkiOS
pod install

After installation, always open BarcodeBenchmark.xcworkspace, not the .xcodeproj file. DynamsoftCaptureVisionBundle (v11.4.1200) and zxing-cpp (v2.3.0) are declared in project.pbxproj and resolved automatically by Xcode on first build. Apple Vision requires no additional setup — it is part of the iOS SDK.

The Podfile targets iOS 16 and links ML Kit 8.0.0:

platform :ios, '16.0'

target 'BarcodeBenchmark' do
  use_frameworks!

  # Google ML Kit Barcode Scanning
  pod 'GoogleMLKit/BarcodeScanning', '8.0.0'
end

Step 2: Define a Shared BarcodeDetector Protocol

All four detectors implement the same two-method protocol. This keeps the benchmark logic SDK-agnostic and makes it straightforward to add or remove a backend in the future:

protocol BarcodeDetector {
    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo]
    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo]
}

The UIImage overload is used for image and video-frame benchmarks; the CVPixelBuffer overload is called directly from the camera sample buffer delegate for live scanning.

Step 3: Implement the Dynamsoft Barcode Reader Detector

Dynamsoft uses CaptureVisionRouter from DynamsoftCaptureVisionBundle. License verification happens asynchronously on init; decoding is a single synchronous call to captureFromImage:

import DynamsoftCaptureVisionBundle
import DynamsoftBarcodeReaderBundle

class DynamsoftBarcodeDetector: NSObject, BarcodeDetector, LicenseVerificationListener {

    private let cvr: CaptureVisionRouter

    override init() {
        cvr = CaptureVisionRouter()
        super.init()
        LicenseManager.initLicense("YOUR_LICENSE_KEY_HERE", verificationDelegate: self)
    }

    func onLicenseVerified(_ isSuccess: Bool, error: Error?) {
        if !isSuccess {
            print("Dynamsoft license error: \(error?.localizedDescription ?? "Unknown error")")
        }
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        let result = cvr.captureFromImage(image, templateName: "ReadBarcodes_Default")
        guard let items = result.decodedBarcodesResult?.items else { return [] }
        return items.map { item in
            let bounds: CGRect? = {
                guard image.size.width > 0, image.size.height > 0 else { return nil }
                let br = item.location.boundingRect
                return CGRect(
                    x: br.minX / image.size.width,
                    y: br.minY / image.size.height,
                    width: br.width / image.size.width,
                    height: br.height / image.size.height)
            }()
            return BarcodeInfo(format: item.formatString ?? "", text: item.text ?? "", decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
        guard let cgImage = CIContext().createCGImage(ciImage, from: ciImage.extent) else {
            throw DetectionError.invalidImage
        }
        return try await detectBarcodes(in: UIImage(cgImage: cgImage))
    }
}

Step 4: Implement the ML Kit Barcode Detector

ML Kit uses an async-callback BarcodeScanner. The detector wraps the callback in withCheckedThrowingContinuation to fit the async throws protocol signature. Passing .all to BarcodeScannerOptions enables every supported format:

import MLKitBarcodeScanning
import MLKitVision

class MLKitBarcodeDetector: BarcodeDetector {

    private let scanner: MLKitBarcodeScanning.BarcodeScanner

    init() {
        let options = BarcodeScannerOptions(formats: .all)
        scanner = MLKitBarcodeScanning.BarcodeScanner.barcodeScanner(options: options)
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        let visionImage = VisionImage(image: image)
        visionImage.orientation = image.imageOrientation
        let imgW = image.size.width
        let imgH = image.size.height
        return try await withCheckedThrowingContinuation { continuation in
            scanner.process(visionImage) { barcodes, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                let results = (barcodes ?? []).map { barcode -> BarcodeInfo in
                    var bounds: CGRect?
                    if imgW > 0 && imgH > 0 {
                        let f = barcode.frame
                        bounds = CGRect(x: f.minX / imgW, y: f.minY / imgH,
                                        width: f.width / imgW, height: f.height / imgH)
                    }
                    return BarcodeInfo(format: barcode.format.description,
                                       text: barcode.rawValue ?? barcode.displayValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: results)
            }
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
        guard let cgImage = CIContext().createCGImage(ciImage, from: ciImage.extent) else {
            throw DetectionError.invalidImage
        }
        return try await detectBarcodes(in: UIImage(cgImage: cgImage))
    }
}

Step 5: Implement the Apple Vision Barcode Detector

Apple Vision requires no SDK installation. VNDetectBarcodesRequest is created per call and submitted via VNImageRequestHandler. Note that Vision's boundingBox origin is bottom-left, so the Y coordinate must be flipped before drawing overlays:

import Vision

class VisionBarcodeDetector: BarcodeDetector {

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        guard let cgImage = image.cgImage else {
            throw DetectionError.invalidImage
        }
        return try await withCheckedThrowingContinuation { continuation in
            let request = VNDetectBarcodesRequest { request, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                guard let results = request.results as? [VNBarcodeObservation] else {
                    continuation.resume(returning: [])
                    return
                }
                let barcodes = results.map { observation -> BarcodeInfo in
                    let bb = observation.boundingBox
                    // boundingBox origin is bottom-left; flip Y for top-left screen coords
                    let bounds = CGRect(x: bb.minX, y: 1.0 - bb.maxY,
                                       width: bb.width, height: bb.height)
                    return BarcodeInfo(format: self.mapSymbology(observation.symbology),
                                       text: observation.payloadStringValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: barcodes)
            }
            request.symbologies = [.qr, .code128, .code39, .code93, .ean8, .ean13,
                                    .upce, .pdf417, .aztec, .dataMatrix, .codabar, .itf14]
            let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
            do {
                try handler.perform([request])
            } catch {
                continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
            }
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        return try await withCheckedThrowingContinuation { continuation in
            let request = VNDetectBarcodesRequest { request, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                guard let results = request.results as? [VNBarcodeObservation] else {
                    continuation.resume(returning: [])
                    return
                }
                let barcodes = results.map { observation -> BarcodeInfo in
                    let bb = observation.boundingBox
                    let bounds = CGRect(x: bb.minX, y: 1.0 - bb.maxY,
                                       width: bb.width, height: bb.height)
                    return BarcodeInfo(format: self.mapSymbology(observation.symbology),
                                       text: observation.payloadStringValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: barcodes)
            }
            request.symbologies = [.qr, .code128, .code39, .code93, .ean8, .ean13,
                                    .upce, .pdf417, .aztec, .dataMatrix, .codabar, .itf14]
            let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
            do {
                try handler.perform([request])
            } catch {
                continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
            }
        }
    }
}

Step 6: Implement the ZXing-CPP Barcode Detector

ZXing-CPP is integrated via SPM using the zxing-cpp package and its iOS ZXingCpp wrapper. The ZXIBarcodeReader class handles both CGImage and CVPixelBuffer directly, and bounding coordinates are extracted from the result's position quad:

import ZXingCpp

class ZXingCppBarcodeDetector: BarcodeDetector {

    private let reader: ZXIBarcodeReader

    init() {
        reader = ZXIBarcodeReader()
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        guard let cgImage = image.cgImage else {
            throw DetectionError.invalidImage
        }
        let results = try reader.read(cgImage)
        let imgW = Int(image.size.width)
        let imgH = Int(image.size.height)
        return results.map { r in
            var bounds: CGRect?
            if imgW > 0 && imgH > 0 {
                let pos = r.position
                let xs = [pos.topLeft.x, pos.topRight.x, pos.bottomRight.x, pos.bottomLeft.x]
                let ys = [pos.topLeft.y, pos.topRight.y, pos.bottomRight.y, pos.bottomLeft.y]
                if let minX = xs.min(), let maxX = xs.max(),
                   let minY = ys.min(), let maxY = ys.max() {
                    bounds = CGRect(x: CGFloat(minX) / CGFloat(imgW),
                                    y: CGFloat(minY) / CGFloat(imgH),
                                    width: CGFloat(maxX - minX) / CGFloat(imgW),
                                    height: CGFloat(maxY - minY) / CGFloat(imgH))
                }
            }
            return BarcodeInfo(format: mapFormat(r.format), text: r.text,
                                decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let results = try reader.read(pixelBuffer)
        let bufW = CVPixelBufferGetWidth(pixelBuffer)
        let bufH = CVPixelBufferGetHeight(pixelBuffer)
        return results.map { r in
            var bounds: CGRect?
            if bufW > 0 && bufH > 0 {
                let pos = r.position
                let xs = [pos.topLeft.x, pos.topRight.x, pos.bottomRight.x, pos.bottomLeft.x]
                let ys = [pos.topLeft.y, pos.topRight.y, pos.bottomRight.y, pos.bottomLeft.y]
                if let minX = xs.min(), let maxX = xs.max(),
                   let minY = ys.min(), let maxY = ys.max() {
                    bounds = CGRect(x: CGFloat(minX) / CGFloat(bufW),
                                    y: CGFloat(minY) / CGFloat(bufH),
                                    width: CGFloat(maxX - minX) / CGFloat(bufW),
                                    height: CGFloat(maxY - minY) / CGFloat(bufH))
                }
            }
            return BarcodeInfo(format: mapFormat(r.format), text: r.text,
                                decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }
}

Step 7: Extract and Benchmark Barcode Detection Over Video Frames

For video benchmarks, VideoFrameExtractor uses AVAssetImageGenerator to sample frames at a fixed interval (default 0.5 s). Each frame is decoded as a CGImage and wrapped into a UIImage before being passed to the detector chain. The tolerance is set to zero to ensure exact frame positions are decoded:

class VideoFrameExtractor {
    static func extractFrames(from videoURL: URL, interval: Double) async throws -> [UIImage] {
        let asset = AVAsset(url: videoURL)
        let duration = try await asset.load(.duration)
        let durationSeconds = CMTimeGetSeconds(duration)

        let imageGenerator = AVAssetImageGenerator(asset: asset)
        imageGenerator.appliesPreferredTrackTransform = true
        imageGenerator.requestedTimeToleranceBefore = .zero
        imageGenerator.requestedTimeToleranceAfter = .zero

        var frames: [UIImage] = []
        var currentTime: Double = 0

        while currentTime < durationSeconds {
            let cmTime = CMTime(seconds: currentTime, preferredTimescale: 600)
            if let cgImage = try? imageGenerator.copyCGImage(at: cmTime, actualTime: nil) {
                frames.append(UIImage(cgImage: cgImage))
            }
            currentTime += interval
        }
        return frames
    }
}

Once frames are extracted, each SDK runs sequentially. The benchmark records total processing time and accumulates unique barcode results (format:text as a deduplication key) across frames — so a barcode visible in five consecutive frames counts as one detection:

private func runDetectorBenchmark(
    detector: BarcodeDetector,
    engineName: String,
    frames: [UIImage],
    totalFrames: Int,
    startProgress: Double,
    endProgress: Double
) async {
    var allBarcodes: [BarcodeInfo] = []
    var uniqueBarcodes = Set<String>()
    var totalTimeMs: Int64 = 0

    for (index, frame) in frames.enumerated() {
        let startTime = Date()
        let barcodes = (try? await detector.detectBarcodes(in: frame)) ?? []
        totalTimeMs += Int64(Date().timeIntervalSince(startTime) * 1000)

        for barcode in barcodes {
            let key = "\(barcode.format):\(barcode.text)"
            if uniqueBarcodes.insert(key).inserted {
                allBarcodes.append(barcode)
            }
        }

        let frameProgress = Double(index + 1) / Double(totalFrames)
        let overallProgress = startProgress + (frameProgress * (endProgress - startProgress))
        await MainActor.run {
            progress = overallProgress
            statusMessage = "\(engineName): Frame \(index + 1)/\(totalFrames)"
        }
    }

    await MainActor.run {
        var result = BenchmarkResult(engineName: engineName)
        result.framesProcessed = totalFrames
        result.totalTimeMs = totalTimeMs
        result.barcodes = allBarcodes
        // assign to the appropriate viewModel property
    }
}

Step 8: Configure the Live Camera Feed with AVFoundation

CameraManager wraps AVCaptureSession, selects the rear wide-angle camera, and configures AVCaptureVideoDataOutput to push sample buffers to a per-SDK delegate. Resolution is selectable between 720p and 1080p from the home screen before starting any scanner:

func setupCamera(resolution: CameraResolution, delegate: AVCaptureVideoDataOutputSampleBufferDelegate) throws {
    let session = AVCaptureSession()

    switch resolution {
    case .hd720:
        session.sessionPreset = .hd1280x720
    case .hd1080:
        session.sessionPreset = .hd1920x1080
    }

    guard let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else {
        throw CameraError.noCameraAvailable
    }

    let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
    if session.canAddInput(videoDeviceInput) {
        session.addInput(videoDeviceInput)
    }

    videoOutput.setSampleBufferDelegate(delegate, queue: DispatchQueue(label: "videoQueue"))
    videoOutput.alwaysDiscardsLateVideoFrames = true
    if session.canAddOutput(videoOutput) {
        session.addOutput(videoOutput)
    }

    if let connection = videoOutput.connection(with: .video) {
        connection.videoOrientation = .portrait
    }

    self.session = session
}

Each live scanner view (DynamsoftScannerView, MLKitScannerView, VisionScannerView, ZXingCppScannerView) draws color-coded bounding-box overlays — blue, green, purple, and orange respectively — using normalized CGRect coordinates returned by the detectors.

Step 9: Run Remote Benchmarks via the Built-In HTTP Server

Why Remote Benchmarking Is Valuable for Mobile SDK Evaluation

Evaluating a barcode SDK properly requires a large, diverse image corpus — often dozens to hundreds of files. Transferring this dataset to a mobile device and triggering each test manually is slow and error-prone. The built-in HTTP server solves this: open the device's IP in a desktop browser, drag-and-drop files in batch, and let the device run detection autonomously using its real CPU, GPU, and Neural Engine. The advantage over emulators or simulator tests is that results reflect the actual hardware pipeline — including ML accelerator scheduling and thermal throttling — which matters when comparing SDKs with different compute backends.

A secondary benefit is reproducibility: the server processes files in a consistent order, runs each SDK on the exact same decoded image bytes, and returns a structured JSON response per file that the browser accumulates into a downloadable HTML report. This workflow was used to produce the benchmark data in this article.

The server starts with a single call and makes the device's local IP available as a URL:

func startWebServer() {
    guard webServer == nil else { return }
    do {
        let server = try BenchmarkWebServer(port: BenchmarkConfig.serverPort, viewModel: self)
        try server.start()
        webServer = server
        isWebServerRunning = true
        if let ip = getLocalIPAddress() {
            serverURL = "http://\(ip):\(BenchmarkConfig.serverPort)"
        }
    } catch {
        print("Failed to start web server: \(error)")
    }
}

For each uploaded file the server determines whether it is an image or video (via MIME type or byte sniffing), runs all four detectors in series, and returns a JSON object with per-SDK barcode counts and timing. Video files are sampled at up to 30 evenly-spaced frames. A real-time progress endpoint (GET /api/video-progress) lets the browser poll SDK-level progress during video processing.

Benchmark Results

All benchmark data was collected via the remote HTTP server using 83 images from the Dynamsoft Barcode Test Sheet — a publicly available set covering QR codes, Data Matrix, PDF417, Aztec, EAN/UPC, Code 128/39/93, Codabar, ITF, and other common linear and 2D symbologies at various image qualities and orientations.

Source Code

https://github.com/yushulx/ios-swiftui-barcode-mrz-document-scanner/tree/main/examples/BarcodeBenchmarkiOS

How to Build a Cross-Platform Barcode, MRZ, and Document Scanner with .NET MAUI Blazor

Xiao Ling — Fri, 10 Apr 2026 06:04:04 +0000

Extracting structured data from barcodes, passports, and physical documents typically demands separate native implementations for every platform. The Dynamsoft Capture Vision SDK bundles barcode reading, MRZ recognition, and document detection into one JavaScript-callable API, and .NET MAUI Blazor lets you host that API inside a cross-platform BlazorWebView — giving you a single Razor codebase that runs on Android, iOS, macOS, and Windows.

What you'll build: A .NET MAUI Blazor app named "Vision Scanner" that scans barcodes, reads MRZ fields from passports and IDs, and detects document boundaries with a perspective-correction editor, running on all four major platforms using Dynamsoft Capture Vision SDK v3.2.5000.

Demo Video: .NET MAUI Blazor Vision Scanner in Action

Prerequisites

.NET 8 SDK
Visual Studio 2022 17.8+ with the .NET MAUI workload installed
A Dynamsoft Capture Vision license key

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Add the Dynamsoft Capture Vision Bundle to the Blazor Host Page

The SDK ships as a single CDN bundle. Add it to wwwroot/index.html before the Blazor framework script so that the Dynamsoft global is available as soon as the WebView loads.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, viewport-fit=cover" />
    <title>Vision Scanner</title>
    <base href="/" />
    <link rel="stylesheet" href="css/bootstrap/bootstrap.min.css" />
    <link href="css/app.css" rel="stylesheet" />
    <link href="BarcodeScanner.styles.css" rel="stylesheet" />
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-bundle@3.2.5000/dist/dcv.bundle.min.js"></script>
    <script src="jsInterop.js"></script>
</head>
<body>
    <div class="status-bar-safe-area"></div>
    <div id="app">Loading...</div>
    <script src="_framework/blazor.webview.js" autostart="false"></script>
</body>
</html>

Step 2: Register Platform-Specific Services and Configure the Android WebView

The camera abstraction (ICameraService) is registered in MauiProgram.cs. On macOS, WKWebView does not support getUserMedia, so a native AVFoundation implementation is wired in instead. On Android, a custom BlazorWebViewHandler replaces the default WebChromeClient to grant the WebView camera permission at the OS level.

// MauiProgram.cs
using Microsoft.AspNetCore.Components.WebView.Maui;
using BarcodeScanner.Services;

#if ANDROID
using BarcodeScanner.Platforms.Android;
#endif

namespace BarcodeScanner;

public static class MauiProgram
{
    public static MauiApp CreateMauiApp()
    {
        var builder = MauiApp.CreateBuilder();
        builder
            .UseMauiApp<App>()
            .ConfigureFonts(fonts =>
            {
                fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
            }).ConfigureMauiHandlers(handlers =>
            {
#if ANDROID
                handlers.AddHandler<BlazorWebView, MauiBlazorWebViewHandler>();
#endif
            });

        builder.Services.AddMauiBlazorWebView();
#if DEBUG
        builder.Services.AddBlazorWebViewDeveloperTools();
#endif

#if MACCATALYST
        builder.Services.AddSingleton<ICameraService, MacCameraService>();
#else
        builder.Services.AddSingleton<ICameraService, DefaultCameraService>();
#endif

        return builder.Build();
    }
}

The Android-specific MyWebChromeClient grants all WebView permission requests — including camera — so that getUserMedia resolves inside the Blazor page:

// Platforms/Android/MyWebChromeClient.cs
public class MyWebChromeClient : WebChromeClient
{
    private MainActivity _activity;

    public MyWebChromeClient(Context context)
    {
        _activity = context as MainActivity;
    }

    public override void OnPermissionRequest(PermissionRequest request)
    {
        try
        {
            request.Grant(request.GetResources());
            base.OnPermissionRequest(request);
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }
    }

    public override bool OnShowFileChooser(global::Android.Webkit.WebView webView,
        IValueCallback filePathCallback, FileChooserParams fileChooserParams)
    {
        base.OnShowFileChooser(webView, filePathCallback, fileChooserParams);
        return _activity.ChooseFile(filePathCallback,
            fileChooserParams.CreateIntent(), fileChooserParams.Title);
    }
}

This handler is injected via the custom MauiBlazorWebViewHandler:

// Platforms/Android/MauiBlazorWebViewHandler.cs
public class MauiBlazorWebViewHandler : BlazorWebViewHandler
{
    protected override global::Android.Webkit.WebView CreatePlatformView()
    {
        var view = base.CreatePlatformView();
        view.SetWebChromeClient(new MyWebChromeClient(this.Context));
        return view;
    }
}

Step 3: Initialize the SDK with a License Key

The Index.razor page presents a license activation screen. When the user taps Activate, Blazor calls the JavaScript initSDK function via IJSRuntime. The function initializes the license, pre-loads the WASM modules for barcode (DBR), label recognition (DLR), and document detection (DDN), loads the MRZ character-recognition models, and creates a CaptureVisionRouter instance.

@* Index.razor — Activate button handler *@
private async Task Activate()
{
    if (string.IsNullOrWhiteSpace(licenseKey)) return;
    isActivating = true;
    statusMessage = "";
    StateHasChanged();

    try
    {
        isActivated = await JSRuntime.InvokeAsync<bool>(
            "jsFunctions.initSDK", objRef, licenseKey);
        statusMessage = isActivated
            ? "SDK activated successfully!"
            : "SDK activation failed.";
    }
    catch (Exception ex)
    {
        statusMessage = "Error: " + ex.Message;
    }

    isActivating = false;
    StateHasChanged();
}

The corresponding JavaScript receives the call and boots the SDK:

// wwwroot/jsInterop.js — initSDK
initSDK: async function (dotnetRef, licenseKey) {
    dotnetHelper = dotnetRef;
    toggleLoading(true);
    try {
        await Dynamsoft.License.LicenseManager.initLicense(licenseKey, true);
        Dynamsoft.Core.CoreModule.loadWasm(["DBR", "DLR", "DDN"]);

        parser = await Dynamsoft.DCP.CodeParser.createInstance();
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD1_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_VISA");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_PASSPORT");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_VISA");
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer("MRZCharRecognition");
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer("MRZTextLineRecognition");

        cvr = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();

        isSDKReady = true;
        toggleLoading(false);
        return true;
    } catch (ex) {
        console.error(ex);
        toggleLoading(false);
        return false;
    }
},

Step 4: Decode Barcodes, MRZ, and Documents from Image Files

The Reader.razor page lets users pick an image from the device file system. After the file is selected, C# calls jsFunctions.loadAndDecodeFile, passing the <input> element ID and the active mode string ("barcode", "mrz", or "document"). The JavaScript side reads the file directly from the DOM, decodes it via cvr.capture(), and returns a JSON object containing the result text and any detected geometry.

The active template is selected from the mode string at call time:

// wwwroot/jsInterop.js — getTemplateName
function getTemplateName() {
    if (currentMode === 'barcode') return 'ReadBarcodes_Default';
    if (currentMode === 'mrz') return 'ReadMRZ';
    if (currentMode === 'document') return 'DetectDocumentBoundaries_Default';
    return 'ReadBarcodes_Default';
}

On the C# side, the Reader.razor component handles the file change event and parses the returned JSON to update the UI:

// Pages/Reader.razor — LoadImage
private async Task LoadImage(ChangeEventArgs e)
{
    result = "";
    showDocumentEditor = false;
    rectifiedImage = "";
    isLoading = true;
    StateHasChanged();

    try
    {
        // JS reads the file directly from the <input> element — no C# stream needed.
        var json = await JSRuntime.InvokeAsync<string>(
            "jsFunctions.loadAndDecodeFile", "reader_file_input", selectedMode);

        if (!string.IsNullOrEmpty(json))
        {
            using var doc = System.Text.Json.JsonDocument.Parse(json);
            var root = doc.RootElement;

            if (root.TryGetProperty("error", out var errProp))
            {
                result = "Error: " + errProp.GetString();
            }
            else
            {
                currentImageBase64 = root.TryGetProperty("image", out var imgProp)
                    ? imgProp.GetString() ?? "" : "";

                var decodeResult = root.TryGetProperty("result", out var resProp)
                    ? resProp.GetString() ?? "" : "";

For MRZ results, the JavaScript parser extracts structured fields from the raw MRZ string before returning them to C#:

// wwwroot/jsInterop.js — extractMrzInfo
function extractMrzInfo(result) {
    const parseResultInfo = {};
    parseResultInfo['Document Type'] = JSON.parse(result.jsonString).CodeType;
    parseResultInfo['Issuing State'] = result.getFieldValue("issuingState");
    parseResultInfo['Surname'] = result.getFieldValue("primaryIdentifier");
    parseResultInfo['Given Name'] = result.getFieldValue("secondaryIdentifier");
    let type = result.getFieldValue("documentCode");
    parseResultInfo['Passport Number'] = type === "P"
        ? result.getFieldValue("passportNumber")
        : result.getFieldValue("documentNumber");
    parseResultInfo['Nationality'] = result.getFieldValue("nationality");
    parseResultInfo["Gender"] = result.getFieldValue("sex");
    // ... date-of-birth and expiry fields
    return parseResultInfo;
}

Step 5: Stream the Live Camera Feed and Handle Platform Differences

Scanner.razor checks the injected ICameraService.RequiresNativeCamera flag to decide whether to use the standard getUserMedia web path or the macOS native AVFoundation path.

// Pages/Scanner.razor — OnAfterRenderAsync (first render)
if (useNativeCamera)
{
    await InitNativeCamera();
}
else
{
    await InitWebCamera();
}

Web camera path (Android, iOS, Windows): Blazor enumerates cameras via getUserMedia, opens the chosen device, and starts a requestAnimationFrame decode loop:

// Pages/Scanner.razor — InitWebCamera
private async Task InitWebCamera()
{
    cameraLabels = await JSRuntime.InvokeAsync<string[]>(
        "jsFunctions.initScanner", objRef, "camera_view", selectedMode);

    if (cameraLabels != null && cameraLabels.Length > 0)
    {
        StateHasChanged();
        await JSRuntime.InvokeVoidAsync("jsFunctions.openCamera", 0);
        await JSRuntime.InvokeVoidAsync("jsFunctions.startScanning", selectedMode);
    }
}

Native camera path (macOS): MacCameraService opens an AVCaptureSession, captures pixel buffers via AVCaptureVideoDataOutput, converts every second frame to a base64 JPEG, and exposes it through GetLatestFrame(). The Blazor component polls this method via a [JSInvokable] callback that the JavaScript timer calls on an interval:

// Pages/Scanner.razor — GetLatestFrame (JS-invokable poll endpoint)
[JSInvokable]
public string? GetLatestFrame()
{
    if (_disposed || !useNativeCamera) return null;
    return CameraService.GetLatestFrame();
}

Scan results from both paths flow back to C# through [JSInvokable] callbacks:

// wwwroot/jsInterop.js — processCameraResult (barcode branch)
if (currentMode === 'barcode' &&
    item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_BARCODE) {
    txts.push(item.text);
    globalPoints = item.location.points;
    if (overlayCtx) drawCameraOverlayQuad(overlayCtx, item.location.points, '#00ff00', 3);
}
// ...
if (currentMode === 'barcode' && txts.length > 0 && dotnetHelper) {
    dotnetHelper.invokeMethodAsync('OnScanResultReceived', txts.join('\n'));
}

Step 6: Rectify Captured Documents with a Browser-Side Homography

When Document mode is active and the user taps Capture, the frozen frame and the detected quad points are sent to C#. The Scanner.razor component opens an overlay editor where the user can drag the four corner handles to refine the boundary, then tap Rectify. The perspective warp is computed entirely in the browser using an inverse-mapping homography — no server round-trip.

// wwwroot/jsInterop.js — warpPerspective (inverse-mapping core)
function warpPerspective(srcCanvas, quadPts) {
    // ...
    const dstPts = [{ x: 0, y: 0 }, { x: W, y: 0 }, { x: W, y: H }, { x: 0, y: H }];
    const Hinv = buildHomography(dstPts, sp);   // inverse: dst → src

    for (let y = 0; y < H; y++) {
        for (let x = 0; x < W; x++) {
            const [sx, sy] = applyH(Hinv, x + 0.5, y + 0.5);
            // bilinear sample from source
            // ...
        }
    }
    dstCtx.putImageData(dstImg, 0, 0);
    return dstCanvas;
}

After rectification, the corrected image is displayed inline. The user can re-open the quad editor by tapping Edit, or export the image via jsFunctions.saveDocument, which triggers the native OS share sheet on Android/iOS or a file-save dialog on Windows and macOS.

Source Code

https://github.com/yushulx/dotnet-maui-blazor-barcode-mrz-document-scanner

Decode Barcodes and QR Codes from Uploaded Images in ASP.NET MVC 5

Xiao Ling — Tue, 07 Apr 2026 08:00:13 +0000

Reading barcodes and QR codes on the server side is a common requirement for document processing, inventory systems, and logistics applications. With the Dynamsoft Barcode Reader SDK and ASP.NET MVC 5 on .NET Framework 4.8, you can add accurate, multi-format barcode decoding to a web application in a few focused steps. The SDK's CaptureVisionRouter API handles the heavy lifting on the backend, while a lightweight JavaScript frontend previews the image and draws color-coded overlays over every detected code.

What you'll build: A responsive ASP.NET MVC 5 web application that accepts an uploaded image, decodes all barcodes and QR codes server-side using the Dynamsoft Barcode Reader SDK 11.4, and renders canvas overlays with format and text results in the browser.

See the ASP.NET MVC Barcode and QR Code Reader Demo

Prerequisites

.NET Framework 4.8 Developer Pack
Visual Studio 2019 or later with the ASP.NET and web development workload
NuGet CLI or the Visual Studio NuGet Package Manager
A Dynamsoft Barcode Reader license key

Get a 30-day free trial license

Step 1: Install and Configure the SDK in ASP.NET MVC 5

Add the Dynamsoft.DotNet.BarcodeReader.Bundle NuGet package to the project. The bundle ships with native binaries for both x64 and x86 Windows, which are automatically copied to the output directory via the included MSBuild targets file.

<!-- packages.config -->
<packages>
  <package id="Dynamsoft.DotNet.BarcodeReader.Bundle" version="11.4.2000" targetFramework="net48" />
  <package id="Microsoft.AspNet.Mvc"      version="5.2.9" targetFramework="net48" />
  <package id="Newtonsoft.Json" version="13.0.3" targetFramework="net48" />
</packages>

Restore packages from the command line before the first build:

nuget restore MvcBarcodeQRCodeFramework.csproj -PackagesDirectory packages

Step 2: Enable Attribute Routing for the Upload Endpoint

The barcode upload endpoint uses MVC attribute routing ([Route("upload")]). Enable it in RouteConfig.cs before the conventional route:

public static void RegisterRoutes(RouteCollection routes)
{
    routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

    // Enable attribute routing so [Route("upload")] works in FileController
    routes.MapMvcAttributeRoutes();

    routes.MapRoute(
        name: "Default",
        url: "{controller}/{action}/{id}",
        defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
    );
}

Step 3: Initialize the License and Decode Barcodes Server-Side

FileController accepts a multipart/form-data POST, saves each uploaded file to ~/App_Data/uploads/, initializes the license once per request, then calls CaptureVisionRouter.Capture() for each file. The decoded results — text, format string, and four-corner location — are serialized to JSON and returned to the browser.

using Dynamsoft.DBR;
using Dynamsoft.License;
using Dynamsoft.CVR;
using Dynamsoft.Core;

public class FileController : Controller
{
    [HttpPost]
    [Route("upload")]
    public ActionResult Upload()
    {
        var files = Request.Files;
        var uploadPath = Server.MapPath("~/App_Data/uploads");

        if (!Directory.Exists(uploadPath))
            Directory.CreateDirectory(uploadPath);

        string errorMsg;
        int errorCode = LicenseManager.InitLicense(
            "LICENSE-KEY",
            out errorMsg);

        if (errorCode != (int)EnumErrorCode.EC_OK &&
            errorCode != (int)EnumErrorCode.EC_LICENSE_CACHE_USED)
        {
            return Json(new { error = "License error: " + errorMsg });
        }

        var barcodes = new List<object>();

        using (var cvr = new CaptureVisionRouter())
        {
            for (int i = 0; i < files.Count; i++)
            {
                HttpPostedFileBase file = files[i];
                if (file == null || file.ContentLength == 0) continue;

                string fileName = Path.GetFileName(file.FileName);
                string filePath = Path.Combine(uploadPath, fileName);
                file.SaveAs(filePath);

                CapturedResult result = cvr.Capture(filePath, PresetTemplate.PT_READ_BARCODES);
                DecodedBarcodesResult barcodesResult = result.GetDecodedBarcodesResult();
                if (barcodesResult == null) continue;

                foreach (BarcodeResultItem item in barcodesResult.GetItems())
                {
                    var loc = item.GetLocation();
                    barcodes.Add(new
                    {
                        text   = item.GetText(),
                        format = item.GetFormatString(),
                        points = new[]
                        {
                            new { x = loc.points[0][0], y = loc.points[0][1] },
                            new { x = loc.points[1][0], y = loc.points[1][1] },
                            new { x = loc.points[2][0], y = loc.points[2][1] },
                            new { x = loc.points[3][0], y = loc.points[3][1] },
                        }
                    });
                }
            }
        }

        return Json(new { barcodes });
    }
}

PresetTemplate.PT_READ_BARCODES selects the SDK's built-in multi-format template, which covers QR Code, Code 128, Code 39, PDF417, DataMatrix, Aztec Code, and many more without any additional configuration.

Step 4: Upload the Image and Draw Canvas Overlays in the Browser

The frontend uses a plain XMLHttpRequest to POST the file as multipart/form-data as soon as an image is selected or dropped. FileReader creates a data-URL preview, and once the server responds, each barcode's four points are scaled from the image's natural pixel dimensions to its rendered size and drawn as a filled, stroked quadrilateral on an absolutely-positioned <canvas> overlay.

function handleFile(file) {
    var reader = new FileReader();
    reader.addEventListener('load', function () {
        previewArea.style.display = 'flex';
        imageEl.src = reader.result;
        imageEl.onload = function () {
            syncCanvas();
            upload(file);
        };
    });
    reader.readAsDataURL(file);
}

function upload(file) {
    spinnerEl.style.display = 'flex';
    var formData = new FormData();
    formData.append('barcodeImage', file, file.name);

    var xhr = new XMLHttpRequest();
    xhr.open('POST', '/upload', true);
    xhr.onreadystatechange = function () {
        if (xhr.readyState !== 4) return;
        spinnerEl.style.display = 'none';
        if (xhr.status === 200) {
            try {
                showResults(JSON.parse(xhr.responseText));
            } catch (e) {
                showError('Failed to parse server response.');
            }
        } else {
            showError('Upload failed (HTTP ' + xhr.status + ').');
        }
    };
    xhr.send(formData);
}

var PALETTE = ['#00b4d8', '#ff6b6b', '#51cf66', '#fcc419', '#cc5de8', '#ff922b'];

function drawQuad(points, sx, sy, index) {
    var color = PALETTE[index % PALETTE.length];
    ctx.save();
    ctx.strokeStyle = color;
    ctx.lineWidth   = 2.5;
    ctx.fillStyle   = color + '30';
    ctx.beginPath();
    ctx.moveTo(points[0].x * sx, points[0].y * sy);
    for (var i = 1; i < points.length; i++)
        ctx.lineTo(points[i].x * sx, points[i].y * sy);
    ctx.closePath();
    ctx.fill();
    ctx.stroke();

    var cx = (points[0].x + points[1].x + points[2].x + points[3].x) / 4 * sx;
    var cy = (points[0].y + points[1].y + points[2].y + points[3].y) / 4 * sy;
    ctx.fillStyle = color;
    ctx.beginPath();
    ctx.arc(cx, cy, 12, 0, Math.PI * 2);
    ctx.fill();
    ctx.fillStyle = '#fff';
    ctx.font = 'bold 12px sans-serif';
    ctx.textAlign = 'center';
    ctx.textBaseline = 'middle';
    ctx.fillText(index + 1, cx, cy);
    ctx.restore();
}

sx and sy are scale factors computed as imageEl.offsetWidth / imageEl.naturalWidth (and the height equivalent), ensuring overlays remain in sync regardless of how the browser scales the image to fit the viewport.

Step 5: Build and Run with IIS Express

Build the project from the command line or open it in Visual Studio. To run without Visual Studio's debug launcher, call IIS Express directly:

msbuild MvcBarcodeQRCodeFramework.csproj /p:Configuration=Debug /t:Build
"C:\Program Files\IIS Express\iisexpress.exe" /path:"%CD%" /port:62873

Then open http://localhost:62873/ in a browser, drop or select an image with barcodes, and results appear instantly.

Source Code

https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/official/aspnet-framework