DEV Community: Howth Technology Factory

Daily batch rescan vs event-driven rescan: an architecture choice

Howth Technology Factory — Sun, 23 Aug 2026 23:00:00 +0000

Daily batch is simpler to operate and reason about, with up to 24 hours of lag baked in. Event-driven rescanning triggers off list updates directly and gets that lag close to zero, at the cost of another moving part in your infra. Sanctions Screening API

How fast a sanctions list update actually reaches your API call

Howth Technology Factory — Sat, 22 Aug 2026 23:00:00 +0000

A regulator publishes an update, we ingest and re-index it, then it's live in every screening call, around 12 minutes end to end in our pipeline. Worth checking this number for whatever you're using, a stale cached list is a silent failure mode. Sanctions Screening API

Exact match vs fuzzy match, worked examples

Howth Technology Factory — Fri, 21 Aug 2026 23:00:00 +0000

Transliteration, abbreviation, and typos all break naive string equality. Running the same four name variants through exact and fuzzy matching side by side, exact match missed three of the four. Sanctions Screening API

What our sandbox test suite actually covers before go-live

Howth Technology Factory — Thu, 20 Aug 2026 23:00:00 +0000

Known test names, a deliberately misspelled variant of each, a name that should never match, and simulated timeouts and errors. Test only the happy path and you'll find out about the rest in production. Sanctions Screening API

Stop collapsing match confidence into a boolean

Howth Technology Factory — Wed, 19 Aug 2026 23:00:00 +0000

Returning true/false from a screening endpoint throws away the one piece of data a caller actually needs to build a good review UI: how close the match was. We return the raw score so callers can set their own thresholds. Sanctions Screening API

Sanctions screening latency, benchmarked

Howth Technology Factory — Tue, 18 Aug 2026 23:00:00 +0000

Self-hosted batch jobs land around 4 seconds. A generic REST wrapper gets under a second. Ours runs under 200ms because it's built as an event-priced API from the ground up, not bolted onto a batch pipeline. Sanctions Screening API

Handling a sanctions API timeout without guessing

Howth Technology Factory — Mon, 17 Aug 2026 23:00:00 +0000

When a screening call times out mid-request, there are three real options: fail closed, fail open, or queue and retry. We built explicit configuration for this instead of leaving it to whatever a generic HTTP client happens to default to. Sanctions Screening API

PDF Conversion Formats Compared: When to Use Image, DOCX, or Flattened Output

Howth Technology Factory — Sun, 16 Aug 2026 23:00:00 +0000

"Convert this PDF" is an underspecified request until someone answers convert it to what, and the three most common answers, image, DOCX, and flattened PDF, aren't interchangeable. Picking the wrong one doesn't usually fail loudly. It just produces an output that's technically correct and practically useless for whatever the next step in the pipeline actually needed.

Image output: when the PDF's content doesn't matter, only its appearance

Rendering a PDF page to PNG or JPEG throws away every notion of text, structure, and searchability, and keeps only exact visual appearance. That's a bad trade for most conversion use cases and exactly the right one for a specific few: thumbnail previews in a document list, a quick visual diff between two versions of a file, or embedding a snapshot of a page somewhere that can't render PDFs natively. If nothing downstream ever needs to search, copy, or edit the content, image output is the cheapest and most reliable option, because there's no layout-fidelity tradeoff to worry about, the image is by definition a pixel-accurate copy.

result = pdf_api.run({"action": "convert", "file": file, "to": "png", "pages": "1"})

The failure mode with image output is almost always someone reaching for it out of habit and then getting stuck later needing to search or edit content that got flattened into pixels with no way back.

DOCX output: when someone needs to actually edit the content

Converting to DOCX aims at the opposite goal: preserving editable text and structure, at the cost of exact visual fidelity. A PDF built from a complex multi-column layout, custom fonts, or precise absolute positioning will come out of a PDF-to-DOCX conversion looking approximately right, not pixel-identical, because DOCX's layout model doesn't work the same way a PDF's does. For a contract that needs redlining, a report someone needs to update the numbers in, or any document whose whole point is further editing, that tradeoff is obviously worth it. For a signed, finalized document nobody should be able to modify, it's the wrong tool entirely.

result = pdf_api.run({"action": "convert", "file": file, "to": "docx"})

The most common mistake here is treating DOCX conversion as a universal "make this PDF editable" button and being surprised when a heavily designed PDF, a marketing one-pager with precise graphic layout, say, comes back looking noticeably different from the source once it's been reflowed into a Word document's layout model.

Flattened PDF output: when you need a PDF that stays a PDF

Flattening isn't a format change at all, it's a PDF that goes in and a PDF that comes out, but with form fields, layers, and annotations merged permanently into the page content so nothing about it can be edited or filled in afterward. This is the right output for anything that needs to look and behave exactly like the source but can no longer be modified: a signed contract, an approved invoice, a filled-out form that's now final. Where image output sacrifices content and DOCX sacrifices exact layout, flattening sacrifices nothing about appearance and instead removes editability on purpose, which is the entire point.

result = pdf_api.run({"action": "flatten", "file": file})
Matching the format to what happens next, not what's easiest to generate

The recurring mistake across all three isn't picking a bad format in isolation, it's picking a format based on what's simplest to generate right now instead of what the next step in the pipeline actually consumes. A document management system that stores flattened PDFs for the archive but needs image thumbnails for the browse view needs both outputs from the same source file, not one format doing double duty badly. Building the conversion step around "what does the consumer need" rather than "what's the default" avoids having to redo the conversion later once the real requirement surfaces.

Combining formats instead of picking just one

Nothing requires committing to a single output per document. A common pattern is generating an image for the list view thumbnail, keeping a flattened PDF as the canonical archived copy, and offering DOCX conversion only on demand when someone explicitly requests an editable copy. Three conversion calls against the same source file, each serving a different downstream need, cost more in API calls than picking one format and hoping it covers every case, but far less than discovering midway through a project that the format chosen up front doesn't support a feature that turned out to matter.

Getting the source quality right before converting at all

None of these three outputs can recover information that wasn't legible in the source PDF to begin with. A scanned document with no underlying text layer converts to DOCX with garbled or missing text no matter how good the conversion engine is, because there was no text to preserve in the first place, only pixels. Checking whether a PDF has real text content before promising a clean DOCX conversion saves the awkward conversation of explaining afterward why the "editable" version isn't actually editable.

Conversion failures don't look the same across formats

A failed image conversion is usually obvious immediately, a blank or corrupted file that nobody would mistake for success. A failed or degraded DOCX conversion is often much quieter: the file opens fine, looks mostly right, and the missing table or misplaced paragraph only surfaces when someone actually reads it closely later. That difference matters for how much manual review each format needs before an output gets treated as final. Image and flattened PDF outputs can generally be trusted on a status-code check alone, since there isn't much room for a subtly wrong success. DOCX output benefits from at least a spot check on complex source documents, tables, multi-column layouts, embedded images, since "the conversion succeeded" and "the conversion is faithful" aren't quite the same claim for that format the way they are for the other two.

One API, three outputs, one decision to make correctly

None of the three formats is a default anyone should reach for without first checking what happens to the output next. All three are handled by the same multi-format PDF conversion API alongside merge, split, compress, rotate, and watermark, priced per successful result, which means the cost of picking the right format for each specific use case rather than one format for everything is a design decision, not an infrastructure one.

What counts as a "successful" sanctions screening API call

Howth Technology Factory — Sun, 16 Aug 2026 23:00:00 +0000

A definitive match/no-match, an inconclusive result, and a technical error are three different outcomes, but plenty of integrations just check for HTTP 200 and call it done. Getting billing and audit logs right means treating those three differently from the start. Sanctions Screening API

I Wrapped a PDF API in a CLI Tool So I'd Never Touch the Docs Again

Howth Technology Factory — Sun, 16 Aug 2026 09:46:00 +0000

Every PDF API I've ever used has the same rough edges the second or third time I need it for something quick: open the docs again, remember the auth header format, remember whether it's files or file_urls, write a throwaway script, run it once, delete it. That workflow is fine the first time. It gets old by the fifth.

So I wrapped the whole thing in a fifteen-minute CLI tool, and it turned out to be one of the higher-leverage things I've built this year, mostly because of how often "quick PDF task" shows up in a normal week of backend work.

What it actually needed to do

Nothing fancy. A thin command-line wrapper around a handful of PDF operations, merge, split, compress, watermark, rotate, convert, that takes file paths as arguments and writes the result to disk. No server, no UI, just pdftk -o , callable from a terminal, a shell script, or a Makefile target.

The implementation

The whole thing is built on Python's argparse, with one subcommand per operation:

import argparse, sys
import pdf_api

def main():
parser = argparse.ArgumentParser(prog="pdftk")
sub = parser.add_subparsers(dest="action", required=True)

merge_p = sub.add_parser("merge")
merge_p.add_argument("files", nargs="+")
merge_p.add_argument("-o", "--output", required=True)

compress_p = sub.add_parser("compress")
compress_p.add_argument("file")
compress_p.add_argument("-o", "--output", required=True)

watermark_p = sub.add_parser("watermark")
watermark_p.add_argument("file")
watermark_p.add_argument("--text", required=True)
watermark_p.add_argument("-o", "--output", required=True)

args = parser.parse_args()

if args.action == "merge":
    result = pdf_api.run({"action": "merge", "files": args.files})
elif args.action == "compress":
    result = pdf_api.run({"action": "compress", "file": args.file})
elif args.action == "watermark":
    result = pdf_api.run({"action": "watermark", "file": args.file, "text": args.text})

if result.status != "success":
    print(f"✗ {args.action} failed: {result.status}", file=sys.stderr)
    sys.exit(1)

result.save(args.output)
print(f"✓ {args.action} → {args.output}")

if name == "main":
main()

That's most of it. Each subcommand maps directly onto one API call, and the whole file is short enough to read top to bottom in under a minute, which matters more than it sounds like it should.

Why this was worth building

The honest case for a CLI wrapper isn't that it's technically impressive, it clearly isn't. It's that "quick PDF task" shows up constantly in ordinary backend work, cleaning up a batch of scanned files before a demo, merging a few reports before sending them somewhere, watermarking a draft before a review, and every one of those used to mean either opening a script I'd half-forgotten or writing a new one from scratch. Now it means one command, tab-completed, with output I can immediately check with ls and open.

There's a second, quieter benefit: once the CLI existed, it became trivial to drop pdftk calls into shell scripts, Makefiles, and pre-commit-style hooks, places where writing a full script every time would never have felt worth it, but running one existing command absolutely does.

What I'd do differently building it again

I'd add a --json flag sooner than I did. The human-readable success and failure lines are great for interactive use and terrible for anything that wants to parse the output programmatically, and I ended up needing both within the first week of using it inside other scripts. A --json flag that switches the output to structured status objects would have saved me from grepping stdout, which is exactly the kind of thing that's obvious in hindsight and easy to skip when you're building something "just for yourself" on a Friday afternoon.

Handling config and auth without making it annoying

The other thing worth getting right early is where the API key lives. My first version took it as a command-line flag, which is exactly as annoying as it sounds the second time you type pdftk merge a.pdf b.pdf -o c.pdf --key sk_live_xxxxx into a terminal history that other processes on the machine can read. It moved to an environment variable almost immediately, PDFTK_API_KEY, read once at startup, with a clear error message if it's missing rather than a stack trace three calls deep into the API client:

import os

def get_api_key():
key = os.environ.get("PDFTK_API_KEY")
if not key:
print("✗ PDFTK_API_KEY is not set. Export it and try again.", file=sys.stderr)
sys.exit(1)
return key

That's a small thing, but it's the difference between a tool that feels safe to hand to a teammate and one that quietly encourages people to paste secrets into shell history. A CLI tool that's going to get used more than once by more than one person is worth treating like a real piece of internal tooling, even when it started as a fifteen-minute Friday project.

Packaging it so it's actually easy to install

The last step that made this genuinely sticky on the team was packaging it as an installable command rather than a script people had to remember the path to. A minimal pyproject.toml with an entry point turns python pdftk.py merge ... into just pdftk merge ..., installable with pip install -e . from the repo, or published internally if it needs to reach more than one machine. It's a small amount of extra setup, and it's the difference between a tool a couple of people know about and a tool that shows up in everyone's $PATH without anyone having to explain it twice. Once that happened, I started seeing it show up in other people's shell scripts and Makefiles without me having anything to do with it, which is usually a decent signal that a small internal tool earned its place.

Where error handling actually needs the most care

The failure mode that mattered most in practice wasn't a missing file or a bad flag, both of which argparse handles cleanly on its own. It was a batch of files where one file in the middle was corrupted or password-protected, and the CLI needed to say exactly which one, rather than failing the whole batch with a generic error and leaving me to guess. That meant surfacing the API's structured failure reason directly in the CLI's output instead of collapsing every non-success response into the same flat "failed" message, a small design choice that made the tool meaningfully more useful the first time a real, messy batch of files hit it.

The API underneath it

None of this would be a fifteen-minute project if the CLI also had to implement merge, split, compress, watermark, rotate, and convert itself. It's a thin wrapper around a PDF API for CLI tools, covering all six operations as simple calls, priced per successful result, which meant the entire build was argument parsing and output formatting, nothing else. If you're reaching for the same API from a terminal more than twice a week, it's worth the fifteen minutes.

Docker, Queues, and PDFs: Containerizing a Document Pipeline That Doesn't Fall Over

Howth Technology Factory — Sat, 15 Aug 2026 23:00:00 +0000

The first version of this pipeline was a single long-running process handling PDF jobs off a queue directly inside the app server. It worked until it didn't: a deploy would restart the app mid-job, an in-flight PDF job would just vanish, and nobody would notice until a customer asked where their file was. Moving the actual PDF work into its own containerized worker pool fixed that, but only once a few specific things were done deliberately rather than assumed to work by default.

Why a worker pool beats a single long-running process

A single process handling jobs sequentially has one obvious failure mode: whatever kills that process kills every job it was holding, whether that's an out-of-memory crash, a deploy restart, or a bad input that takes the process down with it. A pool of worker containers pulling from a shared queue doesn't have that problem in the same way. If one worker dies mid-job, the message it was processing simply becomes visible in the queue again, and any other running worker can pick it up. The unit of failure shrinks from "the whole pipeline" to "one job, briefly delayed."

Making the worker actually safe to kill at any moment

None of that resilience matters if the worker's PDF logic doesn't tolerate being interrupted and retried. A worker needs to be safe to kill at literally any point in its execution, which mostly comes down to not doing partial, unrecoverable writes before the job is confirmed done:

def process_job(message):
job = parse_job(message.body)
result = pdf_api.run({"action": job.action, "files": job.files})
if result.status != "success":
raise JobFailed(result.status)
write_output(job.output_location, result.output_url)
message.delete() # only acknowledge after output is durably written

The message only gets deleted from the queue after the output has actually landed somewhere durable. If the container gets killed anywhere before that last line runs, the message stays in the queue and another worker retries the whole job from scratch, which is safe precisely because nothing partial or misleading got written in the meantime.

Dockerizing the worker without dragging in the whole app

It's tempting to reuse the main app's Docker image for the worker, since most of the dependencies overlap. In practice a slimmer, purpose-built worker image is worth the extra Dockerfile: it starts faster, has a smaller attack surface, and doesn't rebuild every time an unrelated part of the app changes. A worker only needs the queue client, the PDF API client, and whatever minimal logic turns a queue message into an API call, nothing the rest of the app depends on for serving requests.

FROM python:3.12-slim
WORKDIR /app
COPY worker/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY worker/ .
CMD ["python", "worker.py"]
Scaling replicas without overrunning the rate limit

More worker replicas means more concurrent PDF API calls, which is exactly the throughput win the whole setup exists for, right up until the combined concurrency across all replicas exceeds the API's rate limit and starts generating 429s instead of finished jobs. The fix isn't to guess at a safe replica count and hope, it's to make each worker respect a shared concurrency budget, whether that's a per-worker semaphore sized against the replica count or a centralized rate limiter the whole pool checks against before firing a request.

Health checks that actually reflect whether the worker is stuck

A container orchestrator restarting an unhealthy worker is only useful if "unhealthy" is defined correctly. A worker that's alive but has silently stopped pulling from the queue, deadlocked on a bad message, say, will pass a naive liveness check indefinitely while doing nothing. A more useful health check tracks the timestamp of the worker's last successful job pull and fails if too much time has passed with the queue non-empty, which catches the stuck case a simple "is the process running" check misses entirely.

Logging that survives the container's death

A worker container that crashes mid-job takes its local logs with it unless those logs were already shipped somewhere durable. Centralized logging isn't optional infrastructure here, it's the only way to actually diagnose why a particular job failed after the container that was running it no longer exists. Structured logs tagged with the job ID, written as the job progresses rather than only at the end, mean a crash still leaves a trail even if the final "success" line never gets written.

Handling SIGTERM instead of just getting killed

Container orchestrators don't usually kill a container outright on a routine scale-down or deploy, they send SIGTERM first and give it a grace period before the harder kill signal follows. A worker that ignores this and gets killed mid-job the same way an OOM crash would is throwing away a resilience mechanism that's already available for free. Catching SIGTERM, finishing the in-flight job if it's close to done, and only then exiting cleanly turns a routine deploy into something that costs zero retried jobs instead of one per worker that happened to be mid-task:

import signal

shutting_down = False

def handle_sigterm(signum, frame):
global shutting_down
shutting_down = True

signal.signal(signal.SIGTERM, handle_sigterm)

while not shutting_down:
message = queue.receive(wait_seconds=5)
if message:
process_job(message)

The grace period the orchestrator gives before the hard kill needs to be long enough to cover the worst-case job duration, not just the typical one, or this pattern just moves the problem instead of solving it. A worker that occasionally handles a large file taking twenty seconds needs more headroom than the default grace period most orchestrators ship with.

What this pattern buys once it's running

The pipeline stopped losing jobs on deploys, which was the original problem, but the bigger change was that scaling stopped being a code change. Adding capacity for a busier day meant adding replicas, not rewriting the worker. None of the container orchestration logic needed to know anything about PDF operations specifically, that complexity stayed fully inside the containerized PDF processing API handling merge, split, compress, rotate, watermark, and convert, priced per successful result, with the worker pool responsible only for pulling messages, calling it, and acknowledging cleanly once the output actually exists.

From LangChain Tool to Production: Wiring a PDF API into an Agent Framework

Howth Technology Factory — Fri, 14 Aug 2026 23:00:00 +0000

The demo version of a LangChain PDF tool is a few lines: subclass BaseTool, implement _run, done. The production version needed a lot more thought about what happens when the agent picks the wrong tool, passes malformed arguments, or gets a failure response it has to reason about, none of which shows up in the demo because the demo only ever exercises the happy path.

The minimal version
from langchain.tools import BaseTool
from pydantic import BaseModel, Field

class MergeInput(BaseModel):
files: list[str] = Field(description="URLs of PDF files to merge, in order")

class PDFMergeTool(BaseTool):
name = "merge_pdfs"
description = "Merge multiple PDF files into one, in the given order."
args_schema = MergeInput

def _run(self, files: list[str]) -> str:
    result = pdf_api.run({"action": "merge", "files": files})
    if result.status != "success":
        return f"Merge failed: {result.status}"
    return result.output_url

This works fine in a demo with two clean test files. It's also missing almost everything that matters once an agent is calling it with arguments it generated itself, from a natural-language instruction, without a human checking the call before it fires.

Why the tool description carries more weight than it looks like it should

An agent decides which tool to call, and with what arguments, based substantially on the tool's description field. A vague description, "merges PDFs," leaves the agent guessing about input format, ordering behavior, and what happens with more than two files. A more specific description meaningfully improves how reliably the agent picks the right tool and fills in the right arguments:

description = (
"Merge multiple PDF files into a single PDF, in the exact order the "
"'files' list is given. Use this when the user asks to combine, "
"join, or merge two or more PDF documents. Requires at least 2 files."
)

That level of specificity isn't overkill, it's doing real work: reducing the number of times the agent calls this tool for a task it wasn't meant for, or calls it with a single file because the description didn't make the minimum clear.

Making tool output legible to the agent, not just to a human

The _run method's return value becomes part of the agent's context for its next reasoning step. A raw exception traceback or an ambiguous string like "error" gives the agent almost nothing to act on. A specific, structured message lets the agent make a reasonable next decision, retry, apologize to the user, try a different tool, without needing a human to intervene:

def _run(self, files: list[str]) -> str:
if len(files) < 2:
return "Error: merge requires at least 2 files, only 1 was provided."
result = pdf_api.run({"action": "merge", "files": files})
if result.status == "encrypted_input":
return "Error: one or more files are password-protected and can't be merged automatically."
if result.status != "success":
return f"Error: merge failed ({result.status}). Try again or check the input files."
return f"Success: merged {len(files)} files into {result.output_url}"

The distinction between "Error: merge requires at least 2 files" and a generic failure matters a lot in practice, because it's the difference between an agent that can self-correct on the next turn and one that just retries the same broken call.

Guarding against the agent calling the tool with bad arguments

Pydantic's args_schema catches obviously malformed input, wrong types, missing fields, before _run even executes, which handles a meaningful share of agent mistakes for free. It doesn't catch everything: an agent can pass a syntactically valid list of URLs that don't actually point at PDFs, or an empty list that technically satisfies the schema. Validating the semantic content, not just the shape, inside _run itself is still necessary, as shown in the len(files) < 2 check above.

Registering multiple operations without one giant tool

Rather than one tool with an action parameter covering all six operations, merge, split, compress, rotate, watermark, convert, each operation gets its own tool with its own name and description. This costs a bit of boilerplate and buys a lot of reliability: an agent choosing between merge_pdfs, compress_pdf, and watermark_pdf by name is working with a much clearer decision than one choosing an action string buried inside a single generic tool's arguments, which is exactly the kind of narrow-surface design that tends to hold up better with agent callers generally.

Cost control when the agent decides how many times to call it

A human calling this API decides, consciously, when to trigger a merge or a compress. An agent decides that too, but it's deciding based on a chain of reasoning that can go wrong in ways a human wouldn't. An agent that misreads a task and merges the same set of files three times before noticing the output already exists, or one that gets stuck in a loop retrying a tool call that will never succeed because the underlying request is malformed, can run up real usage without anyone intending it to. This is a smaller problem with an API priced per successful result, since a failed or malformed call that never produces output isn't charged, but a tool that keeps producing valid results the agent didn't actually need still costs money for no benefit.

The practical fix isn't in the tool itself, it's in the surrounding agent framework: a simple call counter per session, or a check that flags when the same tool gets invoked with near-identical arguments more than once or twice in a row. Neither of those requires touching the PDF API, they're just guardrails around how liberally the agent is allowed to use the tools it has access to, and they're worth adding before an agent framework goes anywhere near production traffic rather than after a surprising bill shows up.

The API these tools are thin wrappers around

None of the six tools implement any actual PDF manipulation. Each one is a schema, a description, and a call to a PDF API for agent frameworks that handles merge, split, compress, rotate, watermark, and convert, priced per successful result. The work that actually mattered here was almost entirely about making the tool's interface legible to something that has to decide when and how to call it without a human in the loop, not about the PDF operations themselves.