DEV Community: Howth Technology Factory

BuiltWith vs Wappalyzer vs pay-per-use: the 2026 economics of tech stack detection published

Howth Technology Factory — Fri, 24 Jul 2026 14:46:00 +0000

If you sell to companies based on what software they run ("find me every site on Shopify", "which agencies' clients still use Universal Analytics"), you have three ways to get that data in 2026. They differ in price by a factor of about 100, and most people only know the expensive two.

Option 1: BuiltWith

The heavyweight. Its real product is historical data: what a site ran three years ago, when it switched, technology trends across the whole web. If you need that, nothing else comes close. Plans commonly run hundreds of dollars per month, and for trend analysis it earns it.

But most sales and SEO use cases don't need history. They need "what does this list of 2,000 domains run today", and paying a monthly subscription for that is renting a combine harvester to mow a lawn.

Option 2: Wappalyzer

The origin of most detection tech. The browser extension is great for spot checks on a single site. The API and bulk products moved to paid plans years ago, and the open-source fingerprint dataset the community built lives on in maintained forks.

Option 3: pay-per-result detection

The newer model: no subscription, you pay per site checked, typically around half a cent. The engines run the same class of open-source fingerprints (7,500+ technologies: CMS, ecommerce, analytics, frameworks, CDNs, with versions and confidence scores), so detection quality on current-state checks is comparable. What you give up is history; what you get is a bill that matches your actual usage.

Worked example with real numbers: a 1,000-domain prospect list costs about $5 to enrich with full tech stacks at pay-per-result pricing. The same job through incumbent per-result pricing on marketplace tools ranges from $70 to $100, and through a subscription product it costs whatever your monthly plan costs whether you run the list or not.

When each makes sense

Historical trends and web-wide statistics: BuiltWith, no contest. One-off manual checks while browsing: the Wappalyzer extension, free. Bulk current-state enrichment for lead-gen, SEO audits, or competitive research: pay-per-result is the price-performance winner, and it fits AI agents too, since MCP-compatible tools let an agent enrich a list mid-workflow and the bill lands in cents.

Full disclosure: I built one of the pay-per-result options, Website Technology Detector (about $5 per 1,000 sites, MCP-ready, live example output on the listing). If your use case is the third one, it will save you real money; if it's the first, buy BuiltWith and don't look back.

The PDF Function Is the Easy Part: Designing a Reliable Document Step

Howth Technology Factory — Fri, 24 Jul 2026 14:06:41 +0000

Most PDF automation starts with a verb: merge, split, compress, rotate.

The code for that verb may be a library call. The production work begins around it.

A dependable document step has to answer questions the PDF function cannot:

Where may the input come from?
How do we prove that a URL returned a PDF rather than an HTML error page?
What does a page range mean, and what happens when it is invalid?
Where is the result stored?
How long does it remain there?
Which failures are safe to retry?

This article builds a small operational contract for that step. The examples use hosted PDF tools, but the design applies equally to a local library, a containerized CLI or an internal service.

1. Treat retrieval as part of the operation

If an API accepts a URL, it is also a network client. That makes fetching part of the product surface.

At minimum, decide:

whether only HTTPS is accepted;
whether redirects are followed;
which hosts or address ranges are allowed;
the maximum download time and byte count;
whether authenticated or private documents are supported;
what happens when the response is not actually a PDF.

An unrestricted URL field can become a server-side request forgery risk. A private-document URL can leak through logs. A link that works in a browser may return an HTML viewer rather than the PDF bytes.

For sensitive workflows, a pre-signed object-store URL with a short lifetime is usually easier to reason about than a public permanent link. If the tool only supports public URLs, treat that as an architectural constraint—not a footnote.

2. Make one action explicit

Avoid an input that asks the processor to infer what should happen. The caller should choose a narrow action and provide action-specific options.

{
  "action": "merge",
  "files": [
    "https://example.com/report.pdf",
    "https://example.com/appendix.pdf"
  ],
  "options": {}
}

That contract is useful because a workflow can validate it before it moves any bytes. It also gives an AI agent a bounded tool call instead of arbitrary code execution.

The same principle applies to page ranges. Pick one documented syntax, define whether pages are one-based or zero-based, reject overlapping or out-of-range input consistently, and return the normalized selection in the result if possible.

3. Define what “compress” means

Compression is not one operation.

Structural compression removes redundant objects and rewrites streams without deliberately reducing visual quality. Image recompression may lower JPEG quality or resolution. Font subsetting, object cleanup and linearization affect different aspects of the file.

A “high compression” setting is meaningless unless the documentation explains the trade-off. For image-heavy scans, lossless structural compression may barely change the size. For an already optimized PDF, any further reduction may require visible degradation.

Record both input and output sizes. If the tool changes image quality, say so. If it falls back to a different strategy when a dependency is unavailable, expose that fact or at least document it.

4. Return an inspectable result

A binary file is necessary, but it is not a useful workflow response on its own.

A structured result should include:

{
  "action": "merge",
  "inputCount": 2,
  "outputUrls": ["https://storage.example/result.pdf"],
  "sizeBefore": 842190,
  "sizeAfter": 837421,
  "ms": 913
}

Those fields let the next step check that an output exists, compare its size, record processing time and decide whether a retry or human review is appropriate.

Use stable error categories as well: invalid input, unreachable source, corrupt PDF, resource limit, processing failure and result too large are materially different states.

5. Decide where bytes live

“We do not keep your data” is not a storage model.

Document:

whether processing is in memory or on disk;
whether inputs or outputs are written to platform storage;
the retention policy governing that storage;
whether logs include URLs or filenames;
who can retrieve the result;
how a caller can delete it early.

For regulated or confidential documents, a managed API may be the wrong boundary regardless of convenience. A local library or service inside the protected network may be the better design.

6. Give agents decisions, not bytes

An agent can be useful at the orchestration layer:

identify the approved source documents;
choose merge, split or compression;
construct a typed request;
inspect the structured result;
ask for review when the outcome is ambiguous.

The deterministic processor should manipulate the file. The agent should not improvise a binary transformation or receive broader filesystem access than the workflow needs.

Tool descriptions matter here. “Process a PDF” is a poor agent tool. “Split a PDF into these one-based page ranges; reject any range outside the document” is much easier to call safely.

7. A concrete comparison

Howth Technology Factory's PDF Toolkit provides a useful example because it has two related delivery forms.

The Apify Actor documents nine actions: merge, split, compress, rotate, delete pages, add a text watermark, render PDF pages as images, assemble images into a PDF, and render a URL as PDF. It accepts URL-based inputs and returns a dataset record plus files in the run's key-value store.

The MCPize package is not the same interface. It exposes five typed tools: merge, split, structural compression, PDF-to-text and images-to-PDF. Its listing states explicit per-input, per-request, result and text-extraction caps.

That difference is the lesson: select a document tool by its contract, limits and data path—not by the label “PDF toolkit”.

When a specialist service is the right choice

Basic page manipulation is only one part of the PDF world. OCR, certified redaction, digital signatures, form filling, PDF/A validation, accessibility remediation and complex layout extraction need specialist capabilities.

Enterprise document APIs and commercial SDKs can be justified when those requirements matter. Local open-source libraries make sense when data cannot leave your environment or when you need deep control. A hosted narrow tool is attractive when the work is routine, the data policy fits and maintaining the processing stack would be needless overhead.

The decision is not “API or library?” in the abstract. It is: which boundary makes this particular workflow reliable, observable and appropriately constrained?

Bulk-validating EU VAT numbers

Howth Technology Factory — Wed, 22 Jul 2026 13:41:00 +0000

If you invoice EU businesses with reverse-charge VAT, you already know you're supposed to validate your customers' VAT numbers. What surprises most finance teams is the gap between "we checked it" and "we can prove we checked it", and that gap only becomes visible during a tax inspection.

What VIES gives you for free

VIES is the European Commission's official validation service. Paste in a VAT number, it tells you valid or invalid, and for most member states returns the registered business name and address. It's free, it's official, and for one-off checks it's all you need.

The three problems at any real volume

First, there's no bulk mode. Validating a 500-customer ledger means 500 manual checks or writing code against the API.

Second, it falls over constantly. Individual member states go offline for hours ("MS_UNAVAILABLE" is a rite of passage), and the service rate-limits aggressively. Naive scripts that hammer it with 20 parallel requests get error codes and, eventually, IP blocks. A batch that dies at number 347 of 500 with no record of which succeeded is worse than no batch at all.

Third, and least known: the audit evidence. When you supply YOUR VAT number along with the one you're checking, VIES returns a consultation number, an official identifier proving that check happened on that date. Under the rules of several member states, that consultation number is the evidence a tax authority expects when you justify zero-rating a cross-border sale to a customer whose VAT number later turns out dead. A screenshot of a green tick carries a lot less weight.

What a sane bulk setup looks like

Whatever tool you use (including your own script), it should: accept messy input formats and normalise them, keep concurrency low (around 3) with backoff retries so member-state downtime and rate limits don't kill the batch, output one row per input including failures so reconciliation is possible, and request consultation numbers on every check so the audit trail builds itself.

Disclosure: I built a tool that does exactly this pattern, EU VAT Validator (VIES), about $5 per 1,000 validations, no subscription, MCP-ready so finance automations and AI agents can call it directly. But the pattern matters more than the product: if your current process can't show a consultation number per customer per period, that's the gap to close before an inspector finds it for you.

(Not tax advice; check your member state's specific evidence requirements with your advisor.)

We scored major websites on AI readiness. A national newspaper got 33/100.

Howth Technology Factory — Mon, 20 Jul 2026 11:10:25 +0000

AI assistants are becoming a referral channel whether sites prepare for it or not. People ask ChatGPT, Claude and Perplexity for recommendations, and those assistants read the web to answer. Which raises a question almost nobody is checking: can they actually read YOUR site?

There are three mundane, checkable things that decide it.

The three checks

llms.txt. A plain-text file at yoursite.com/llms.txt that tells AI systems what your site is and where the important content lives, like robots.txt but for guidance instead of prohibition. It went from proposal to quiet standard fast: in our scans this month, Shopify, Stripe's docs, Anthropic's docs and WordPress.org all serve one.

AI crawler access in robots.txt. GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended, CCBot: each can be allowed or blocked in robots.txt, and many sites block them without remembering they did (often a blanket rule added in 2023). Blocking is a legitimate choice for some businesses. Blocking accidentally while your marketing team invests in "AI visibility" is just self-harm.

A sitemap. Boring, decades old, still how any crawler, AI or classic, discovers your content systematically.

What we found

We ran a scored audit (0 to 100 across those checks) on a mix of docs sites and publishers. WordPress.org scored 88: llms.txt present and well-formed, every major AI crawler explicitly allowed, sitemap in place. Stripe's and Anthropic's docs scored 75 and 85.

The Irish Times scored 33: no llms.txt, no llms-full.txt, and GPTBot explicitly blocked in robots.txt. A national newspaper, invisible by choice or accident to the fastest-growing answer engine there is. (For a publisher wanting licensing leverage, that may be strategy. If so, the llms.txt file stating terms is still missing.)

Check your own site in 30 seconds

Manually: visit yoursite.com/llms.txt, yoursite.com/robots.txt (search it for "GPTBot" and "ClaudeBot"), and yoursite.com/sitemap.xml. That's the whole audit for one site.

At scale (client portfolios, prospect lists, agencies selling AI-visibility work): I built llms.txt Checker, which runs all the checks on a list of URLs and returns the 0-100 score with a plain-English fix list, about $5 per 1,000 sites. The example run on the listing shows the exact newspaper result above.

The fixes are usually a day's work: write an llms.txt, audit the robots.txt rules against actual intent, confirm the sitemap. Cheap insurance on a channel that's only growing.