DEV Community: Nova Solutions

The AI data economy just started paying small businesses too

Nova Solutions — Sun, 27 Sep 2026 02:06:44 +0000

Reddit, Shutterstock, and the Associated Press have already sold data to AI labs. Less visible: the same demand for real operational data is now reaching small businesses, in deals structured very differently from a publisher license.

The deals that already made the news

Over the past two years, several large publishers signed licensing deals with AI labs to let their content train models. Reddit struck an agreement with Google. Shutterstock has licensed image and caption data to multiple AI companies. The Associated Press signed a deal with OpenAI. None of the exact terms are fully public, but the pattern is consistent: a company that already has a large, structured archive of content licenses a copy of it, non-exclusively, for training use.

What's less visible is that the same underlying demand doesn't stop at publishers. Model developers need more than clean, professionally written text. They need messy, realistic examples of how work actually gets done: how a dispatcher schedules a service call, how a sales rep answers an unusual customer question, how an ops team handles an edge case a policy document never anticipated. That kind of data mostly lives inside small and mid-size businesses, not media companies.

Two real deals, further down market

Two licensing deals closed in this smaller, less visible tier of the market. Names are withheld below, both by request and because the terms one small business gets don't depend on the industry press knowing whose CRM it was.

Seller	Deal size	Terms	Time to close
A business with roughly $3 million in annual revenue	$100,000, paid upfront	Non-exclusive, de-identified license	3 business days
A manufacturer with fewer than 100 employees	Comparable terms	Non-exclusive, de-identified license	Similarly fast

Two closed data-licensing deals

Both deals were arranged independently by a data-licensing partner, not by Nova Solutions. We're describing them because they're real, verifiable proof that this category exists and moves fast, not because we closed them ourselves.

What “non-exclusive” and “de-identified” actually mean here

Non-exclusive: the seller can still license the same underlying data to a different buyer later. Nothing about the deal locks the data to one buyer forever.
De-identified: the buyer receives a processed copy with names, account numbers, and other identifying details stripped or masked before it ever leaves the seller's systems.
The seller's own systems and originals are untouched. Nothing about how the business operates changes after the deal closes.
Price and buyer are approved by the seller before anything moves. There's no obligation to accept an offer that comes in.

That structure is why the deal above closed in three business days: there was no negotiation over exclusivity, no migration of live systems, and no ongoing operational change to review. A one-time, de-identified export is a much smaller decision than it sounds.

Why this is happening now, not five years ago

Two things changed at once. First, the supply of freely usable, high-quality text on the open web is thinner than it was: a lot of it is now paywalled, licensed, or contested in litigation, and labs that scraped without a license are facing lawsuits over it. Second, the models themselves have gotten good enough at general language that the next gains come from realistic, domain-specific examples, not more generic text. Ordinary operational data, the kind that sits in a CRM or a dispatch log and was never written for an audience, is exactly that kind of example.

That's a different pitch than “your data is valuable,” which every business has been told for a decade. What changed is that there's now a real, structured buyer for a specific, narrow slice of it, at a price and speed that make it worth a business owner's actual attention.

What this means if you're the seller

If you're evaluating whether this applies to your own business, the two deals above suggest a rough shape: the categories most likely to have licensable value right now are manufacturing, freight and logistics, accounting, finance, insurance, distribution, construction, professional services, and retail, businesses where operational records reflect judgment calls a model can learn from. Healthcare and legal records are currently held out of this category entirely, pending buyer-side legal clearance on protected information.

We set up data licensing partnerships for businesses in the cleared categories above: preparing a de-identified sample, taking it to buyers, and letting the business approve the price and buyer before anything moves. If that's relevant to your business, it's worth a conversation even if nothing closes.

Questions this post answers

What does “non-exclusive” mean in a data-licensing deal like this?

It means the seller isn't locked into one buyer. The same underlying data could theoretically be licensed to a second buyer later, since the first deal doesn't grant exclusive rights to it.

Why would an AI lab pay for a small business's CRM data instead of just using public web data?

Public web text is thinner than it used to be, partly paywalled and partly contested in litigation over unlicensed scraping, and models have gotten good enough that the next gains come from realistic, domain-specific examples rather than more generic text. Ordinary operational records like CRM histories or dispatch logs are exactly that kind of example, and they mostly don't exist anywhere on the open web.

Who actually arranged the two deals described here?

Both were arranged independently by a data-licensing partner, not by Nova Solutions. They're described here as real, verifiable evidence that this category of deal exists and moves quickly, not as our own track record.

Originally published at www.advai.cloud/blog/ai-labs-licensing-small-business-data.

What is an AI OS? A practical definition

Nova Solutions — Fri, 25 Sep 2026 06:25:02 +0000

An AI OS is not a chatbot and not an agent framework. It is the connective layer between the messy state of your real business and the AI workflows you want to run on top of it.

The gap an AI OS fills

A common pattern among mid-size Philippine companies is saying, in the same breath, "we already use AI" and "we have no idea what our customers are actually doing." That is the gap an AI OS fills. It is not a chatbot. It is not even an agent framework. It is the connective layer between the messy state of your real business and the AI workflows you want to run on top of it.

This is the working definition I wish someone had handed me when I first sat down to design Nova AIOS. We will cover what an AI OS actually is, why Palantir Foundry is the canonical example, where the term breaks down, and what to look for if you are evaluating one for a PH or SEA enterprise.

A working definition

An AI OS, sometimes written AI Operating System or AIOS, is the layer of software that does four things.

Models the real-world things your business cares about (customers, loans, properties, students, transactions, equipment) as typed objects with relationships.
Provides a runtime for AI agents to read, reason about, and write to those objects.
Enforces a permission model deciding which agent, on whose behalf, can touch which object in what way.
Logs every action with enough fidelity to pass an audit, a regulator check, or a post-incident review.

Strip away the marketing and that is the whole shape. An ontology of business objects. A runtime that executes agents against them. A permission and audit layer that makes the result usable in regulated industries. If a vendor selling you an AI OS cannot show you, on a whiteboard, where each of those four pieces lives, you are buying a chatbot.

Note

An AI OS knows the state of your business. A chatbot only knows the sentence you just typed.

Why the term started showing up

Three things happened at roughly the same time in 2023 and 2024 that made AI OS a useful phrase. First, language models stopped being parlor tricks and started being usable as workflow engines. You could give a model a complex multi-step task involving structured data and it would actually finish.

Second, enterprises looked at the resulting work and realized something uncomfortable. Every team was building the same plumbing. Each new AI workflow needed its own data pipe, its own retrieval layer, its own observability, its own permission model. Twelve workflows meant twelve duplicated stacks of infrastructure.

Third, Palantir Foundry had been quietly proving for a decade that you could build the plumbing once, model your business objects once, and ride that foundation through hundreds of downstream AI use cases. The model worked. The price tag was usually north of $500,000 USD per year, which kept it inside the Fortune 100 and the US Department of Defense. The phrase AI OS emerged as shorthand for the Foundry shape, available to people who are not the Pentagon. That is what almost every credible vendor in this category is building today, including us.

The four layers, with examples

Layer 1, the ontology, is the typed model of your business. For a microfinance institution the core objects are Borrower (name, contact methods, KYC documents, credit history, language preference), Loan (principal, balance, schedule, status, arrears state, with a relationship back to Borrower), Payment (amount, channel, date, applied-to-loan reference), and CollectionsAction (channel, script used, outcome, timestamp, plus a compliance flag tied to RA 11765 and the SEC MC 18 collection-conduct rules).

Notice what is missing. There is no "chat message" object and no "document" object. The ontology is the things your business cares about, not the things your AI happens to produce. Documents and transcripts are inputs and outputs, not first-class objects. This sounds pedantic. It is the most important call you will make. Get it wrong and every downstream agent reasons about chat transcripts instead of loans, which means it can only make conversational decisions, not business ones.

Layer 2, the agent runtime, is where AI workflows live. It needs to read the ontology fully typed and fast (every loan and payment for a borrower in under 200 milliseconds), write to the ontology transactionally, and compose agents. A collections agent does not exist alone. It calls a payment-status agent, which calls a credit-history agent, which calls a sentiment classifier on the last three messages. The runtime decides what runs in what order and what each step is allowed to see.

Layer 3, the permission model, is not a feature. It is the difference between a tool you can ship to production and a science project. Can the collections agent see the borrower's health-related KYC notes? No. Can it see the outstanding balance? Yes. Can a customer-facing chatbot create a Payment object directly, or only suggest one for human approval? Depends on amount and channel. Can a marketing agent send promotional SMS to a borrower flagged as in arrears and sensitive? No, and the system should refuse to compose that message at all. This is the layer where RA 10173, RA 11765, and the BSP circulars live.

Layer 4, audit and observability, logs every agent action, every read, every write, every refused permission. Not in the casual "we have logs somewhere" sense. In the sense that a BSP examiner can sit down and reconstruct any decision the system made in the last 18 months. This is the layer most pre-2024 AI products skipped, and the reason most never made it past pilot in a regulated industry.

AI OS versus chatbot versus agent framework

Three things people sometimes call an AI OS that are not one.

A chatbot is a stateless conversational interface. It knows the current message and maybe a short rolling history. It does not know your business objects and cannot do an audited write to a loan record. It sits on top of an AI OS at best.
An agent framework like LangChain, LangGraph, or CrewAI is a useful tool for composing agents, but it gives you layer two and partial layer four while leaving layers one and three to you. Building an AI OS on top of one is reasonable. Calling the framework itself an AI OS is a category error.
A RAG pipeline grounds a model in your documents. It is not an AI OS for the same reason a document object is not a business object. RAG retrieves text; an AI OS reasons about typed state.

When you should adopt one, and when not

The honest test: if the same underlying business object will be touched by more than one AI workflow over the next 18 months, you need an AI OS. If only one workflow is on the roadmap, buy the single tool.

A 200-loan microfinance startup that wants AI collections, AI loan screening, and a customer-facing FAQ chatbot needs an AI OS. Same borrower, same loan, three workflows. Without a shared ontology, each workflow grows its own borrower model and they drift. An HVAC contractor with one phone line that misses after-hours calls does not need an AI OS. They need a missed-call text-back automation. Buy the single tool, and come back when you have three more workflows that need to share customer state.

How to evaluate an AI OS vendor

Five questions to ask before you sign anything.

Can you see the ontology in a UI, in plain language, before any agent is built? If the answer is "the ontology emerges from your data," walk away. A real AI OS makes the ontology explicit and editable.
Show me an audit log entry from a real customer, with PII redacted. If they cannot produce one in under five minutes, they have not shipped to a regulated customer.
What happens when an agent tries something the permission model forbids? You want a loud, audited refusal with an explanation. Not a silent skip. Not a hallucinated success.
Who owns the data residency? In the Philippines, "our data is in AWS Singapore" is sometimes acceptable and sometimes not, depending on the regulator. The vendor should know the difference for your industry.
What is the actual price after the first 90 days? Discovery and setup are usually quoted at a loss. The retainer for ongoing operations is where the cost lives. Ask for it in writing.

What an AI OS costs in the Philippines and SEA

Round numbers, as of mid-2026.

Foundry-tier US vendor: $500,000 USD per year and up. Includes professional services, but you are paying for the brand and the certification stack.
Mid-market US vendor: $80,000 to $250,000 USD per year. Usually a thinner version of the four-layer pattern with one or two layers stripped down.
Nova AIOS build: ₱150,000 for a single-product deployment, scaling to about ₱1.2M for a multi-product enterprise rollout, plus a monthly operations retainer typically between ₱50,000 and ₱200,000.
Build it yourself: free in cash, expensive in time. Twelve to eighteen months of senior engineering to reach a usable runtime. Often right for large banks. Almost never right for anyone below 500 employees.

The order-of-magnitude gap between US vendors and Philippine-built systems is not a quality gap. It is an engineering-economics gap. A senior Filipino ML engineer at a Philippine company costs about one-fifth of a senior ML engineer at a US enterprise software company. That ratio shows up in the price.

The mistake most teams make

They start with the agent. Someone reads a blog post about a clever agent, copies the pattern, gets a demo working in a week, then spends six months trying to plug it into their actual business. The agent works in isolation. Connecting it to real customer data, real permission rules, and real audit requirements is where the project dies.

The teams that ship to production start with the ontology. They map their Borrower, Loan, and Payment objects on a whiteboard, argue about field names for two days, and only then build the agent that reasons over them. The first version is dumber than the demo. The production version, three months later, is more useful by an order of magnitude. The ontology-first move is the single biggest predictor of whether an AI OS deployment makes it to production. If your vendor is not making you do the ontology work first, find a new vendor.

Originally published at www.advai.cloud/blog/what-is-an-ai-os.

We almost let our own automation cold-email its own founder

Nova Solutions — Fri, 25 Sep 2026 06:16:16 +0000

The fix was supposed to be a one-line merge. Looking at the actual rows before running it found a FOIA desk, two charities, and the founder's own inbox sitting in the send queue.

The plan was to merge the backlog forward

We were in the middle of repairing a broken cold-email queue on our own outreach pipeline. Part of the fix meant a backlog of contacts, stranded by the earlier bug, needed to rejoin the working pipeline. The plan, as written, was simple: merge the stranded backlog forward.

A merge like that is easy to trust. The rows had been collected before, the fields lined up, and the fix elsewhere was already done. Nothing about the plan looked risky on paper: it was a data operation, not new code, and data operations tend to get less scrutiny than the code around them.

The tempting move was to run it and move on. We did not do that. Before merging anything, we opened the actual rows and read them, the same way we would read the diff on a code change before shipping it.

What was actually sitting in the backlog

The list was not what a clean prospect backlog should look like. It contained a federal .gov address belonging to a FOIA desk, two charity addresses, an immigration legal-aid nonprofit, a support inbox for a hosting platform, and a placeholder address left behind by a website-builder tool.

The row that stopped us

One row in the backlog resolved to the founder's own email address. Had the merge run as planned, our own cold-email automation would have sent a prospecting email to its own founder.

None of these rows were ever real prospects. They were the accumulated residue of however the original list had been built over time: a scrape, a form submission, a support ticket, an address typed into the wrong field once and never corrected. A list assembled this way does not announce which rows are bad. It just sits there looking like the rest of the data, until someone reads it closely enough to notice.

A merge treats every row the same. It has no concept of "this one is a government agency" or "this one is us." Only a human looking at the actual data notices that some of those rows do not belong in an outreach queue at all, which is exactly why the step of looking cannot be automated away along with everything else.

A permanent filter, then the real migration

Instead of merging the backlog as it was, we built a junk filter first: rules to catch government and military domains, our own company's own domains, known platform-support domains, and common nonprofit signals. Only after the filter existed did we run the actual migration.

Backlog rows	Junk (filtered out)	Already known	Clean, migrated
163	7	33	123

The 2026-07-29 backlog migration, after the junk filter was in place.

Seven of 163 rows were junk by the filter's own rules. Thirty-three were contacts already known elsewhere in the pipeline. The remaining 123 were genuinely clean and migrated forward. The filter did not just catch the founder's own address. It caught six other rows an automation should never have been allowed to touch on its own.

The cheaper version of the same habit: look before you act

Reading 163 rows by hand does not scale, and it was never meant to be the permanent answer. The filter is the durable version of the same instinct: instead of a human reading every row every time, a fixed set of rules reads them automatically and refuses to pass through anything that matches a known-bad pattern, before a single message goes out.

The same habit shows up in a plainer form as a dry run: a mode where an automation reports exactly what it would do, to whom, and how many times, without actually doing it. A dry run over this backlog would have printed the founder's own address as a send target in plain text, which is a much cheaper way to catch the problem than finding out after the fact. Any automation that touches real addresses, real accounts, or real money is worth running once in a mode that only reports, before it is ever allowed to act.

Guardrails matter as much as retries

Most of the engineering attention that goes into an automation goes into making it run reliably: retries, error handling, logging, alerts when something fails. All of that matters. None of it would have caught this. The problem here was not that the automation broke. It was that it was about to work exactly as designed, on data nobody had actually looked at.

An automation inherits every defect already present in its inputs. "The list", whatever list it is, almost always contains something you would be embarrassed to act on automatically, because it was assembled over time by processes that were never built to guarantee it was clean. A gate that checks who an automation is about to touch, before it touches them, is not optional polish. It is as load-bearing as the retry logic everyone remembers to build.

We keep this filter in place permanently now, on every list that feeds this pipeline, not just the one that happened to catch a founder's own inbox. The cost is a few seconds of automated checking before anything sends. The alternative, on a bad day, is an automation doing exactly what it was told to do, to exactly the wrong address.

This is the same standard we hold any automation to, ours or one we build for someone else: before it runs unattended at volume, someone has actually looked at what it would touch, and a permanent check stands in that person's place going forward. A list is never as clean as it looks from the outside, and the automation running against it will not notice the difference on its own.

Questions this post answers

Why does an automated contact list need a human to look at it before it runs?

Because an automation inherits every defect already sitting in its input, and a list built up over time will always contain rows you would be embarrassed to act on automatically. In our case that meant a government address, charity contacts, and even the founder's own email, all queued for the same automated send.

What kind of "junk" ends up in a cold-outreach list?

Addresses that were never real prospects to begin with: government and institutional domains, nonprofit and charity contacts, platform support inboxes, placeholder addresses left over from tools like website builders, and occasionally an internal address that got swept in by accident.

What does a junk filter for an outreach automation actually check?

Ours checks for gov and mil domains, our own company domains, known platform-support domains, and common nonprofit signals, and excludes any row that matches before it ever reaches the send queue. It runs before volume, not after a complaint.

Does inspecting the data before automating it slow things down?

It costs the time it takes to look. In our case that was one migration pass across 163 rows before anything moved. A dry run that prints what an automation would do, without doing it, costs the same small amount of time and catches the same problems before they become an actual send.

Originally published at www.advai.cloud/blog/the-junk-filter-that-caught-us-cold-emailing-our-founder.

OCR document extraction for Philippine banks: what actually works

Nova Solutions — Sat, 19 Sep 2026 11:04:53 +0000

A BIR Form 2316 printed by payroll software and a Torrens title issued in 1974 are not the same kind of document. Treating them as one is why production accuracy collapses.

Two documents, two different problems

A BIR Form 2316 printed by payroll software and a Torrens title issued in 1974 are not the same kind of document. Not the same OCR accuracy profile, not the same preprocessing pipeline, not the same confidence threshold before the data lands in a KYC record.

The common pattern in Philippine bank OCR builds looks the same almost everywhere: scan or upload, run extraction, get a result. Demo accuracy looks fine, 88, 91, sometimes 94 percent. Then the system goes live. Failure patterns emerge that are specific to Philippine document types, completely predictable, and fixable, but only if you understand the problem before you build the pipeline.

The three-tier problem

Before you write a line of OCR code, know this: your corpus splits into three tiers that need completely different treatment.

Tier 1, machine-generated structured documents. BIR Form 2316, payslips from payroll software, system-generated PhilHealth MDR forms, GSIS UMID cards, passport MRZ zones. Fixed layout, printed text, consistent fonts, predictable field positions. With off-the-shelf tools you get 92 to 96 percent field-level accuracy on clean copies. "Clean copy" is doing a lot of work there: a 2316 folded three times, stapled to a payslip, and run through a flatbed scanner at 150 DPI is no longer clean. But for standard machine-generated versions, these are the most reliable documents in the stack. Build your happy path here.

Tier 2, scanned handwritten forms. Loan applications from rural branches, income declarations filled by hand. Accuracy drops hard, 60 to 75 percent character-level without preprocessing. That sounds acceptable until you realize a 75 percent accurate extraction of a bank account number produces a garbage number that looks real. Invisible error, catastrophic record. Preprocessing, binarization, noise removal, deskew, handwriting-specific model selection, can push this to 82 to 88 percent on clean handwritten documents. You will never reach Tier 1 accuracy, which means Tier 2 needs a different verification architecture regardless of model quality.

Tier 3, phone-photographed documents, where the most aggressive failure happens, and where the most volume is heading as banks add mobile onboarding. A phone photo of a Torrens title taken at an angle with a flash and a shadow across the bottom third: 40 percent accuracy baseline, maybe. Required preprocessing before any OCR: perspective correction, glare and shadow removal, contrast normalization, and resolution assessment with automatic rejection below 200 DPI equivalent. After that, accuracy on a clean government ID recovers to 78 to 88 percent. Still not Tier 1. Good enough for a first pass if you have human review flagging anything below threshold. Most deployments run the same model and threshold whether the input is a crisp 2316 or a phone photo of a 1982 title taken in Leyte. The demo number masks real production accuracy, which for banks with rural customers is often below 70 percent.

What works well

Philippine passports. The machine-readable zone is essentially 100 percent reliable with any decent OCR library. MRZ was engineered for automated reading. If your pipeline takes passport data, start there and treat it as ground truth.
GSIS UMID cards. Consistent layout, machine-printed text, modern card stock. 90 to 95 percent field accuracy on clean captures, handled well out of the box.
PhilHealth ID (post-2015 format). Same story as UMID. Fixed layout, machine-generated, high reliability.
BIR Form 2316 (machine-generated copy). Once you have trained a layout model on the current template, fixed-position fields like gross compensation, tax withheld, and employer TIN extract at 92 to 96 percent. The highest-volume KYC document for employed borrowers and the clearest automation ROI in the stack.
SSS ID (post-2019 biometric format). Reliable. Pre-2019 printed-card formats are Tier 2.

What does not work, and why

These are the documents where vendors demo beautifully on curated samples and fail embarrassingly in your production queue. Handwritten loan application forms from rural branches: the problem is not just handwriting, it is that the forms vary between branches and years, with different field positions and handwriting from neat block letters to barely legible cursive. Photocopied Torrens titles with carbon copy layers: multiple generations of photocopying pile artifacts onto the text until characters are partially overwritten by scan noise. PSA birth certificates in older formats use typewriter fonts with varying ink density and faded toner, and the layout changed several times across decades. And a BIR 2316 the moment a human hand touches it drops from Tier 1 to Tier 2. No exceptions.

Torrens titles: their own category

Torrens title extraction is a common place for vendor demos to look credible and then produce complete garbage in production. Torrens titles accumulate history physically. A title from 1965 may have handwritten margin annotations noting easements. The original TCT number may have been crossed out and replaced as lot partitioning occurred over decades. Stamps in Tagalog record registration. Owner names appear in Spanish-era conventions mixed with modern format. Carbon copy layers create ghost text sitting directly on top of primary content. That is all before physical condition: humidity damage, fold lines, edge fraying, faded ink. No off-the-shelf model was trained on this.

Preprocessing first. Deskew to correct rotational distortion, denoise to reduce scan artifacts, normalize contrast to pull faded text out of the background, and for carbon copy artifacts, a descreen filter to remove the moiré pattern from scanning a copy of a multipart form.
Multi-zone parsing. The primary record area (TCT number, registered owner, lot and technical description) is structurally different from the margin annotations and the stamp area. A single unified model fails on all three. You need a zone detector that identifies which region it is reading, then applies the appropriate logic per zone.
Human review for anything issued before 1985. Not a crutch, an architectural requirement. Quality variance for pre-1985 titles is too high to accept machine extraction without review. Getting a collateral document wrong in a loan file is not recoverable with a model update. Build the review queue from day one.

Note

Ask any vendor demoing Torrens title OCR: what was the issuance year range of the titles in your test set, and what was your scan-quality selection criterion? If they cannot answer precisely, the accuracy number they are quoting is fiction.

BSP KYC compliance and the audit trail

This is where most builds miss something that matters more than accuracy rates. BSP's KYC framework requires that customer identification and verification be auditable: you must demonstrate what data was collected, how it was verified, and who was responsible. For an automated pipeline that translates into three things you must build.

A confidence threshold, not just a confidence score. Every extraction produces a score; most teams look at it in logs. You need a hard threshold below which the extraction does not pass to the KYC record automatically and routes to human review. The threshold depends on document type and risk tolerance, but it must exist and be documented.
A review queue integrated into the same application. The KYC officer sees extracted text alongside the original image, corrects errors, and confirms. Built into the runtime, not a spreadsheet or an email chain. Same application, same session, same record context, with the correction tracked as a discrete action by a named user.
An immutable audit log. Every document gets a record: original extraction result, confidence score, whether it was auto-approved or reviewed, reviewer identity and timestamp, and any corrections. Append-only. BSP examiners will want to see this. "The system extracts it" is not an audit trail.

It is common to find OCR implementations with none of these three elements, even when the underlying accuracy metrics look good internally. An implementation like that would fail a compliance audit on the audit trail question alone.

The practical KYC stack for mid-size banks

If you are a thrift, rural, or savings bank rather than BPI, BDO, or UnionBank scale, you do not need an enterprise OCR platform. You need something proportionate. Use a managed document AI service for structured government forms: 2316, PhilHealth ID, UMID, SSS ID, passports. High-volume, high-reliability, pre-built processors, low per-page cost. You are calling an API, not maintaining a model.

Use a custom layout parser for Torrens titles and pre-standardization documents. This is where you need local development, not because the engineering is hard but because training data for Philippine Torrens titles in various conditions does not exist inside any US vendor's model. You need a team with access to real samples who can label them and understands the physical history of what they are looking at. Add a human review queue in the same web application: reviewer sees the image and extracted fields side by side, corrects, confirms, and confirmation writes to the audit log automatically.

This stack handles 85 to 90 percent of KYC document volume automatically at most mid-size banks. The remaining 10 to 15 percent, complex titles, handwritten forms, damaged documents, routes to review without blocking the pipeline. A full enterprise OCR platform typically costs $40,000 to $150,000 annually for a mid-size deployment plus implementation. The managed-service-plus-custom-parser-plus-review-queue approach costs $800 to $3,000 per month in API fees plus a one-time build of ₱350,000 to ₱700,000 for the custom Torrens parser and review interface. For a bank processing 500 to 2,000 KYC documents per month, the pragmatic stack is both cheaper and better suited to the actual document mix.

Test set requirements before you go live

Your pre-launch test set must come from your actual production corpus, not a curated demo set. That means titles from the 1960s and 1970s, not just recent ones. Handwritten loan forms from your rural branches, not just clean printed versions. Phone photographs taken under realistic branch lighting, not studio-quality captures. SSS IDs from 2008, not just 2024. Run your accuracy metrics on those. If your vendor's numbers collapse on real samples, you have found your actual production accuracy before it finds you.

Test the failure modes explicitly too. What happens when a document is rotated 15 degrees? When a title has a hand-drawn border around a corrected field? When someone submits a screenshot of a PDF instead of the original scan? Your pipeline's behavior on bad input is as important as its accuracy on good input.

Originally published at www.advai.cloud/blog/ocr-document-extraction-philippine-banks.

We audited our own indexing last month. We missed two of our own product pages.

Nova Solutions — Sat, 19 Sep 2026 11:04:10 +0000

Publishing a fix and confirming it worked are two different steps. Last month we only did the first one.

We went back to check our own fix, not to look for something new

Last month we wrote about a sitemap that gave a search engine no reason to recrawl 88 of our 103 pages, fixed it, and published the diagnosis. That post shipped, the fix shipped, and the natural next move is to consider the problem closed and turn to something else.

We went back instead. Not because anything looked wrong, but because a fix that looks complete and a fix that has actually been verified against the live site are two different claims, and we had only made the first one. Opening the exact same Search Console report a month later, expecting a quiet confirmation, surfaced three separate problems the original pass never caught.

A second, older version of the redirect bug we thought we had already fixed

The August fix redirected a retired network of URLs, one folder deep under a /local/ path, to their current equivalents. It worked, and a spot check confirmed it. What that spot check did not surface was an even older version of the same URL pattern, one folder shallower, predating the one we had already caught, covering the same trades and the same cities.

The trap in checking your own redirect

A plain status check on these older URLs returned 308, a redirect, which reads as fixed. It was not. Following that redirect all the way through, rather than trusting the first response code, showed it landing on a page that itself returned 404. The first hop looked healthy. The destination did not exist.

The fix was mechanical once found: the same trade-to-page mapping already proven correct for the newer network, applied to the older one, verified end to end this time rather than by status code alone.

A page quietly declaring the wrong domain as its own canonical address

One of our own trade pages showed up flagged as crawled but not indexed. Inspecting it directly in Search Console showed why: as of its last crawl, the page declared its own canonical URL on our bare domain, while every other page on the site, and the domain's own redirect rules, treat the www address as canonical. A page contradicting the site's own stated structure is exactly the kind of signal that makes a crawler hesitate.

Checking the live page directly found the contradiction was already gone. Somewhere between that last crawl and today, the canonical tag had been corrected to match every other page on the site. Nothing needed fixing in the code. What was missing was simply a nudge back to the crawler that the page was worth looking at again.

The one that mattered most: two of our own product pages, never crawled

The largest section of that report is pages a search engine has discovered through our sitemap but never visited at all. Thirty-four of them, as of this check. Most were pages that made sense to still be waiting their turn. Two were not: our own pages for roofing and for restaurants, live, correctly built, linked internally, present in the sitemap since launch, and never crawled a single time.

Note

These are not new pages and this is not a new bug. It is the same root cause from last month's post, an authority and crawl-budget ceiling that has not moved, now fully counted rather than partially estimated.

We are not going to pretend a manual nudge to recrawl a handful of pages fixes that ceiling. It does not. What it does is keep the pages that are actually finished from sitting invisible while the underlying constraint, which is a separate and slower problem, gets worked on.

What we actually changed, and what we are not claiming

We are not claiming any of this moved our search position. Nothing here has had time to show that yet, and we said the same thing last month for the same honest reason: isolating the effect of one fix takes longer than a same-day check allows.

What changed is the process, not just the fix. Verifying that a fix landed is now its own explicit step, done by testing the live site directly rather than by trusting a report that may be describing a version of the site that no longer exists. A report that says something is broken is a claim about the past. The only way to know the present state is to go and look.

Questions this post answers

How do you verify that a Search Console fix actually worked?

By live-testing the actual current behavior of the site, not by trusting the report. Search Console's indexing data reflects the last time a crawler visited a URL, which can be weeks or months old. A page can already be fixed in production while the report still shows the old, broken state.

What is the difference between Discovered and Crawled, currently not indexed?

Discovered means a search engine knows a URL exists, usually from a sitemap, but has not visited it yet. Crawled means it visited and chose not to index what it found. Discovered is a queue problem. Crawled is closer to a quality or trust judgment, though it can also just mean the page changed since that one visit.

Can a real, correctly built page simply never get crawled?

Yes. A page can be live, linked internally, present in the sitemap, and completely correct, and still sit for months with zero crawls if a search engine has not allocated the budget to reach it. That allocation tracks a site's overall authority, not any single page's own quality.

Why does a canonical tag pointing at the wrong domain matter?

A canonical tag tells a search engine which URL is the authoritative version when more than one URL could serve the same content. If a page declares itself canonical at an address that immediately redirects elsewhere, that is a direct contradiction a crawler has to resolve, and the safest resolution for it is often to index neither version until the confusion clears.

Originally published at www.advai.cloud/blog/the-audit-we-should-have-run-the-first-time.

Silent automation failures: how four of our own systems broke while reporting success

Nova Solutions — Sun, 06 Sep 2026 10:29:14 +0000

Exit code zero means the process ran. It does not mean the work happened. We learned the difference four times in one week, on our own systems.

The embarrassing part first

Our stack runs on automations we built ourselves: prospect list refueling, cold email, social posting, backend health checks, a morning briefing. For months, the monitoring on all of it was me occasionally remembering to check whether something had happened lately. The audit behind this post started with me admitting, in writing, that the only way I found out about broken automations was going and looking.

We suspect most small-business automation is monitored the same way. Somebody wires a workflow, watches it succeed twice, and walks away. From that day on, the system reports whatever its happiest code path reports, and the owner finds out it died whenever they next happen to wonder about it.

Four failures, zero alerts

The 2026 audit of our own stack. Every row looked healthy from the outside.

System	Quiet for	What it reported	What was actually true
Cold email queue	20 days	Queue empty, clean exit	A half-finished config change pointed the list builder at a file the sender had retired. About 130 verified prospects piled up in a file nothing read.
LinkedIn posting	18 days	Offline, skipping run	The connectivity check ran under a Python install with no certificate bundle. Every HTTPS probe failed certificate verification and a catch-all handler relabeled it offline.
Backend failure alerts	7+ weeks	Nothing, by design	Slack alerting for scheduled-job failures sat behind an environment flag nobody had set. A second, independent mute flag was stacked on top of it.
Three scheduled scripts	Intermittent	Success, some days	Under the scheduler's minimal environment, plain python3 sometimes resolved to a different interpreter without file access. Same script, different interpreter, different day.

Every row shares the same shape. The process ran. The process exited cleanly. The process was wrong about the one thing it exists to do.

The LinkedIn failure deserves a closer look, because it was built with good intentions. After a real outage, we added a connectivity check so the poster would skip runs while the network was down. That check happened to execute under a Python installation with no certificate authority bundle wired in, so every single HTTPS probe failed certificate verification. A broad exception handler caught the error and reported the network as offline. The safety check could never once pass. A guard that cannot distinguish the network is down from I am misconfigured will spend its life reporting the flattering one.

The cold-email failure was even smaller: one line, half finished. The script that promotes verified prospects was updated to write to a new file one evening, and the sender that reads the queue was not updated to match. Both jobs kept running daily. Both kept exiting cleanly. One filled a file nothing read, the other read a file nothing filled and logged queue empty, for 20 days. Two healthy-looking automations, connected by a filename, doing nothing.

Tails lie

The intermittent failure was the most instructive. The last few log lines looked clean, because recent runs happened to succeed. Only reading the full log, about three thousand lines back, showed success and failure alternating for weeks, which ruled out every theory that started with it broke on some particular day. The tail of a log shows the weather. The whole log shows the climate.

Why silent is the default, not the exception

None of this is exotic, and none of it required unusual bad luck. Error handling gets written for the errors the author imagined, so the surprise arrives as a calm log line instead of a page. Schedulers know whether a process ran, not whether the work happened. Alert channels ship behind flags, and flags default to off. Each of these is a reasonable engineering choice on its own. Stacked together, they guarantee that the failure you did not imagine is exactly the one nobody hears about.

The deeper problem is that no errors and working are different claims. A sender with an empty queue has no errors. A poster that believes it is offline has no errors. A muted alert system has, by definition, no errors. If your definition of healthy is the absence of red, a system can be dead for a month and stay green the whole time. Ours did.

The proof-of-life rule we build to now

Everything we wire now has to carry a signal tied to the outcome, not the process. The specific fixes from the audit took an afternoon each. The rules they left behind:

Assert the outcome, not the exit code. The email sender now checks the delivery count it actually achieved against the queue it believed it had. Zero sends on a day the queue says otherwise is an alert, not a log line.
Alert on quiet, not just on error. If a system that normally does something daily does nothing for days, that silence should page a human, because silence is exactly what the worst failures produce.
Un-mute by default. Alert switches that need a flag turned on will spend part of their life off without anyone having decided that. We found two of those stacked on one system.
Pin the environment. Scheduled jobs get explicit interpreter paths and explicit certificate bundles. Whatever the environment resolves to is a different program on different days.
Read the whole log once. The tail shows the weather. The full log shows the climate.

What to ask about your own automations

If someone built automation for you, or you built it yourself, you can audit it without reading a line of code. Ask three questions of each automation you depend on:

What number should move when this works, and where do I see it? If the answer is a log file, that is a no.
Who finds out, and how fast, if that number stops moving?
When did a human last confirm the outcome end to end, instead of the dashboard saying fine?

When we build automations for a business now, the outcome signal and the alert wiring ship with the workflow, as part of the same job. Not because it is sophisticated. Because we watched four of our own systems fail quietly in one week, and quiet is the expensive kind of broken.

Questions this post answers

How do I know if an automation is actually running?

Do not trust the exit code or the dashboard. Tie the automation to a number that should move when it works, a send count, a row count, a posted item, and check that number against what the system believed it would do. If the number and the belief disagree, or the number stops moving on a normal day, it is broken no matter how clean the logs look.

Why do automations fail silently instead of showing an error?

Because most error handling only covers the failures the author imagined. Catch-all exception handlers convert surprises into calm log lines, schedulers only know whether a process ran, not whether the work happened, and alert channels are often muted behind configuration nobody set. Silence is the default failure mode of unattended systems.

What is a proof-of-life signal for an automation?

A signal tied to the outcome the automation exists to produce, asserted every run and alarmed on absence: a delivery count that matches the queue, a heartbeat that fires only after real work completes, an alert when a normally daily system does nothing for days. Exit code zero is not proof of life; it only proves the process ran.

Originally published at www.advai.cloud/blog/silent-automation-failures.

We shipped a month of SEO work and our position never moved. Here is why.

Nova Solutions — Sun, 06 Sep 2026 10:23:22 +0000

A month of correct on-site work sat completely unseen. Not because the work was wrong, but because we never checked what the crawler had actually decided to do with it.

A month of the right work, and nothing moved

We run our own automation for advai.cloud the same way we build it for clients, so when our own search position sat completely flat for a month despite real on-site improvements, we treated it as a bug to diagnose rather than a reason to work harder on the same thing.

The instinct when a number will not move is to assume the fix is more: more pages, more content, more links. We had that instinct too, and it would have been the wrong next step. Before adding anything, we checked what a search engine had actually done with the month of work already shipped. What we found was not a content problem. It was a visibility problem, and it was almost entirely our own doing.

The sitemap was telling the crawler to ignore almost everything

A sitemap carries a field called lastmod: the date a page last changed. It is the one field a search engine actually acts on. The other common fields, priority and how often a page changes, are widely ignored. Ours emitted a lastmod on 15 of 103 pages. The other 88 carried none at all.

No lastmod does not mean neutral. It means the crawler has no signal that anything changed, so it has no particular reason to look again. A month of real improvements had shipped across the site, and for the vast majority of pages, nothing in our own sitemap told anyone to come check.

The tempting fix, and why we did not take it

The easy patch is to stamp every page with the current date on every deploy. We did not do that, because it is dishonest in a way that gets punished. It asserts that every page changed today, every day, whether or not anything actually did. Search engines are specifically built to notice that pattern and discount it.

The honest fix took longer: derive each page's lastmod from the real date its content last changed, sourced from actual commit history rather than the moment the build happened to run. A page with no real history gets no date at all, rather than a guessed one. It is more work than one global timestamp. It is also the only version that tells the truth.

A second problem hiding behind the first: pages already declined

Fixing the sitemap surfaced a second, separate issue. A batch of our pages followed the same template with one variable swapped, the kind of page that is fast to produce because the same shape repeats. Checking each one's status individually showed that only a small fraction had ever been indexed. The rest had been crawled once and declined.

The pages that had been indexed shared a pattern too: each one was a specific, independent answer to a specific question. The refused pages were the same sentence structure with a place name changed twenty times over. We stopped building more of that template. Continuing would have meant adding volume to a pattern already measured, not assumed, to be one a search engine had decided against.

What actually moved, and what we are not claiming

We are not going to claim this one fix moved our ranking. Isolating the effect of a single change on search position honestly requires more time and more controls than we have run. What we can say plainly is the diagnosis: a month of correct work was sitting completely unread by the one system whose opinion decides whether it counts, and the reason was visible the moment we checked instead of assumed.

The habit we took from this is the part worth repeating to anyone running their own site. A crawler's view of your work is a measurable, checkable thing, not a background assumption. Shipping the work is not the same event as it being seen. And a formatting shortcut that quietly asserts something false, a build timestamp standing in for a real edit date, is exactly the kind of thing an automated system exists to notice.

Questions this post answers

Why would a site's search ranking stay flat after real SEO improvements ship?

Because shipping a change and a search engine seeing that change are two different events. If the sitemap does not signal that a page changed, the crawler has no reason to look again, and improvements can sit live and completely unread for as long as that signal is missing.

What does the lastmod field in a sitemap actually do?

It is the one field a search engine uses to decide whether a page is worth re-crawling. Priority and change-frequency fields are commonly ignored. A sitemap with no lastmod, or one covering only a fraction of a site's pages, gives the crawler nothing to act on for the rest.

Why not just stamp every page with the current date on every deploy?

Because that asserts freshness that never happened, on every single page, on every build. Search engines are built to notice exactly this pattern and discount it. The honest version derives the date from when a page's content actually last changed, which for us meant reading it from real commit history rather than the build clock.

What should you do if a search engine has stopped crawling some of your pages?

Find out which ones, and consider whether continuing to publish more of the same pattern is worth it. In our case, a set of templated pages had already been declined, and building more of that template would have added to a pattern already proven not to work rather than fixing anything.

Originally published at www.advai.cloud/blog/the-month-google-never-saw-our-site.