Fernando Azevedo

Posted on Nov 6 Originally published at fernando.moretes.com

LLM-generated data quality rules are cheap. Owning them for years is not

#dataplatforms #dataquality #agentesdeia #awsglue

After 16 years building financial platforms on AWS, the question I ask about data quality isn't "who writes the rule?", it's "who answers for it at 2 AM, when it blocks a legitimate load?". The Indegene case, published on November 4, 2026, matters because of that second question: an agent proposes rules and code, but only what passes human approval enters the central repository. Combined with the ADVANCED mode in AWS Glue Data Quality, it shows where the cost is heading: out of authoring and into maintenance.

Three numbers to calibrate the decision

17: Regions with ADVANCED mode. The documented list does not include sa-east-1
14d: Default sample retention in S3. Lifecycle rule on the Athena query results
267h: Review in my own hypothetical scenario. 200 tables × 40 columns × 1 rule × 2 minutes

The signal: generating rules became a commodity

Indegene's Data Quality Agent is part of its NEXT Unified Data Platform. It was built with Strands Agents, runs on Amazon Bedrock AgentCore and serves Amazon Redshift and Snowflake behind AWS WAF, API Gateway and Cognito, with OAuth 2.0 and JWT validation on each request. The flow has 10 steps: source, schema, profiling, anomalies against a historical baseline, existing rule management, rule mode (column, table or custom query), suggestions with severity, review, SQL and Python generation and validation, and storage in the central repository.

Two knowledge bases provide the context. The internal one combines glossaries, data dictionaries, SOPs, regulatory guidance, approved rules, validated scripts, profiling results, historical anomalies and expert feedback. The external one comes from approved frameworks, web search and controlled scraping. That's what lets the agent suggest the 10-digit format for an NPI column without anyone teaching it.

In September 2026, Glue Data Quality gained ADVANCED mode, which uses Bedrock over metadata and a random sample of the table to generate DQDL rules. Once the data plane itself ships generation out of the box, generation stops being a platform differentiator.

One limit on what I read: the part of the post I could open ends at the architecture description, with no detail on the validation mechanism and no results figure. That's why I don't quote any percentage gain here.

What changes for whoever designs the platform

Rules become design-time artifacts: the documentation warns that ADVANCED can return different rules for the same table, so generation can't sit in the execution path.
The bottleneck moves: writing stops being the limit, human review capacity takes its place.
Technical and business rules now share one lifecycle: the one living in code and the one living in an SOP enter the same versioned repository.
The sample becomes a data surface: generating with context means sending real rows to a model, and that requires a classification decision before the first run.
The knowledge base can propagate errors: approved rules return as context, so an error approved today can be suggested again tomorrow.

Where a generated rule fails silently

The risk isn't the rule that breaks, it's the rule that passes. A rule with a syntax error fails on the first run. The ones below stay green for months:

Inverted semantics: in row-level CustomSql, a record returned by the query counts as passed and a record not returned counts as failed. A model that writes the natural audit query, select id from primary where amount < 0, passes exactly the wrong records.
The approving filter: with where, filtered-out rows are labeled Passed by default. Use rowLevelConfiguration.filteredRowLabel = SKIPPED, or a filter that matches nothing looks like compliance.
Column-scoped composites: by default, (A) or (B) evaluates each rule across the whole table and combines the results afterward. The documentation's example fails in that mode and passes with compositeRuleEvaluation.method = ROW. Completeness, Uniqueness, DatasetMatch and ReferentialIntegrity aren't even supported in ROW mode.
Empty history: with no prior data points, last(k) returns 0.0, so RowCount > last() passes on the first run without proving anything.
Frozen domain: a rule derived from the sample, such as ColumnValues "territory" in [...], treats a new legitimate value as a defect, and can treat an old defect as a valid domain.

The post's business example, market share totaling 100% per territory, carries the same numeric trap as a financial reconciliation: it needs an explicit tolerance. One way to write it in DQDL, with a tolerance you have to calibrate: CustomSql "select count(*) from (select territory_id from primary group by territory_id having abs(sum(market_share) - 100) > 0.5) t" = 0.

Where human approval separates generation from execution

The diagram answers one question: what has to happen for a model suggestion to become a production rule. It simplifies the 10 steps described in the post. The dashed arrows are the asynchronous return to the knowledge base, and that's the path by which an approved error can come back as a suggestion.

Generation

Knowledge bases internal + external (data)
Data Quality Agent Strands + AgentCore (ai)

Governance

Data steward review and edit (user)
Code validation SQL + Python (security)
Rule repository approved rules (storage)

Runtime

Redshift, Snowflake rule execution (data)
Monitoring and audit results (compute)

Flows

kb -> agent: RAG
agent -> steward: suggests with severity
steward -> validate: approves
validate -> repo: stores
repo -> engine: runs
engine -> monitor: publishes results
repo -> kb: approved rules
monitor -> kb: historical anomalies

The sample boundary is an architecture decision

ADVANCED mode is practical, and the data protection documentation is clear about what it does. Athena samples rows through the glue-dataquality-sampling workgroup, and Glue doesn't exclude any column from the sample. Metadata and rows go to Bedrock, which doesn't retain inputs or outputs after processing and doesn't use them to train models. For many scenarios, that's enough.

For healthcare professional, patient or bank customer data, three details change the math:

No masking in the call: ADVANCED doesn't support PreProcessingQuery, so the SELECT without the sensitive column, the obvious way out, isn't available. Point the run at an already masked table.
Cross-Region inference: the run uses geographic cross-Region inference, and sampled content can leave the source Region, within the geographic boundary of the profile. With sa-east-1 off the list, that path has to enter your LGPD transfer inventory and, where applicable, your BACEN 4.893 assessment.
Bedrock IAM isn't the control: the recommendation role doesn't need Bedrock permissions, so denying bedrock:InvokeModel on it doesn't guarantee the send is blocked. The useful control is pinning aws:SourceArn to the table in the trust policy and not giving the role access to the sensitive table.

BASIC deserves attention too. In supported Regions, it attempts to enhance the ruleset with Bedrock by sending the ruleset and column metadata, without you enabling anything. A statistical ruleset can carry real values in a ColumnValues ... in [...] list, so check that against your classification.

Three paths for generating rules

Criterion	Glue `BASIC`	Glue `ADVANCED`	Own agent (Indegene pattern)
Rule context	Table statistics	Metadata and sampled rows	Glossary, SOP, history and expert feedback
Sources	Data Catalog tables and ETL jobs	Data Catalog tables, in 17 Regions	Whatever you connect; in this case, Redshift and Snowflake
Data sent to the model	Ruleset and column metadata, when enhanced	Metadata and rows, with no column exclusion	Whatever your design allows, and the control is yours
The cost that stays	Reviewing statistical rules	Reviewing every run and paying for Athena	Maintaining agent, knowledge bases, evals and all of the above
Use when	The dataset is new and the goal is structural coverage	The data can be sampled and Glue is already the catalog	The rule depends on knowledge outside the table, across several platforms

Human approval is capacity, not a checkbox

Human-in-the-loop usually appears in a diagram as a person icon. In operations, it's a queue. Run the math for your environment. In my hypothetical scenario, 200 tables with 40 columns and a single rule per column give you 8,000 rules. At 2 minutes per rule, that's 267 hours of review before the first execution. Nobody reviews that carefully: at some point in the queue, approving becomes clicking, and the control stays on paper while it stops existing in practice.

The per-rule severity that Indegene's agent already suggests is what makes that queue manageable. Review the critical ones one by one, batch-approve low-risk structural rules with audit sampling, and discard without guilt any rule nobody would claim as owner. A rule with no named owner shouldn't enter the repository.

Then comes the recurring cost. Going by the examples on the Glue pricing page, a 5-DPU evaluation running 20 minutes costs about US$ 0.73, at US$ 0.44 per DPU-hour. Across 200 tables, daily, that's about US$ 4,380 per month, and it's the smallest of the costs. The largest is the false alert: every bad rule triggers an investigation, and a team that chases three false alarms stops looking at the fourth. Measure per rule how many failures turned into a real fix, and retire the ones that never did.

How I would position for this

Treat rules as code: the approved ruleset goes to a versioned repository, with hash, owner, severity and reason. Execution reads only that version and never calls the model. This also resolves the nondeterminism: running generation twice doesn't change what's in production.

Test the rule against seeded defects: for each rule, one fixture with good records and another with the defect it promises to catch. The rule must pass on the first and fail on the second. This is the test that catches the inverted CustomSql, and no text review catches it as reliably. Strands evaluation tools help measure the agent, but an LLM judge evaluating LLM output doesn't replace executing the rule.

Don't confuse a valid plan with a safe query: Redshift's EXPLAIN shows the plan without running it, and it also accepts UPDATE and DELETE. Passing it proves the query compiles, not that it only reads. Combine it with a SELECT allowlist in a parser and a read-only role.

Separate the generation identity from the execution identity: the November 5 post on AgentCore and Lake Formation shows the right path for exploration: the query runs under the user's mapped role, with LF-Tags and data cell filters, and enforcement doesn't depend on the SQL the agent wrote. Production execution is a different story. Glue's support matrix marks tables with Lake Formation column or data filters as unsupported across several formats, so the quality role tends to have broad access. Treat that role as a privileged principal and audit it as one.

What I wouldn't do

Generating rules at run time: the same data can get different rules between runs, and the audit loses the answer to "which rule applied that day?".
Measuring success by rule count: per-column coverage is easy to inflate, and every extra rule is one more alert source for someone to maintain.
Feeding web content straight into the rules RAG: a scraped page is untrusted input. Require the same curation as the internal base, or a suggested rule can originate from text nobody approved.
Relying on labels for severity everywhere: DQDL labels exist only in Glue ETL, not in Data Catalog-based Data Quality. Whoever standardizes on them finds the gap in the report.

The lesson in the feedback loop: Approved rules and validated scripts enter the internal knowledge base. That improves the next suggestion and also gives a bad approval a reach it didn't have before: the error stops affecting one table and starts guiding the next ones. When you revoke a rule, remove its vector from the knowledge base too. Deleting it only from the repository leaves the error alive in the context.

Curator's note: I'd start with Glue BASIC on a non-sensitive dataset and use ADVANCED only on a masked table, before writing a line of agent code. I'd write the seeded-defect fixtures in the first week, because it's the one item on this list that gets more expensive with every rule approved without it. I'd only move to my own agent when the rule that matters depended on SOPs and glossaries, across more than one platform, which is the actual condition of the Indegene case. The lesson I take from financial systems: the cost of an automated control isn't the cost of creating it, it's the cost of proving years later why it exists and whether it still works.

My recommendation

Use the native Glue capability when your catalog already lives there, the rule can be explained by the data and the sample can go to the model. Build an agent when the rule depends on knowledge outside the table, across several platforms, and you can pay for its maintenance. In both cases, don't approve at scale before you have an owner per rule, seeded-defect tests and retirement criteria: without those, generating faster only brings the debt forward.

Rating: Adote com governança / Adopt with govern

References

Originally published at fernando.moretes.com. By Fernando F. Azevedo: Senior Solutions Architect.

DEV Community