DEV Community

ArisynData
ArisynData

Posted on

Your AI Data Agent Needs a “Do Not Use” Layer

The missing layer in many enterprise AI systems is not more metadata. It is a machine-readable record of the mistakes the organization already knows not to make.

A surprising amount of enterprise data expertise sounds like this:

Don't use that table after the migration.

Don't use invoice_amount as recognized revenue.

Don't join those two tables directly.

Don't use created_at for financial reporting.

Don't aggregate those rows before deduplicating accounts.

These rules are rarely visible in the database schema. They may not exist in the catalog. Often they live only in the heads of experienced analysts, engineers, and business owners.

That worked when humans were the primary consumers of enterprise data. It becomes a serious problem when AI agents start querying data directly.

The usual response is to give the model more context.

But what if the missing context is not another description of what the data is?

What if the missing layer is a structured description of what the AI must not do?


## The Data Model Contains Less Knowledge Than We Think

Consider:

CREATE TABLE invoice (
    invoice_id      BIGINT,
    customer_id     BIGINT,
    invoice_amount  DECIMAL(18,2),
    invoice_date    DATE,
    status          VARCHAR(20)
);
Enter fullscreen mode Exit fullscreen mode

An AI can identify a customer reference, an amount, and a date. It can probably generate valid SQL for:

What was recognized revenue last quarter?

But the enterprise may already know:

invoice_amount is not recognized revenue.
Enter fullscreen mode Exit fullscreen mode

That fact is not a schema fact.

It is organizational knowledge.

Without it, the agent can be technically competent and still wrong.


## “Do Not Use” Is a Real Data Asset

Data platforms traditionally capture positive knowledge:

What tables exist?
What do columns mean?
What is the metric definition?
How are entities related?
Enter fullscreen mode Exit fullscreen mode

Expert users also carry negative knowledge:

Which source is misleading?
Which join is dangerous?
Which field looks correct but isn't?
Which timestamp should not be used?
Which aggregation creates double counting?
Enter fullscreen mode Exit fullscreen mode

This knowledge is valuable precisely because the wrong choice often looks reasonable.

A production data agent therefore needs:

Positive Knowledge
→ What can I use?

Negative Knowledge
→ What must I avoid?
Enter fullscreen mode Exit fullscreen mode

## A “Do Not Use” Layer

Imagine structured constraints attached to enterprise data objects.

For a field:

object:
  type: column
  name: invoice.invoice_amount

meaning:
  business_term: Invoice Value

usage:
  valid_for:
    - invoice_analysis

  invalid_for:
    - recognized_revenue

reason:
  Invoice value may include amounts that have
  not yet met revenue recognition rules.

owner:
  finance

status:
  active
Enter fullscreen mode Exit fullscreen mode

For a relationship:

object:
  type: relationship

from:
  customer.customer_id

to:
  order.customer_id

constraint:
  invalid_when:
    account_structure: consolidated

required_path:
  - customer
  - account
  - order

reason:
  Direct joins can duplicate orders across child accounts.
Enter fullscreen mode Exit fullscreen mode

For a time field:

object:
  type: time_rule

metric:
  recognized_revenue

required_field:
  finance_revenue.recognition_date

do_not_use:
  - invoice.invoice_date
  - sales_order.created_at
Enter fullscreen mode Exit fullscreen mode

This is not a blacklist.

It is contextual enterprise knowledge.


## Why a Blacklist Is Too Primitive

Marking:

invoice.invoice_amount = forbidden
Enter fullscreen mode Exit fullscreen mode

would be wrong.

The field is perfectly valid for Invoice Value. It is invalid only when interpreted as Recognized Revenue.

So the real rule is:

Object + Business Intent + Context → Valid / Invalid
Enter fullscreen mode Exit fullscreen mode

For example:

field: invoice.invoice_amount

valid_for:
  - invoice_value

invalid_for:
  - recognized_revenue
  - cash_received
Enter fullscreen mode Exit fullscreen mode

The same physical field can be valid or invalid depending on the analytical task.


## Put Constraints Before the LLM

A naive pipeline:

Question
   ↓
Schema Retrieval
   ↓
LLM
   ↓
SQL
Enter fullscreen mode Exit fullscreen mode

A stronger pipeline:

Question
   ↓
Business Intent Resolution
   ↓
Candidate Retrieval
   ↓
“Do Not Use” Constraint Check
   ↓
Trusted Relationship Resolution
   ↓
Allowed Query Context
   ↓
LLM
   ↓
SQL
   ↓
Constraint Validation
Enter fullscreen mode Exit fullscreen mode

The constraint layer appears twice.

First, it prevents known-invalid data from entering the model's decision space.

Second, it validates that generated SQL did not violate a known rule.


## Filter Invalid Candidates Before Generation

Suppose retrieval returns:

candidates = [
    "finance_revenue.recognized_amount",
    "invoice.invoice_amount",
    "sales_order.total_amount",
    "payment.received_amount"
]
Enter fullscreen mode Exit fullscreen mode

For a recognized-revenue question:

def filter_by_constraints(candidates, intent, knowledge):
    allowed = []
    excluded = []

    for candidate in candidates:
        rule = knowledge.evaluate(
            object=candidate,
            intent=intent
        )

        if rule.is_hard_violation:
            excluded.append({
                "object": candidate,
                "reason": rule.reason
            })
            continue

        allowed.append(candidate)

    return allowed, excluded
Enter fullscreen mode Exit fullscreen mode

The model now receives a much cleaner problem.

But keep the exclusion reason:

excluded_context:
  - object: invoice.invoice_amount
    reason: not valid for recognized revenue
Enter fullscreen mode Exit fullscreen mode

That helps explanation and debugging.


## Relationship Constraints May Matter Even More

Wrong joins are dangerous because they can return plausible results.

Imagine:

Customer ───── Order
    │
    └──── Account ───── Order
Enter fullscreen mode Exit fullscreen mode

Both paths may technically exist.

For consolidated accounts, the direct path may duplicate orders.

Represent the rule:

constraint:
  type: relationship_path

  from: customer
  to: order

  when:
    account_structure: consolidated

  forbidden_path:
    - customer
    - order

  required_path:
    - customer
    - account
    - order
Enter fullscreen mode Exit fullscreen mode

Then build a task-specific graph:

Customer ──X── Order
    │
    └──── Account ───── Order
Enter fullscreen mode Exit fullscreen mode

Conceptually:

valid_graph = relationship_graph.apply_constraints(
    business_context=context
)

path = valid_graph.shortest_trusted_path(
    source="customer",
    target="order"
)
Enter fullscreen mode Exit fullscreen mode

This is safer than asking an LLM to infer join topology from names every time.


## Hard Rules and Soft Rules

Not every constraint should block execution.

### Hard Constraint

severity: hard

rule:
  do_not_use: customer_legacy

when:
  query_date >= 2026-01-01
Enter fullscreen mode Exit fullscreen mode

Violation should stop or regenerate the query.

### Soft Guidance

severity: soft

rule:
  prefer: billing_region

fallback:
  registered_region
Enter fullscreen mode Exit fullscreen mode

Violation may lower ranking or trigger review.

This avoids two extremes: an inflexible rule engine and a prompt that treats every rule as optional advice.


## Compile Governance Into Runtime Constraints

Suppose a governed metric says:

metric:
  id: recognized_revenue

source:
  finance_revenue.recognized_amount

time_field:
  finance_revenue.recognition_date

invalid_sources:
  - invoice.invoice_amount
  - sales_order.total_amount

required_filters:
  - status=recognized
Enter fullscreen mode Exit fullscreen mode

Compile it:

policy = {
    "required_source":
        "finance_revenue.recognized_amount",

    "required_time_field":
        "finance_revenue.recognition_date",

    "forbidden_sources": [
        "invoice.invoice_amount",
        "sales_order.total_amount"
    ],

    "required_filters": [
        "status=recognized"
    ]
}
Enter fullscreen mode Exit fullscreen mode

The same policy can shape retrieval, context construction, generation, and validation.

One governed definition drives multiple runtime controls.


## Validate What Must Not Appear

After generation, parse the SQL and check the policy:

def validate_query(ast, policy):
    violations = []

    for source in policy["forbidden_sources"]:
        if ast.references(source):
            violations.append({
                "type": "forbidden_source",
                "object": source
            })

    if not ast.references(policy["required_time_field"]):
        violations.append({
            "type": "wrong_time_field"
        })

    return violations
Enter fullscreen mode Exit fullscreen mode

If violations exist:

if violations:
    regenerate_or_escalate(
        sql=sql,
        violations=violations
    )
Enter fullscreen mode Exit fullscreen mode

The agent is no longer trusted to police itself.


## Where Does Negative Knowledge Come From?

One of the best sources is historical failure.

Example:

Problem:
Revenue dashboard overstated.

Root Cause:
Orders joined directly to customers.

Fix:
Use account-level relationship.

Lesson:
Never use direct Customer → Order join
for consolidated accounts.
Enter fullscreen mode Exit fullscreen mode

Most organizations stop after fixing the dashboard.

A stronger system captures the lesson:

Failure
   ↓
Root Cause
   ↓
Data Rule
   ↓
Structured Constraint
   ↓
Future Prevention
Enter fullscreen mode Exit fullscreen mode

Operational experience becomes AI infrastructure.


## Ask Experts for Mistakes, Not Documentation

If you ask a senior analyst:

Please document these 50 columns.

you may get accurate but generic descriptions.

Instead ask:

What are the five mistakes a new analyst is most likely to make with this data?

You may hear:

Don't use that amount as revenue.
Don't use this table after migration.
Don't join those IDs directly.
Don't use created_at for monthly reporting.
Don't SUM before deduplicating accounts.
Enter fullscreen mode Exit fullscreen mode

Those answers encode experience, not just definitions.

They are often extremely valuable for AI.


## Version and Own the Constraints

A constraint can become outdated.

It should include:

ID
Version
Owner
Status
Effective Date
Reason
Validation State
Enter fullscreen mode Exit fullscreen mode

For example:

constraint:
  id: C-REV-014
  version: 3

  rule:
    invalid_for:
      object: invoice.invoice_amount
      metric: recognized_revenue

  owner:
    finance

  status:
    active

  effective_from:
    2026-01-01

  reason:
    revenue recognition follows finance_revenue
Enter fullscreen mode Exit fullscreen mode

Now the system knows whether the rule is still authoritative.


## Make the Layer Observable

Useful metrics include:

Constraint Coverage
% of critical metrics with explicit negative rules

Prevented Violations
Invalid candidates removed before LLM generation

Generated SQL Violation Rate
% of queries violating known constraints

Known-Mistake Recurrence
How often a known incident repeats after its rule is encoded
Enter fullscreen mode Exit fullscreen mode

The last metric captures the real purpose.

The goal is not to create more documentation.

It is to stop known mistakes from recurring.


## Do Not Turn Everything Into a Rule

Not every piece of human judgment should become deterministic policy.

Some decisions are ambiguous. Some rules have many exceptions. Some knowledge is uncertain.

Encode negative knowledge when it is:

Known
Repeated
High-impact
Business-specific
Stable enough to govern
Hard for the model to infer from schema alone
Enter fullscreen mode Exit fullscreen mode

Leave genuinely ambiguous reasoning to the model.

The goal is not:

Replace AI reasoning with rules.
Enter fullscreen mode Exit fullscreen mode

The goal is:

Stop using AI reasoning for mistakes
the enterprise already knows how to prevent.
Enter fullscreen mode Exit fullscreen mode

## A More Mature Data Agent Runtime

Putting it together:

User Question
      ↓
Business Intent
      ↓
Candidate Retrieval
      ↓
Positive Knowledge
      +
Negative Knowledge
      ↓
Constraint Filtering
      ↓
Trusted Relationship Planning
      ↓
Allowed Context
      ↓
LLM Reasoning / SQL Generation
      ↓
Constraint Validation
      ↓
Execution
      ↓
Answer
Enter fullscreen mode Exit fullscreen mode

The LLM no longer operates over everything that looks relevant.

It operates inside a data space shaped by enterprise knowledge.


## The Bigger Engineering Principle

A wrong enterprise answer is not always a hallucination.

Sometimes it is:

valid computation over an invalid business assumption.

A bad join can execute.

A wrong metric can aggregate.

A deprecated table can return real rows.

A wrong date field can produce a believable trend.

That is why more context alone is not enough.

The system also needs boundaries.


## Final Thoughts

The most valuable enterprise data knowledge may not be another description of what a field means.

It may be:

Don't use this field for that metric.

Or:

Don't take that join path.

Or:

Don't use that timestamp for reporting.

Those warnings represent accumulated organizational experience.

If they remain in people's heads, every new AI agent can repeat old mistakes.

If they become structured, versioned, and enforceable, they become something more useful:c

A “Do Not Use” layer for enterprise AI.

And that may be one of the missing pieces between an AI agent that can query data and one that can use enterprise data reliably.

Top comments (0)