How we let an AI read production databases without trusting it

Every engineering team knows this ticket: "I was charged twice."

Someone stops what they're doing, queries the database, greps the code, reads recent deploys and checks the logs. Eventually they explain the answer to support, who explain it to the customer.

I'm building TraceX, which does that investigation. To do it, it has to read production data, and the obvious reaction to "an AI reads your production database" is "absolutely not". That reaction is reasonable, so most of our design work has gone into one question:

How do you make this safe without relying on the model to behave?

Here's what we ended up with, including what's still weak. The rule behind all of it: anything that matters is enforced in code, not requested in a prompt.

1. Don't trust the credentials: check them

Asking people to "please use a read-only user" isn't a control. So TraceX checks the account's privileges when you connect, and refuses it if it can write:

SELECT EXISTS (
  SELECT 1 FROM information_schema.tables
  WHERE table_schema NOT IN ('pg_catalog', 'information_schema')
    AND has_table_privilege(format('%I.%I', table_schema, table_name),
                            'INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER')
  UNION ALL
  SELECT 1 FROM information_schema.schemata
  WHERE schema_name NOT IN ('pg_catalog', 'information_schema')
    AND has_schema_privilege(schema_name, 'CREATE')
  UNION ALL
  SELECT 1 WHERE has_database_privilege(current_database(), 'CREATE')
);

If that returns true, the connection is rejected with a message telling you to use a read-only account.

2. The model never writes SQL

This is the decision I'd make again first. Giving a model a SQL tool means your safety depends on parsing whatever it generates. Instead, the model can only ask for a structured lookup: a table, some filters, some columns and a limit.

That request is checked against the introspected schema, then compiled into a parameterized statement. An unknown table or column fails validation. There's no string for the model to inject into, because it never produces a query string.

You lose flexibility. It can't write a clever five-way join. For "show me the records for this order", which is most of what an investigation needs, that's a trade I'm happy with.

3. Limit what a query can do, even when it's valid

Every query runs inside a read-only transaction, with a timeout and a row cap:

async with conn.transaction(readonly=True):
    await conn.execute("SET LOCAL statement_timeout = 3000")
    rows = await conn.fetch(sql, *params)  # at most 20 rows

So the database itself refuses writes, even if the checks above had a bug.

A commenter on our launch pointed out the next problem: read-only doesn't help if your connection pool is already saturated during an incident. They were right. So TraceX uses one connection per investigation, closes it afterwards, and tells you whether you've connected a replica or a primary:

SELECT pg_is_in_recovery();

If it's a primary, you still can connect, but you get a warning suggesting a read replica. You can also add your own ceiling:

ALTER ROLE tracex CONNECTION LIMIT 2;

4. Be honest about what reaches the model

Secret-looking fields and values are masked before a prompt is built. But the records TraceX reads do go to the model provider, because that's how it reasons about them. I'd rather say that plainly than imply otherwise. You can limit what it sees by restricting the database user to specific tables, and you can bring your own API key.

5. Don't let the model grade itself

Ask a model how confident it is and you'll get a confident answer. So the confidence score is computed from the findings, not taken from the model:

def overall_confidence(findings, supporting_ids, contradicting_ids=()):
    grounded = [f for f in supporting(findings, supporting_ids) if is_grounded(f)]
    if not grounded:
        return min(best_confidence, 0.3)  # no evidence, so it can't sound sure
    value = max(confidence(f) for f in grounded)
    value += 0.05 * (independent_sources(grounded) - 1)  # logs, database and code agree
    value -= 0.15 * len(grounded_findings_against(contradicting_ids))
    return min(value, 0.95)

A finding is "grounded" only if it cites real evidence: a record, a code location or a log line. A cause with no evidence is capped at 30%, however persuasive it sounds. The score never reaches 100%.

6. Show what was ruled out

This came directly from another comment we got:

A root-cause answer I can't check is just another confident guess, and the ruled-out hypotheses are the bit I'd actually want in front of a customer.

When we looked, we found a gap: our dashboard had a "Ruled out" section, but real investigations never filled it in. So now they do, with one strict rule: an explanation is only listed as ruled out if it cites a real finding that rules it out. If the model can't back it up, it's dropped, so "ruled out" can't become another unchecked claim.

The same comment asked what happens when evidence points two ways. If a competing cause is supported within 0.15 of the leading one, TraceX shows both side by side, caps confidence at 60%, and doesn't draft a fix. A close call should go to a human.

7. A fix is a suggestion, and some suggestions need a grown-up

TraceX can write a fix, but it never opens a pull request on its own. A person clicks, and nothing is merged without your team.

Risky fixes have stricter rules. A change is high risk if it touches migrations, the schema, auth, payments or config, or runs over 80 lines. We also check the content of the change, so a rename hidden in a file with an innocent name still counts:

DDL = re.compile(
    r"\b(ALTER|DROP|CREATE|TRUNCATE|RENAME)\s+(TABLE|INDEX|COLUMN|TYPE|SCHEMA|CONSTRAINT)\b"
    r"|\bop\.(add_column|drop_column|alter_column|create_table|drop_table|rename_table)\b",
    re.I,
)

(That's shortened; the real pattern covers Django and Rails migrations too.) High-risk fixes open only as draft pull requests, only an admin can open them, and the PR explains why it's flagged.

What's still weak

• It only connects to PostgreSQL and MongoDB.
• It doesn't always reach an answer. Sometimes it comes back inconclusive and says what's missing.
• The structured queries limit it. Some questions really do need a join it can't express.
• The DDL check is a pattern match, so it can miss things. It's a safety net, not a guarantee, which is why a human still reviews every pull request.
• The numbers are judgment calls. The 0.15 penalty and 60% cap are our best current choices, not measured constants.
• Ruled-out explanations depend on the model thinking of the alternatives. We can check that each one cites evidence, but not that it considered everything.

Try it

TraceX is in private beta. There's a sandbox with made-up data and no signup. Open "Refunds stuck in pending since Tuesday's release" and look at the Resolution tab to see the ruled-out explanations.

I'd really like to hear from you: what would you need to see before letting a tool like this read your production systems? And which of these guardrails do you think is weakest?