Building an Evidence-Governed AI Operator for Power BI Engineering

oscar gutierrez — Mon, 31 Aug 2026 20:30:23 +0000

Building an Evidence-Governed AI Operator for Power BI Engineering

What we learned by separating agent reasoning, deterministic engineering facts, and evidence-based completion

Power BI projects rarely fail because teams have no diagnostics.

There are already analyzers, rules, warnings, documentation tools, and engineering practices that can identify problems in a semantic model.

The harder question comes afterward:

What does all of that evidence mean for the next engineering step?

Is the project ready for production?

Is a technical handoff actually complete?

Can a governance claim really be verified from the project?

And perhaps most importantly:

How should an AI agent behave when the available evidence is not enough to know?

Those questions became the starting point for BI Guardian Operator.

The project explores a simple architectural principle:

The engines know. The agent coordinates. The AI interprets. Evidence rules.

Rather than asking a language model to inspect a Power BI project and directly decide what is true, we separated the responsibilities of reasoning, execution, evidence, and completion.

The result is an agentic engineering workflow where the model helps coordinate the work, but does not become the source of engineering truth.

The problem was not finding warnings

A traditional Power BI analyzer can identify issues.

That is useful, but an engineering workflow often needs more than a list.

Suppose an analyzer finds several maintainability problems.

That still does not answer:

Are any of them blockers?
Is the project ready with conditions?
What evidence is still missing?
Has the requested engineering goal actually been completed?
Which findings should become part of a technical handoff?
Which governance questions cannot be established from the current project?

An agent can help coordinate those decisions, but giving the language model unrestricted authority creates a different problem.

A model can reason fluently about evidence while still being wrong about the underlying engineering facts.

So we started with a constraint:

The model should be able to reason about the work without being allowed to manufacture the facts.

Separating reasoning from engineering truth

BI Guardian Operator uses deterministic Power BI analysis capabilities to establish observable project facts.

Those capabilities inspect PBIP or TMDL project metadata and produce structured engineering evidence.

The agentic layer operates above that boundary.

At a high level:

The user selects an engineering goal.
Operator determines what evidence the mission requires.
Google ADK coordinates the available bounded capabilities.
Gemini interprets the current mission state and can select an allowed next action.
Deterministic tools execute that action and return evidence.
Operator updates the mission state.
The system evaluates what remains unresolved.
Completion is checked against explicit evidence requirements.

The important detail is what does not happen.

Gemini does not invent audit findings.

Gemini does not silently modify deterministic results.

And Gemini does not declare a mission complete simply because it believes enough work has probably been done.

The model helps coordinate the process.

The evidence determines what can be claimed.

Three missions, one governed runtime

To test the architecture, we implemented three different Power BI engineering goals using the same Operator runtime.

1. Review Before Production

The first mission asks a familiar engineering question:

Is this project ready for the next production step?

Operator coordinates deterministic audit capabilities, collects findings, and evaluates the evidence against the mission requirements.

The result can be something like:

Ready

Ready with Conditions

rather than a generic success message.

That distinction matters.

The agent can complete its mission correctly while the engineering result still contains unresolved work.

In other words:

mission completion and project quality are not the same thing.

A finding also remains traceable to its underlying deterministic rule and supporting evidence.

The interface may summarize or group results for presentation, but presentation does not replace provenance.

2. Technical Handoff

The second mission uses the same Operator runtime for a different objective.

Instead of asking whether the project is ready, the goal is to produce an engineering handoff.

Operator gathers the required project inventory, maintainability signals, findings, and engineering work items.

It can then produce durable outputs such as:

structured handoff information,
engineering findings,
and a semantic-model diagram derived from project evidence.

The interesting part is not the artifact itself.

It is that the architecture did not require a completely different agent.

The mission changes.

The required evidence changes.

The expected outputs change.

But the governed execution model remains consistent.

This was one of the most important design goals of the project:

different missions should reuse the same control principles rather than becoming separate hard-coded workflows.

3. Governance Review

The third mission exposed a different problem.

Some governance questions can be established from Power BI project metadata.

Others cannot.

A conventional system may be tempted to collapse missing information into a binary result.

No evidence might become false.

Or failed.

Or even pass.

BI Guardian Operator deliberately avoids that behavior.

The system preserves states such as:

UNKNOWN

and

NOT OBSERVABLE

when the available source cannot establish the claim.

That means:

No evidence is not the same as a pass.

It is also not automatically the same as a failure.

Sometimes the correct engineering result is simply:

This cannot be established from the evidence currently available.

For a governed agent, preserving that uncertainty is a feature, not a weakness.

Completion should be an engineering contract

One of the most useful lessons from the project was that agentic systems need a stronger definition of “done.”

A successful tool call is not completion.

A model deciding that the task appears finished is not completion either.

For BI Guardian Operator, a mission has explicit outcome requirements.

At a conceptual level, those requirements describe what evidence must exist before the mission can legitimately close.

This changes the execution loop.

Instead of asking:

“Has the model finished?”

the system asks:

“Does the current evidence satisfy the requirements of this mission?”

If the answer is no, Operator can continue.

If the required evidence cannot be obtained through the available capabilities, the mission should reflect that rather than inventing success.

This evidence-based notion of completion became one of the most important architectural ideas in the project.

Privacy shaped the architecture too

Power BI projects can contain sensitive technical and business metadata.

We did not want using an AI agent to automatically imply uploading an entire project to a model.

BI Guardian Operator therefore keeps raw PBIP/TMDL project inspection local to the browser-side analysis boundary.

The cloud agent works from sanitized operational evidence required for the mission.

This creates a useful separation:

local project inspection → structured evidence → cloud orchestration

rather than:

raw project → model

It is not a universal solution to every privacy problem, but it substantially reduces unnecessary exposure in the workflow demonstrated by the project.

The Google Cloud agent stack

The production version of BI Guardian Operator uses several Google technologies, each with a distinct responsibility.

Google Agent Development Kit

Google ADK provides the agent execution and coordination layer.

It allows the Operator runtime to expose bounded capabilities rather than giving the model unrestricted control over the underlying engineering tools.

Gemini on Vertex AI

Gemini interprets mission context and helps select appropriate next actions from the capabilities available to the agent.

The important architectural boundary is that Gemini reasons about the evidence but does not replace the deterministic engines that establish Power BI facts.

Cloud Run

The Operator backend is deployed on Google Cloud Run.

This provides the hosted execution environment for the cloud portion of the system.

Firestore

Firestore stores sanitized operational mission state required by the agent workflow.

The raw Power BI project remains outside that cloud persistence boundary.

The hardest parts were not the AI calls

The most difficult engineering questions were architectural.

What should the model be allowed to decide?

Giving the model access to a tool is easy.

Defining what authority that tool call should have is harder.

We wanted the model to choose useful actions without allowing it to redefine deterministic engineering truth.

How do you know when a mission is finished?

Agents are good at continuing conversations.

Engineering systems need something stronger than conversational closure.

Explicit evidence requirements gave the system a deterministic boundary around mission completion.

How should missing information behave?

This became particularly important in governance.

We had to resist the temptation to force every question into a binary answer.

Preserving UNKNOWN and NOT OBSERVABLE produced a system that was less absolute, but more trustworthy.

How much project information should reach the cloud?

This influenced the local-analysis boundary and the decision to send only sanitized mission evidence into cloud orchestration.

An important implementation boundary

BI Guardian Operator builds on pre-existing deterministic Power BI analysis capabilities.

Those tools already knew how to inspect specialized BI artifacts.

For this project, the focus was intentionally different:

How can an agent govern and coordinate specialized deterministic capabilities into complete engineering missions?

The hackathon work concentrated on the Operator layer around those capabilities, including mission orchestration, bounded execution, evidence-governed completion, operational state, Google ADK and Gemini integration, Cloud Run deployment, Firestore persistence, and the multi-mission experience demonstrated in the final project.

That boundary turned out to reinforce the architecture.

The deterministic engines provide domain knowledge.

The Operator provides governed coordination.

What we learned

The biggest lesson was that an AI agent does not need to become the authority for everything it touches.

In fact, the system became easier to reason about once responsibilities were separated.

A useful division emerged:

Deterministic systems establish facts.

The agent coordinates work.

The model interprets context.

Evidence determines what can legitimately be claimed.

We also learned that uncertainty deserves an explicit representation.

A system that can say “I cannot establish this from the available evidence” may be more useful in engineering and governance than a system that always produces an answer.

Finally, completion itself deserves to be designed.

For multi-step agentic workflows, deciding when there is enough evidence to stop can be just as important as deciding what to do next.

Current limitations

BI Guardian Operator is intentionally scoped.

The current implementation focuses on Power BI engineering artifacts and a bounded set of demonstrated missions.

It does not attempt to make every possible governance fact observable from PBIP/TMDL metadata.

It also does not treat generative reasoning as a replacement for deterministic BI analysis.

Those are deliberate constraints.

The goal of the current project is not to maximize how many decisions the AI can make.

It is to explore how much useful engineering work an agent can coordinate without sacrificing evidence, traceability, and justified completion.

What's next

The next step is to expand the library of engineering capabilities and missions while preserving the same execution principles.

More capabilities should not require giving the model more authority over engineering truth.

Instead, the system should become useful by coordinating richer specialized tools while keeping the boundaries explicit.

That raises a broader question beyond Power BI:

Can increasingly capable AI agents perform real engineering work while remaining constrained by evidence, authority, and explicit completion criteria?

BI Guardian Operator is one experiment toward answering that question.

Demo

Demo video:
https://youtu.be/txrXsJ-pdu4

This article was created for the purpose of entering BI Guardian Operator in the Google Cloud All Things Agentic Hackathon 2026.

DEV Community: oscar gutierrez

Building an Evidence-Governed AI Operator for Power BI Engineering

Building an Evidence-Governed AI Operator for Power BI Engineering

What we learned by separating agent reasoning, deterministic engineering facts, and evidence-based completion

The problem was not finding warnings

Separating reasoning from engineering truth

Three missions, one governed runtime

1. Review Before Production

2. Technical Handoff

3. Governance Review

Completion should be an engineering contract

Privacy shaped the architecture too

The Google Cloud agent stack

Google Agent Development Kit

Gemini on Vertex AI

Cloud Run

Firestore

The hardest parts were not the AI calls

What should the model be allowed to decide?

How do you know when a mission is finished?

How should missing information behave?

How much project information should reach the cloud?

An important implementation boundary

What we learned

Current limitations

What's next

Demo