LLMs are good at interpreting messy engineering context. They can read a bug report, summarize a pull request, or spot a pattern in deployment metrics.
But should a model’s interpretation directly decide whether a deployment proceeds?
JevOps explores a more controlled approach: use an LLM to extract useful facts, then pass those facts to a deterministic Jev policy engine. The application acts on the policy result and keeps a trace of how it got there.
How the flow works
- Engineering signals come in. A pull request, bug report, or deployment metrics provide the context.
- The LLM analyzes the evidence. It turns unstructured text and signals into structured facts, with supporting evidence where available.
- The Jev policy engine evaluates those facts. Explicit policy rules produce a bounded decision, such as a risk assessment, triage result, or deployment gate.
- The application takes the action. It can annotate a pull request, route a bug, or pass, hold, or request approval for a deployment.
The important boundary is that the LLM supplies interpreted evidence; the policy engine governs the decision.
Why separate analysis from policy?
A model can be useful without being the final authority. Separating interpretation from policy makes the decision path easier to inspect:
- The extracted facts can be reviewed against their source evidence.
- Policy rules can be versioned and evaluated independently.
- The result can include reason codes and a trace of the policy version used.
- Sensitive actions can require human approval.
That structure also makes failures easier to investigate. If a result is wrong, you can ask whether the evidence was misread, the policy was incomplete, or the application handled the outcome incorrectly.
Auditability and feedback
A useful decision record should connect the input evidence to the outcome: the extracted facts, policy version, reason codes, action, and timestamp. Explanations help engineers understand why an outcome was reached, while human corrections and later results can inform evaluation and future policy improvements.
This doesn’t remove the need for testing or human judgment. It gives teams a clearer way to constrain automated decisions and review them afterward.
JevOps was selected for the XERJ Corpus Hub. I’m continuing to develop the project around a practical question: how can AI help with engineering workflows while keeping consequential decisions understandable and governed?
What engineering decisions would you want an AI system to explain before it acts?
Architecture diagram: JevOps architecture
Top comments (0)