In Thinking, Fast and Slow, Daniel Kahneman describes two modes of thinking. System 1 is fast and intuitive: you know that 2 + 2 = 4 without any effort. System 2 is slow and deliberate: the result of 17 × 24 takes time and real cognitive effort.
That distinction was on the Typesafe.ai team's mind when they built Jev.
Jev isn't a model for writing text, generating code, or creating images. It's a decision model. You give it a context (plain text, JSON, email…) and predefined questions, then, it returns a predefined answer with a probability. Jev supports three question types:
- Choice picks an option from a list.
- Yes/No says whether a statement is true, with a probability between 0 and 1 (returned as noul)
- Score rates the state against a rubric.
In other words, Jev gives you an answer you can act on directly: "yes, 87%" or "category B". An LLM, by contrast, writes a full response. The first is built for chaining automated actions; the second shines at explanations and documentation.
Jev is also fast and cheap. Answers are tiny, and only input tokens are billed; output is free.
The best use case are:
- The possible answers are known in advance. Is this email a scam? How urgent is this request? Which team should handle it? If you can't enumerate the answers, or the task needs deep reasoning, a decision model probably isn't the right tool.
- It happens a lot. Logs, metrics, tickets, emails: at that volume, speed and cost make all the difference.
- It needs judgment, not deep thinking. Think of Jev as a human making a quick call on something too fuzzy for a regex or a script.
- Something happens next. The answer feeds a workflow: route, alert, block, approve.
In one sentence: a decision model is worth it when you have a large pile of things that each need a quick human-like judgment, and that judgment should trigger an automatic next step.
There are a lot of example on Youtube and elsewhere, but they are very basic. To better understand Jev, I will use it in every day tasks of a cloud engineer.
To see what Jev can really do, I'll apply it to the everyday work of a cloud engineer, automated with PowerShell and the JEV PowerShell module from Doug Finke.
I will create another post later to explain who to work with Jev in detail with PowerShell.
Before starting, the TYPESAFE_API_KEY environment variable should be setup with the API key from TypeSafe.ai and store it in the TYPESAFE_API_KEY environment variable. Outside a quick test, don't hardcode it: pull it from a vault instead.
$env:TYPESAFE_API_KEY = "apikey_XXXXX"
First example, how to work with Jev in PowerShell
Now the first test to understand how Jev work.
# import module
Import-Module ../Jev/Jev.psd1 -Force
# create the state, the context for the question
$feedback = [pscustomobject] @{
message = 'The customer is blocked by a connection issue to the database.'
}
# define the criteria for the question
$criteria = @{
support = 'The issue needs technical support.'
sales = 'The issue concerns sales.'
}
# create the question based on the criteria
$question = New-JevQuestion -Name TeamRouting -Type Choice -Instructions 'Which team should handle this?' -Criteria $criteria
# invoke the question with the current state
$result = Invoke-Jev -State $feedback -Question $question
# check the result of the question
$result.answers.TeamRouting
The result look like
type choice confidence probabilities
choice support 1.00 @{support=1; sales=0}
The decision model pickup “support” with confidence, but it was just a test, and this example says nothing on how Jev can help in day to day task off a Cloud Engineer.
How to handle resource change and catch risky operation with Bicep -whatif
One of the first use case I have in mind is Infrastructure as Code change. In Terraform or Bicep it you have a complex infrastructure, the Terraform Plan or the Bicep -WhatIf will be dense. An you will have often several questions:
- Do the change after a part of critical resource?
- Do it weaken security on resources?
- What will happen if something go wrong
If for example you use Bicep to manage a complex Azure Network Infrastructure, with Express Route Gateway, Circuit, VPN connection, you probably review the -whatif output before executing the change in a pipeline.
Here, Jev can help, the WhatIf output will be our state, or context.
Bicep WhatIf output is verbose and need to be cleaned and shorted for Jev. The output I used here is a 55Kb file.
{
"ResourceId": null,
"ChangeType": "Create",
"Before": null,
"After": {
XXXX
}
Some parts of the file are useless. When the “changeType” property is “noChange” or “ignore” should be striped before sending to Jev.
$whatIf = Get-Content -Path $WhatIfFile -Raw | ConvertFrom-Json
$changes = $whatIf.Changes | Where-Object -Property ChangeType -NotIn -Value @('NoChange', 'Ignore')
But the data still have the full Before and After payload of the resource, so the few properties that change are buried in ids. We only need to have the change type, the resource and the changed property paths.
$summary = foreach ($c in $changes) {
$resource = if ($c.After) { $c.After } else { $c.Before }
"$($c.ChangeType): $($resource.type) $($resource.name)"
if ($c.ChangeType -eq 'Create') {
' properties: ' + ($resource.properties | ConvertTo-Json -Compress -Depth 10)
}
foreach ($d in $c.Delta) {
" $($d.PropertyChangeType) $($d.Path)"
}
}
Now we can create an array of question.
$questions = @(
New-JevYesNoQuestion -Name security `
-Question 'Do the changes in `whatif` weaken the security of the environment?' `
-TrueCriteria 'Password login enabled on a VM, no NSG on a NIC or subnet, a security setting removed or relaxed' `
-FalseCriteria 'Security settings are unchanged or stronger'
New-JevYesNoQuestion -Name public_ip `
-Question 'Do the changes in `whatif` expose a resource to the internet with a public IP?' `
-TrueCriteria 'A public IP address is created or attached to a NIC, a VM or a load balancer' `
-FalseCriteria 'No public IP is created or attached'
New-JevQuestion -Name risk -Type Score `
-Instructions 'How risky is it to deploy the changes in `whatif`?' `
-Criteria @(
'No risk: cosmetic or read-only property changes'
'Low: new isolated resources'
'Medium: changes to shared network resources (VNets, peerings, route tables)'
'High: an outage or a security incident is likely'
)
)
And get the answer. Here you can see that the decision stays in the code and can be used later
$state = @{ whatif = $summary -join "`n" }
$answers = (Invoke-Jev -State $state -Question $questions -Mock:$Mock -Raw).answers
$reasons = @(
if ($answers.security.noul -gt 0.7) { 'Weakens security' }
if ($answers.public_ip.noul -gt 0.7) { 'Exposes a public IP' }
if ($answers.risk.score -ge 2) { "Risk level $($answers.risk.score)" }
)
[pscustomobject]@{
Decision = if ($reasons) { 'Review' } else { 'Pass' }
Reasons = $reasons -join '; '
Changes = @($changes).Count
Security = [math]::Round($answers.security.noul, 2)
PublicIp = [math]::Round($answers.public_ip.noul, 2)
Risk = $answers.risk.score
}
The result
Decision : Review
Reasons : Weakens security; Exposes a public IP; Risk level 2.72
Changes : 16
Security : 0.9
PublicIp : 0.97
Risk : 2.72
Mixing PowerShell result and Jev insights.
Another use case for a decision model is reviewing NSG configurations, especially when a simple PowerShell query is insufficient. A decision model can provide insights beyond PowerShell’s capabilities.
In this example, PowerShell is used for deterministic checks (like a management port open to the Internet) and Jev to check the rule to NSG best practices.
First part PowerShell need to test rules.
$managementPorts = 22, 3389, 5985, 5986
$internetSources = '*', 'Internet', '0.0.0.0/0', 'Any'
# merge singular and plural rules
function Get-RuleValue {
param([object]$Single, [object]$Plural)
@(@($Single) + @($Plural) | Where-Object -FilterScript { $_ })
}
# Then a function to test the port
function Test-PortMatch {
param([string[]]$Range, [int[]]$Port)
foreach ($r in $Range) {
if ($r -eq '*') { return $true }
if ($r -match '^(\d+)-(\d+)$') {
$low, $high = [int]$Matches[1], [int]$Matches[2]
if ($Port | Where-Object -FilterScript { $_ -ge $low -and $_ -le $high }) { return $true }
}
elseif ([int]$r -in $Port) { return $true }
}
$false
}
function Get-DeterministicFinding {
param([Parameter(Mandatory)][pscustomobject]$Rule)
if ($Rule.Direction -ne 'Inbound' -or $Rule.Access -ne 'Allow') { return @() }
if (-not ($Rule.Source | Where-Object -FilterScript { $_ -in $internetSources })) { return @() }
if ('*' -in $Rule.DestinationPort) { 'All ports open inbound from the internet' }
elseif (Test-PortMatch -Range $Rule.DestinationPort -Port $managementPorts) { 'Management port open inbound from the internet' }
}
$config = Get-Content -Path $Path -Raw | ConvertFrom-Json -AsHashtable
$rules = foreach ($nsg in $config.networkSecurityGroups) {
$subnets = @($nsg.properties.subnets | ForEach-Object -Process { ($_.id -split '/')[-1] })
$nics = @($nsg.properties.networkInterfaces | ForEach-Object -Process { ($_.id -split '/')[-1] })
foreach ($r in $nsg.properties.securityRules) {
$p = $r.properties
$rule = [pscustomobject]@{
Nsg = $nsg.name
Rule = $r.name
Priority = $p.priority
Direction = $p.direction
Access = $p.access
Protocol = $p.protocol
Source = @(Get-RuleValue -Single $p.sourceAddressPrefix -Plural $p.sourceAddressPrefixes)
Destination = @(Get-RuleValue -Single $p.destinationAddressPrefix -Plural $p.destinationAddressPrefixes)
DestinationPort = @(Get-RuleValue -Single $p.destinationPortRange -Plural $p.destinationPortRanges)
Description = $p.description
}
$rule | Add-Member -NotePropertyName HardFindings -NotePropertyValue @(Get-DeterministicFinding -Rule $rule)
$rule | Add-Member -NotePropertyName State -NotePropertyValue ([ordered]@{
nsg = [ordered]@{
name = $nsg.name
associated_subnets = @($subnets | ForEach-Object -Process {
[ordered]@{ name = $_; purpose = $config.subnetPurpose[$_] ?? 'unknown' } })
associated_nics = $nics
}
rule = [ordered]@{
name = $rule.Rule
priority = $rule.Priority
direction = $rule.Direction
access = $rule.Access
protocol = $rule.Protocol
source = $rule.Source
destination = $rule.Destination
destination_ports = $rule.DestinationPort
description = $rule.Description ?? '(none)'
}
nsg_standard = $standard
})
$rule
}
}
Now the Jev part. We need to setup the standard for NSG rules.
$standard = @'
NSG STANDARD (sample)
- Management ports (22, 3389, 5985, 5986) are only allowed from the Azure Bastion subnet or the snet-mgmt subnet.
- No inbound allow rule may use source * or Internet, except on subnets explicitly marked as public-facing, and only for 80/443.
- Data subnets accept traffic only from the application subnets of the same landing zone, on the data ports they need.
- Use specific ports; port ranges wider than 10 ports need a documented reason in the rule description.
- Prefer Application Security Groups or specific subnet prefixes over the VirtualNetwork service tag.
- Every custom rule must have a description stating its purpose and owner.
- Temporary rules must include an expiry date in the description.
'@
$questions = @(
New-JevQuestion -Name intent -Type Choice `
-Instructions 'What is `rule` most likely for, given the purpose of the subnets in `nsg`?' `
-Criteria ([ordered]@{
app_traffic = 'Application traffic between tiers or from clients'
management_access = 'Administrative access (SSH, RDP, WinRM, Bastion)'
platform_dependency = 'Platform needs: DNS, identity, monitoring, backup, load balancer probes'
vendor_or_partner = 'Access for a third party, vendor or partner network'
temporary_or_test = 'Troubleshooting, test, migration or other temporary access'
unclear = 'Purpose cannot be determined from the rule and its context'
})
New-JevYesNoQuestion -Name compliant `
-Question 'Does `rule` comply with every applicable point of `nsg_standard`?' `
-TrueCriteria 'No point of the standard is violated by the rule as configured' `
-FalseCriteria 'At least one point of the standard is violated'
New-JevYesNoQuestion -Name broader_than_needed `
-Question 'Is the source, destination or port scope of `rule` broader than its apparent purpose requires?' `
-TrueCriteria 'All ports for a web tier, VirtualNetwork tag where one subnet would do, large CIDR, * source' `
-FalseCriteria 'Specific ports, specific subnets or ASGs matching the purpose'
New-JevYesNoQuestion -Name looks_temporary `
-Question 'Does the name or description of `rule` suggest a temporary, test or troubleshooting rule?' `
-TrueCriteria 'Words like temp, test, debug, troubleshoot, migration, POC, a ticket number, or a past date' `
-FalseCriteria 'Name and description describe a permanent, owned purpose'
New-JevYesNoQuestion -Name description_mismatch `
-Question 'Is the description of `rule` missing, or does it contradict what the rule actually allows?' `
-TrueCriteria 'No description, or the description names other ports, sources or purposes than configured' `
-FalseCriteria 'The description accurately states what the rule allows and why'
New-JevQuestion -Name severity -Type Score `
-Instructions 'What is the security severity of `rule` as currently configured?' `
-Criteria @(
'None: correct and tightly scoped'
'Low: hygiene or documentation issue'
'Moderate: broader than needed, no internet exposure'
'High: exposes a sensitive subnet or breaks segmentation'
'Critical: direct internet exposure of sensitive services'
)
)
Then we can invoke Jev for each rules
foreach ($rule in $rules) {
$a = (Invoke-Jev -State $rule.State -Question $questions -Mock:$Mock -Raw).answers
$action = if ($rule.HardFindings.Count) { 'security-ticket-p1' } # deterministic, never overridden
elseif ($a.severity.score -ge 3) { 'security-review' }
elseif ($a.intent.confidence -lt 0.5) { 'human-review' }
elseif ($a.looks_temporary.noul -gt 0.7) { 'owner-confirm-or-remove' }
elseif ($a.broader_than_needed.noul -gt 0.7) { 'tighten-rule' }
elseif ($a.compliant.noul -lt 0.3) { 'non-compliant-owner-ticket' }
elseif ($a.description_mismatch.noul -gt 0.7) { 'fix-description' }
else { 'ok' }
$findings = @(
$rule.HardFindings
if ($a.compliant.noul -lt 0.3) { 'Not compliant with NSG standard' }
if ($a.broader_than_needed.noul -gt 0.7) { 'Scope broader than needed' }
if ($a.looks_temporary.noul -gt 0.7) { 'Looks temporary' }
if ($a.description_mismatch.noul -gt 0.7) { 'Description missing or misleading' }
)
[pscustomobject]@{
Nsg = $rule.Nsg
Rule = $rule.Rule
Action = $action
Severity = if ($rule.HardFindings.Count) { 4 } else { [math]::Round($a.severity.score, 2) }
Findings = $findings -join '; '
Intent = $a.intent.choice
Compliant = [math]::Round($a.compliant.noul, 2)
Ports = $rule.DestinationPort -join ','
Source = $rule.Source -join ','
}
}
Result
Nsg : nsg-vm-legacy-01
Rule : allow-winrm
Action : security-review
Severity : 3.08
Findings : Not compliant with NSG standard
Intent : vendor_or_partner
Compliant : 0.06
Ports : 5985
Source : 203.0.113.0/24
Nsg : nsg-vnet-spoke
Rule : allow-ssh-any
Action : security-ticket-p1
Severity : 4
Findings : Management port open inbound from the internet; Not compliant with NSG standard; Scope broader than needed
Intent : management_access
Compliant : 0.02
Ports : 22
Source : *
Nsg : nsg-vnet-spoke
Rule : allow-port-range
Action : tighten-rule
Severity : 2.1
Findings : Not compliant with NSG standard; Scope broader than needed
Intent : app_traffic
Compliant : 0.06
Ports : 1024-65535
Source : 10.2.1.0/24
Using Azure Policy Linter to control policy definition
Last example, working with Azure Policy definition. For testing definition, we first need a linter.
The result.json file contains the finding of the linter on specific definition part with a severity (error/warning/informational).
The rule for a decision model is to turn these finding into actionable items.
First step, get the content for each policy and run the linter.
$resultFile = Join-Path -Path $Path -ChildPath 'PolicyResult.json'
$files = Get-ChildItem -Path $Path -Filter '*.json' -File | Where-Object -Property Name -NE -Value 'PolicyResult.json'
$null = policylinter $files.FullName -o $resultFile
if ($LASTEXITCODE -ne 0) { throw "policylinter failed with exit code $LASTEXITCODE" }
$lint = @{}
(Get-Content -Path $resultFile -Raw | ConvertFrom-Json -AsHashtable).GetEnumerator() | ForEach-Object {
$lint[(Split-Path -Path $_.Key -Leaf)] = @($_.Value)
}
Then the Jev part.
$questions = @(
New-JevYesNoQuestion -Name findings_matter `
-Question 'Does at least one item in `linter_findings` mean `policy` may not evaluate or enforce as intended?' `
-TrueCriteria 'A finding changes which resources match: wrong array logic, missing alias, broken condition' `
-FalseCriteria 'Findings are style, old API versions, optional properties or best-practice hints, or there are no findings'
New-JevYesNoQuestion -Name description_matches `
-Question 'Do displayName and description of `policy` match what policyRule really evaluates and its default effect?' `
-TrueCriteria 'Same resource types, same condition, same effect as the rule' `
-FalseCriteria 'The text promises other resource types, another condition or a stronger effect than the rule'
New-JevYesNoQuestion -Name weakens_security `
-Question 'Could assigning `policy` weaken security?' `
-TrueCriteria 'The remediation opens network access, uses hard-coded public IP ranges, or grants a broader role than needed (Owner, Contributor)' `
-FalseCriteria 'The policy only audits or denies, or remediates with a least-privilege role'
)
For each policy, Jev is called
foreach ($file in $files) {
$definition = (Get-Content -Path $file.FullName -Raw | ConvertFrom-Json).properties
$findings = $lint[$file.Name]
$state = [ordered]@{
policy = [ordered]@{
displayName = $definition.displayName
description = $definition.description
parameters = $definition.parameters
policyRule = $definition.policyRule
}
linter_findings = @($findings | ForEach-Object { "$($_.severity) $($_.ruleIdentifier) at $($_.path)" })
}
$a = (Invoke-Jev -State $state -Question $questions -Mock:$Mock -Raw).answers
$action = if ($a.weakens_security.noul -gt 0.7) { 'Security review' }
elseif ($a.findings_matter.noul -gt 0.7) { 'Fix the rule' }
elseif ($a.description_matches.noul -lt 0.3) { 'Fix the description' }
elseif ($findings.Count -gt 0) { 'Backlog' }
else { 'OK' }
[pscustomobject]@{
Policy = $file.Name
Warnings = @($findings | Where-Object -Property severity -EQ -Value 'Warning').Count
Informational = @($findings | Where-Object -Property severity -EQ -Value 'Informational').Count
FindingsMatter = [math]::Round($a.findings_matter.noul, 2)
DescriptionMatches = [math]::Round($a.description_matches.noul, 2)
WeakensSecurity = [math]::Round($a.weakens_security.noul, 2)
Action = $action
}
}
The result:
Policy : policy4.json
Warnings : 1
Informational : 0
FindingsMatter : 0.28
DescriptionMatches : 0.68
WeakensSecurity : 0.87
Action : Security review
These are a few examples of what a decision model can do in cloud engineering. Decision model can do more. Our daily work generates a constant stream of data: metrics, logs, IaC plans, traces, network routes. Most of it contains noise, and we already rely on scripts and observability tools to turn it into tasks. Jev fits right into that chain. It applies judgment to your data, against your rules, and leaves the final decision in your code.
Volume is also why a general-purpose LLM is often too expensive for this kind job. Jev isn't: everything in this article used 103,525 tokens, for a total of $0.0039.
You can find the code of this post here
Top comments (0)