My PowerShell 7 prompt is where I already live. Putting the LLM there means zero context switching and zero cloud.
TL;DR
I put an LLM directly into my PowerShell 7 profile so I can talk to it without leaving the terminal. The result is a terminal-native AI ops companion: system recon, curated news, command authoring and review, all invoked like any other command. The model runs locally through Ollama on loopback, so nothing leaves the machine. Every action the model suggests waits for a human Y/N. This post shows the integration, the use cases, a side-by-side comparison of hardware and model tiers, the trade-offs, and the guardrails that make it safe.
The Problem
The terminal is where I already work. Switching to a browser, a chat window, or a separate app to ask an AI a question breaks flow and loses context - the model doesn't know what's in my clipboard, my current directory, or my last error.
Three things made the obvious approaches fail:
- Cloud chat is disqualified for sensitive work. Pasting system output, file paths, or secrets into a browser is a non-starter on locked-down or air-gapped machines.
-
Copy-paste round trips lose fidelity. By the time I paste
Get-Processoutput into a chat and get an answer back, the moment has passed and the context is stale. -
An agent with shell access and no gate is a liability. Anything that can run
Remove-ItemorInvoke-WebRequestneeds a human in the loop, not a system prompt that says "be careful".
What I Built
A PowerShell 7 profile that wires an LLM into the terminal as a set of first-class commands: ask, fix, brief, mem, news. The model (Ollama, loopback only) receives a curated context packet - time, health, cwd, $Error[0], git status, recent history - and answers in place. Every destructive suggestion is displayed, never executed. Mutating cmdlets in the full module require -Confirm/ShouldProcess and are logged.
Stack: PowerShell 7.2+ (7.6 LTS recommended), Ollama @ http://127.0.0.1:11434, nomic-embed-text for local vector memory, Pester 5+ for tests.
Getting Started: Wiring an LLM into Your Profile
The integration point is your PowerShell profile ($PROFILE). The goal: a function that takes a prompt, assembles context, calls the model, and prints the answer - callable like any cmdlet.
# ~/.config/powershell/Microsoft.PowerShell_profile.ps1 (or $PROFILE)
function Invoke-OpsAI {
param(
[Parameter(Mandatory, Position = 0, ValueFromRemainingArguments)]
[string[]]$Prompt,
[switch]$Stream
)
$text = $Prompt -join ' '
$ctx = Get-AmbientContext # time, health, cwd, $Error[0], git, history
$body = @{
model = 'qwen2.5:7b'
prompt = "$ctx`n`nUser: $text"
stream = $Stream.IsPresent
} | ConvertTo-Json -Depth 6
Invoke-RestMethod -Uri 'http://127.0.0.1:11434/api/generate' `
-Method POST -Body $body -ContentType 'application/json'
}
Set-Alias -Name ask -Value Invoke-OpsAI
Three rules make this usable rather than fragile:
-
Loopback only. The endpoint is
127.0.0.1. No DNS, no TLS, no egress. If the model isn't local, the command fails closed. - Context is assembled by code, not by the model. The ambient packet is structured PowerShell output, not a dump of raw terminal scrollback.
-
Output is structured. The function returns a
[PSCustomObject]you can pipe, format, or convert to JSON - not a wall of text.
Architecture
Ports and adapters is doing real work here: the AI adapter is one adapter among five, so "use the model" is never a special path. It receives the same envelope as everything else. Swap Ollama for another backend and nothing else changes.
Engineering Philosophy
The design rests on a handful of principles, each visible in the sections above:
- The model is the least trusted component. Code assembles context; humans gate actions. The model never runs a command, never reads a secret, and never touches the system directly - it only reads a sanitized snapshot and proposes.
- Put the AI where you already work. The terminal is home. Zero context switching, zero copy-paste round trips, zero stale context.
- Loopback only. No egress, no cloud model for sensitive work. If the model isn't local, the command fails closed.
- Ports and adapters. The AI adapter is one adapter among five. Swap the backend and nothing else changes; the envelope is identical.
-
Structured over raw.
[PSCustomObject]output pipes, formats, and converts. Raw terminal dumps waste context and bury signal. - Human-readable by default. Tables, color, truncation, and personal style. The tool adapts to taste, not the other way around.
-
Validate, escape, mask, gate. Every input, every output, every action. No
Invoke-Expressionon model output, ever. - Personalize the harness. Aliases, themes, verbosity, and guardrails live in config. The setup reflects how you work, not a generic template.
Daily Drivers
1. Curated news from favorite topics
A daily news command fetches RSS/JSON feeds, filters by topic keywords, and pipes the result through the model for a digest. Parameterization matters here - topic, source, and count are validated before any network call.
function Get-CuratedNews {
param(
[ValidateSet('ai','powershell','security','devtools')]
[string]$Topic = 'ai',
[ValidateRange(1,20)]
[int]$Count = 5,
[string]$FeedUrl = 'https://example.com/feed.xml'
)
# Validate URL scheme before fetching
if ($FeedUrl -notmatch '^https://') { throw 'Feed URL must be HTTPS' }
$items = Invoke-RestMethod -Uri $FeedUrl -TimeoutSec 10
$filtered = $items | Where-Object { $_.title -match $Topic } | Select-Object -First $Count
$prompt = "Summarize these headlines in 5 bullets, neutral tone:`n" + ($filtered | ConvertTo-Json)
Invoke-OpsAI $prompt
}
Set-Alias -Name news -Value Get-CuratedNews
function Add-MemoryNote {
param([Parameter(Mandatory, ValueFromRemainingArguments)][string[]]$Note)
$text = $Note -join ' '
$embedding = Invoke-RestMethod -Uri 'http://127.0.0.1:11434/api/embeddings' `
-Method POST -Body (@{ model = 'nomic-embed-text'; prompt = $text } | ConvertTo-Json) `
-ContentType 'application/json'
# Persist note + embedding to Memory/ (cosine search via Search-MemoryNote)
[pscustomobject]@{ note = $text; vector = $embedding.embedding; saved = (Get-Date) }
}
Set-Alias -Name mem -Value Add-MemoryNote
Security notes: validate the URL scheme, cap the count, validate the topic against an allow-list, set a timeout, and never interpolate user input directly into a shell string. Prefer Invoke-RestMethod with a parameter object over curl string building.
2. System recon and analysis
brief gathers system state - processes, services, disk, network, firewall, Defender - and pipes it through the model for a structured analysis with recommendations.
function Invoke-SystemBrief {
$snapshot = [ordered]@{
OS = (Get-CimInstance Win32_OperatingSystem).Caption
CPU = (Get-CimInstance Win32_Processor).Name
MemGB = [math]::Round((Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory/1GB,1)
DiskFree = (Get-PSDrive C).Free
TopProc = Get-Process | Sort-Object CPU -Descending | Select-Object -First 5 Name,CPU
Services = Get-Service | Where-Object Status -eq 'Running' | Select-Object -First 10 Name
Listeners= Get-NetTCPConnection -State Listen | Select-Object LocalPort,OwningProcess
}
$prompt = "Analyze this system snapshot. Flag anomalies, security risks, and 3 improvements:`n" + ($snapshot | ConvertTo-Json -Depth 4)
Invoke-OpsAI $prompt
}
Set-Alias -Name brief -Value Invoke-SystemBrief
The model never touches the system directly - it only reads a sanitized snapshot. Secrets in environment variables or config files are masked (Protect-OpsSensitiveText) before the packet is built.
3. Command authoring and review
Two modes: send a command to the model for review, or ask in plain language for a command.
# Review a command before running it
function Test-CommandSafety {
param([Parameter(Mandatory)][string]$Command)
$prompt = "Review this PowerShell command for correctness and risk. Reply with: SAFE/REVIEW/UNSAFE, reason, and a corrected version if needed:`n$Command"
Invoke-OpsAI $prompt
}
Set-Alias -Name fix -Value Test-CommandSafety
# Ask for a command in plain language
function New-CommandFromIntent {
param([Parameter(Mandatory, ValueFromRemainingArguments)][string[]]$Intent)
$ctx = "PowerShell 7.6, modules: Az, Pester, PowershellOps. Aliases: ask, fix, brief, mem, news."
$prompt = "$ctx`n`nWrite a PowerShell command or script for: $($Intent -join ' '). Output only code."
Invoke-OpsAI $prompt
}
Set-Alias -Name gen -Value New-CommandFromIntent # generate a command from plain language
The harness supplies the model with the exact PowerShell version, installed modules, aliases, and guardrails - so suggestions match my environment instead of generic StackOverflow snippets.
Side-by-Side: Hardware and LLM Tiers
Not every machine can run every model. Pick the tier that fits the hardware, then accept the trade-offs.
| Tier | Hardware | Model | Latency | Quality | Use |
|---|---|---|---|---|---|
| Nano | 8 GB RAM, no GPU |
phi3:mini (3.8B) |
<1s | Basic | Quick lookups, command syntax |
| Small | 16 GB RAM, CPU |
qwen2.5:7b (7B) |
2-4s | Good | Daily driver: news, brief, fix |
| Medium | 32 GB RAM + GPU (8GB VRAM) |
qwen2.5:14b (14B) |
1-2s | Strong | Deep system analysis |
| Large | 48 GB+ VRAM (or 64 GB unified) |
llama3.1:70b (70B, Q4) |
5-15s | Excellent | Complex reasoning, code review |
| Cloud | Any + egress | GPT-4 / Claude | 1-3s | Best | Disqualified for sensitive/air-gapped work |
Trade-offs:
- Smaller models are fast and fit on laptops but hallucinate on complex multi-step reasoning and long context.
- Larger models reason better but need VRAM; CPU-only inference at 70B is unusable (minutes per response).
- Quantization (Q4_K_M) halves VRAM at ~2-3% quality loss - worth it on constrained hardware.
- Context window scales with model: 4K (nano) → 32K (medium) → 128K (large). The ambient packet must fit; trim aggressively on small tiers.
Recommendation: start at Small (qwen2.5:7b on 16 GB). It handles 90% of daily tasks. Upgrade only when you hit a quality wall on deep analysis.
Ollama vs llama.cpp
Both run local models on your own hardware. The choice is about ergonomics vs control.
| Dimension | Ollama | llama.cpp |
|---|---|---|
| Setup | One-line install, ollama pull, ollama run. Zero config. |
Build from source or download binaries; manage GGUF files and flags manually. |
| API | Clean HTTP REST (/api/generate, /api/chat, /api/embeddings). Stable contract. |
OpenAI-compatible server (llama-server) plus raw llama-cli. More flags, more power. |
| Model mgmt | Built-in registry (ollama pull qwen2.5:7b). Auto-downloads, checksums, caching. |
You fetch GGUFs yourself (Hugging Face). You own versioning. |
| Quantization | Picks sensible defaults; supports Q4/Q5/Q8 via Modelfiles. | Full control: every quant type, custom splits, GPU layer offload (-ngl), thread tuning. |
| Performance | Good defaults, slightly conservative. | Often faster on the same hardware with tuned -ngl/-t/-b. |
| Modelfiles / system prompts | First-class. Bake a system prompt + params into a named model (ollama create mymodel -f Modelfile). |
System prompt via flag; less ergonomic for named personas. |
| Embeddings |
/api/embeddings out of the box. |
llama-server --embedding, or separate tool. |
| Ecosystem | Huge; most tools (LangChain, Open WebUI, Continue) target it. | Lower-level; more DIY wiring. |
| Best for | Daily driver, profile integration, stable contract. | Benchmarking, squeezing max throughput, exotic quants, air-gapped custom builds. |
Recommendation: start with Ollama for the profile integration - the REST contract is what Invoke-OpsAI calls, and Modelfiles let you bake the system prompt and guardrails into a named model. Move to llama.cpp only when you need to tune throughput on a specific GPU or run a quant Ollama doesn't package.
Mitigating Hallucination: Specialization Over Size
The brute-force answer to hallucination is a bigger model. The better answer is a smaller, specialized one used in a narrower way. The LLM in this setup is not a general-purpose assistant - it is a PowerShell-terminal specialist. That constraint is what makes small models reliable.
Principles:
-
Narrow the task, don't widen the model.
ask "explain Get-Process"is a documentation lookup.briefis a structured analysis over a fixed snapshot.fixis a command review. Each call has a bounded input and a bounded output. Small models excel at bounded tasks; they fail on open-ended reasoning. - Ground every call in structured data. The ambient packet is deterministic PowerShell output, not free-form memory. The model summarizes and flags; it never invents the underlying facts.
- Specialize the prompt, not just the model. The system prompt injects the PowerShell version, installed modules, aliases, and guardrails. A 7B model with the right context outperforms a 70B model guessing at your environment.
- Finetune or train when the task is stable. If a task repeats (e.g., "review this command for risk"), a small finetuned model on that exact distribution beats a general model 10x its size. Training data is cheaper than GPU hours for a fixed task.
- Constrain the output. Ask for a specific shape - "Reply with: SAFE/REVIEW/UNSAFE, reason, corrected version." Constrained decoding and format instructions reduce hallucination more than extra parameters.
- Show sources. When the model cites a cmdlet or flag, require it to echo the input it based the answer on. If it can't, treat it as unverified.
- Human in the loop for the last mile. The model proposes; the human disposes. Hallucination that reaches a human is a nuisance; hallucination that executes is an incident.
The balance: bigger models buy you generality. Specialization - narrow tasks, grounded context, finetuned weights, constrained output, human gates - buys you reliability at a fraction of the cost. For terminal ops, reliability per watt beats raw capability. A 7B model that never hallucinates on Get-Process is more useful than a 70B model that sometimes invents flags.
Risks and Mitigation
| Risk | Mitigation |
|---|---|
| Command injection via model output | Never Invoke-Expression on model output. Display suggestions; user copies or explicitly confirms. |
| Secret leakage in context |
Protect-OpsSensitiveText masks tokens, keys, passwords before the packet is built. |
| Prompt injection via fetched content | Treat fetched web content as data, not instructions. Wrap in delimiters; instruct model to summarize only. |
| Unbounded network calls | Validate URL scheme (HTTPS only), set timeouts, cap payload size. |
| Destructive actions | Suggestions are displayed, never executed. Mutating cmdlets require -Confirm/ShouldProcess; logged to local audit file. |
| Model hallucination | Ground answers in structured data, not free-form memory. Show sources in output. |
| Egress / data exfiltration | Loopback endpoint only; no cloud model for sensitive work. |
| Stale context | Rebuild the ambient packet per invocation; don't cache across calls. |
Human-Readable Formatting and Personal Style
Raw PowerShell output is hostile to both humans and models. The profile applies a consistent formatting layer:
-
Tables for lists (
Format-Table -AutoSize), cards for single objects, JSON for piping. -
Color coding: errors red, warnings yellow, success green - via
$PSStyle(PS 7.2+). - Truncation: cap output at N lines before sending to the model; summarize the rest.
-
Personal style: aliases (
ask,fix,gen,brief,mem,news), prompt theme, and output verbosity are all configurable inops-settings.json. The harness adapts to taste - verbose for learning, terse for ops.
# ops-settings.json
{
"verbosity": "terse",
"colorTheme": "dark",
"maxContextLines": 40,
"defaultModel": "qwen2.5:7b",
"confirmDestructive": true
}
The Harness: Context, Guardrails, and Control
The model is the least trusted component. The harness around it does the work:
-
Context assembly -
Get-AmbientContextbuilds a structured packet: timestamp, OS health, cwd, last error, git status, recent command history. This is deterministic code, not model inference. - Command/alias/docs injection - the system prompt includes the PowerShell version, installed modules, available aliases, and relevant help topics so suggestions are environment-accurate.
-
Guardrails -
- No
Invoke-Expressionon model output. - All mutating cmdlets require
-Confirm. - Sensitive text masked before context assembly.
- URL scheme validation and timeouts on all network calls.
- Model output is displayed, never executed automatically.
- No
-
Audit trail - every AI interaction and every confirmed action is logged to
Reports/with timestamp, prompt, and result.
Quick Start
- Install Ollama; pull a model:
ollama pull qwen2.5:7b. - Add
Invoke-OpsAIand aliases to$PROFILE. - Create
ops-settings.jsonwith your preferences. - Add
Get-AmbientContextandProtect-OpsSensitiveTexthelpers. - Test:
ask "explain Get-Process"andbrief.
Verification
- Loopback check:
Test-NetConnection 127.0.0.1 -Port 11434returns open. - No egress: disconnect network;
askandbriefstill work. - Secret masking: set
$env:SECRET_TOKEN = 'abc123'; runbrief; confirm token absent from logged packet. - Gate check:
fix "Remove-Item C:\temp\*"returns a suggestion but does not execute. - Pester:
Invoke-Pester→ all tests green.
Takeaways
- Put the LLM where you already work - the terminal. Zero context switching.
- The model is the least trusted component. Code assembles context; humans gate actions.
- Loopback only. No egress for sensitive work.
- Route by hardware: nano for laptops, small for daily drivers, medium/large for deep analysis.
- Validate, escape, mask, and gate - every input, every output, every action.
- Personalize: aliases, themes, verbosity. The harness adapts to you.



Top comments (0)