DEV Community

Cover image for Detecting Dangerous Shell Commands in Rust — Building a Safety Layer
hiyoyo
hiyoyo

Posted on

Detecting Dangerous Shell Commands in Rust — Building a Safety Layer

All tests run on an 8-year-old MacBook Air.
All results from shipping 7 Mac apps as a solo developer. No sponsored opinion.
HiyokoHelper explains terminal commands and warns when they're dangerous. Building the danger detection layer was more interesting than I expected.

Why bother with local detection
The obvious approach: send the command to Gemini, ask if it's dangerous.
The problem: you want the warning to appear before the user runs the command — ideally as they're looking at it. Gemini latency (even fast) adds friction. A local check is instant.
Use local detection for obvious cases. Use Gemini for explanation and nuance.

What makes a command dangerous
The obvious patterns:
rustconst DANGEROUS_PATTERNS: &[&str] = &[
"rm -rf",
"rm -r /",
"> /dev/sda",
"dd if=",
":(){ :|:& };:", // fork bomb
"chmod -R 777 /",
"mv /* /dev/null",
"> /etc/passwd",
"curl | bash",
"wget | sh",
"sudo rm",
];

fn is_dangerous(command: &str) -> bool {
let lower = command.to_lowercase();
DANGEROUS_PATTERNS.iter().any(|p| lower.contains(p))
}
This catches the classics. It doesn't catch clever obfuscations. That's fine — you're building a helper, not a security product.

The severity levels
Not all dangerous commands are equally dangerous. A warning system with levels is more useful than binary safe/unsafe:
rust#[derive(Debug, Serialize)]
pub enum DangerLevel {
Safe,
Caution, // Potentially destructive, easily recoverable
Warning, // Destructive, hard to recover
Critical, // Irreversible, system-level impact
}
sudo rm -rf ~/Documents is Warning. rm -rf / is Critical. Different UI treatment. Different user friction.

What Gemini adds
Local detection catches patterns. Gemini catches context.
"Delete all log files older than 30 days" looks safe as a pattern. In the wrong directory with wrong permissions, it isn't. Gemini can reason about context in a way pattern matching can't.
The prompt I use:
Analyze this shell command for safety issues.
Command: {command}
Rate safety: safe / caution / warning / critical
Explain in one sentence what it does.
Explain in one sentence any risks.
Respond as JSON only.
Local check first for instant feedback. Gemini analysis for explanation. Two-layer approach.

The UX
Show danger level inline, next to the command. Don't block. Don't require confirmation. Just inform.
Users who copy dangerous commands from Stack Overflow often don't know they're dangerous. A clear, non-blocking warning before they run it is genuinely useful. A modal that blocks them is annoying.

If this was useful, a ❤️ helps more than you'd think — thanks!
Hiyoko PDF Vault → https://hiyokoko.gumroad.com/l/HiyokoPDFVault
X → @hiyoyok

Top comments (0)