Detecting Dangerous Terminal Commands Before Sending Them to an AI — My Safety Layer

#ai #gemini #rust #security

If this is useful, a ❤️ helps others find it.

All tests run on an 8-year-old MacBook Air.

HiyokoHelper sends terminal errors to Gemini for diagnosis. But what if the user pastes rm -rf / or a fork bomb?

Two problems: wasting API quota on non-errors, and potentially getting an "explanation" of a command that could destroy the system.

Here's the safety layer I built.

Layer 1: Is it actually dangerous?

#[derive(Debug, PartialEq)]
pub enum SafetyLevel {
    Safe,
    Warning(String),
    Danger(String),
}

pub fn assess_safety(input: &str) -> SafetyLevel {
    let input_lower = input.to_lowercase();

    let critical = [
        ("rm -rf /", "This deletes your entire filesystem."),
        ("rm -rf ~", "This deletes your home directory."),
        ("rm -rf *", "This deletes everything in the current directory."),
        ("mkfs", "This formats a disk, erasing all data."),
        ("dd if=/dev/zero", "This overwrites a disk with zeros."),
        (":(){:|:&};:", "This is a fork bomb — it will crash your system."),
        ("chmod -R 777 /", "This removes all security from your filesystem."),
    ];

    for (pattern, reason) in &critical {
        if input_lower.contains(pattern) {
            return SafetyLevel::Danger(reason.to_string());
        }
    }

    let warnings = [
        ("sudo rm", "This runs a deletion command as administrator."),
        ("sudo chmod", "This changes file permissions as administrator."),
        ("kill -9", "This forcefully terminates a process."),
        ("pkill", "This kills processes by name."),
    ];

    for (pattern, reason) in &warnings {
        if input_lower.contains(pattern) {
            return SafetyLevel::Warning(reason.to_string());
        }
    }

    SafetyLevel::Safe
}

Layer 2: Different UI for each level

Danger → block entirely. Warning → user decides. Safe → proceed silently.

function SafetyBanner({ level }: { level: SafetyResult }) {
  if (level.type === 'danger') {
    return (


        ⚠️ 危険なコマンドが含まれています

{level.reason}



AIへの送信をブロックしました。




    );
  }
  if (level.type === 'warning') {
    return (


        🔶 注意: {level.reason}
        表示する


    );
  }
  return null;
}

Layer 3: sudo gets automatic explanation

pub fn detect_mode(input: &str) -> DiagnosisMode {
    let trimmed = input.trim();
    if trimmed.starts_with("sudo ") && !trimmed.contains('\n') {
        return DiagnosisMode::CommandExplain;
    }
    if looks_like_error(input) {
        return DiagnosisMode::ErrorDiagnosis;
    }
    DiagnosisMode::CommandExplain
}