DEV Community

gaurav kundu
gaurav kundu

Posted on

Why SOC analysts get inconsistent results from ChatGPT (and how structured workflows fix it)

If you've ever handed a security alert to ChatGPT and gotten a different answer each time — you've hit the real problem.

It's not the model. It's the prompt.

Most analysts paste an alert and ask "what do you think?" That's like asking a junior analyst to investigate without a runbook. You'll get something back, but the quality depends entirely on how the question was framed.

The real problem: no structure

Experienced SOC analysts don't wing investigations. They follow a process:

  • Triage the alert
  • Map to MITRE ATT&CK
  • Check for lateral movement
  • Build a containment recommendation
  • Write a ticket summary

The issue is that most AI-assisted workflows skip steps 2–5 and jump straight to "is this bad?"

What I built

I spent time building SOC.Workflows — a free collection of structured investigation workflows for SOC analysts. Each workflow breaks an investigation into 4 steps, with specific prompts for each step, designed to run in ChatGPT or Claude.

Current workflows:

  • Phishing Email Investigation
  • AWS VPC Flow Log Analysis
  • PowerShell & Script Analysis
  • Credential Dumping Investigation
  • Ransomware Triage
  • Identity Compromise Investigation
  • URL & Domain Analysis
  • SOC Alert Triage
  • Explain This Alert

How it works

  1. Pick a workflow matching your alert type
  2. Copy the workflow prompt
  3. Paste into ChatGPT or Claude
  4. Get structured, step-by-step analysis

No login. No setup. No API keys.

Why structure matters

When I ran the same phishing alert through an unstructured prompt vs. the structured workflow, the difference was clear:

Unstructured: "This looks like a phishing email. Check the sender domain."

Structured: SPF/DKIM validation → header analysis → sender reputation → verdict with risk score → recommended response actions

Same model. Completely different output quality.

Try it

If you work in a SOC or do blue team work, I'd love feedback on which investigation types are missing.

👉 socworkflows.com — free, no login required

Top comments (0)