Polliog

Posted on Dec 9

Understanding SIEM: Security Information and Event Management

#security #cybersecurity #resources

What is a SIEM?

SIEM (Security Information and Event Management) is a security solution that provides real-time analysis of security alerts generated by applications and network hardware.

Think of it as a security operations center in software form—a system that continuously monitors your infrastructure, detects threats, and helps you respond to incidents before they cause damage.

The Core Problem SIEM Solves

Modern IT environments generate millions of log events per day:

Authentication attempts (successful and failed)
Network connections
File access and modifications
Application errors and warnings
System configuration changes
API calls and database queries

The challenge: How do you identify the 5 malicious events hidden among 5 million legitimate ones?

This is where SIEM comes in.

How SIEM Works: The Four Pillars

1. Collection (Data Aggregation)

A SIEM collects logs from every source in your infrastructure:

Servers: Linux syslog, Windows Event Logs
Applications: Web servers (Apache, Nginx), databases (PostgreSQL, MySQL)
Network devices: Firewalls, routers, switches
Security tools: Antivirus, IDS/IPS
Cloud services: AWS CloudTrail, Azure Activity Logs, Google Cloud Audit Logs

All these disparate formats are normalized into a unified schema.

2. Correlation (Pattern Detection)

The SIEM engine analyzes incoming logs and correlates events across different sources.

Example correlation rule:

IF multiple_failed_logins(source_ip, count > 10, timeframe = 5min)
AND successful_login(same_source_ip)
THEN alert("Brute Force Attack - Account Compromised")

This detects a classic attack pattern: an attacker tries many passwords (fails), then succeeds.

3. Alerting (Real-Time Notification)

When suspicious patterns are detected, the SIEM triggers alerts via:

Email
SMS
Slack/Discord/Teams
PagerDuty
Webhook integrations

Alerts are prioritized by severity:

Critical: Active attack in progress (e.g., reverse shell detected)
High: Suspicious activity (e.g., privilege escalation attempt)
Medium: Policy violation (e.g., unauthorized file access)
Low: Informational (e.g., config change)

4. Investigation (Forensics & Response)

When an incident occurs, security analysts need to:

Reconstruct the attack timeline: What happened, when, and how?
Identify the blast radius: Which systems were affected?
Determine root cause: How did the attacker get in?

SIEM provides search, filtering, and visualization tools to answer these questions quickly.

MITRE ATT&CK: The Universal Language of Cyber Threats

What is MITRE ATT&CK?

MITRE ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) is a globally recognized framework that documents how real-world attackers operate.

Think of it as a "playbook" that catalogs:

Tactics: The attacker's goals (e.g., "Initial Access", "Persistence", "Exfiltration")
Techniques: The methods used to achieve those goals (e.g., "Phishing", "Credential Dumping", "Data Encrypted for Impact")
Procedures: Specific implementations seen in the wild

The ATT&CK Matrix

The framework is organized as a matrix:

Tactic	Example Techniques
Initial Access	Phishing, Exploit Public-Facing Application, Valid Accounts
Execution	Command-Line Interface, PowerShell, Scheduled Task
Persistence	Create Account, Registry Run Keys, Web Shell
Privilege Escalation	Sudo, Exploitation for Privilege Escalation
Defense Evasion	Clear Command History, Obfuscated Files, Disable Security Tools
Credential Access	Brute Force, Credentials from Files, Keylogging
Discovery	Network Service Scanning, System Information Discovery
Lateral Movement	Remote Services, SSH, RDP
Collection	Data from Local System, Screen Capture
Exfiltration	Exfiltration Over C2 Channel, Data Transfer Size Limits
Impact	Data Encrypted for Ransom, Service Stop, Defacement

Why MITRE ATT&CK Matters for SIEM

Modern SIEMs map detection rules to MITRE ATT&CK techniques. This allows security teams to:

Prioritize defenses: "We detect 80% of Initial Access techniques but only 20% of Defense Evasion."
Communicate clearly: Instead of saying "weird bash command," say "T1070.003: Clear Command History."
Benchmark maturity: Compare your detection coverage against industry standards.

Example Detection Mapping:

title: Clearing Bash History
mitre_attack:
  - tactic: Defense Evasion
    technique: T1070.003 (Indicator Removal: Clear Command History)
description: |
  Attackers often clear shell history to hide their tracks after compromising a system.
detection_logic: |
  Look for commands like:
  - history -c
  - rm ~/.bash_history
  - cat /dev/null > ~/.bash_history

Sigma Rules: The Open Standard for Threat Detection

What is Sigma?

Sigma is a generic and open signature format that allows you to describe log events in a standardized way. Think of it as "YARA for logs" or "Snort rules for SIEM."

Sigma rules are:

Vendor-agnostic: Work across Splunk, Elastic, QRadar, etc.
Human-readable: Written in YAML
Community-driven: Thousands of rules available at github.com/SigmaHQ/sigma

Anatomy of a Sigma Rule

Here's a real-world example:

title: Suspicious Reverse Shell Detection
id: 4d1fa1f6-4c1b-4b8e-a8a9-3f9c8e7d6c5b
status: stable
description: Detects execution of commands commonly used to establish reverse shells
references:
    - https://attack.mitre.org/techniques/T1059/004/
author: Security Researcher
date: 2024/01/15
modified: 2024/12/09
tags:
    - attack.execution
    - attack.t1059.004
logsource:
    product: linux
    category: process_creation
detection:
    selection_bash:
        CommandLine|contains:
            - 'bash -i >& /dev/tcp/'
            - 'nc -e /bin/sh'
            - 'nc -e /bin/bash'
            - '/bin/sh | nc'
            - 'python -c "import socket'
            - 'perl -e "use Socket'
    condition: selection_bash
falsepositives:
    - Legitimate remote administration tools
    - Security testing with permission
level: high

Key Components Explained

1. Metadata

title: Human-readable name
id: Unique identifier (UUID)
status: stable, experimental, or deprecated
tags: MITRE ATT&CK mapping

2. Logsource

product: linux, windows, macos, web, network
category: process_creation, file_event, network_connection
service: sshd, apache, dns

3. Detection Logic

selection: Define what patterns to look for
condition: Boolean logic (AND, OR, NOT)
keywords: Used for quick text matching

4. Context

falsepositives: Known benign scenarios that might trigger this rule
level: critical, high, medium, low, informational

The Power of Sigma

Because Sigma rules are standardized, you can:

Share detections globally: The security community contributes rules for new threats daily
Port between tools: Write once, deploy on any SIEM
Version control: Treat detection logic as code (Git, CI/CD)

Threat Intelligence: Know Your Enemy

What is Threat Intelligence?

Threat Intelligence (TI) is actionable information about current and emerging threats. It helps you answer:

Who is targeting organizations like mine?
What techniques are they using?
Which IPs, domains, and file hashes are known to be malicious?

Types of Threat Intelligence

1. Strategic Intelligence

High-level trends and motivations.

Example: "Ransomware groups are increasingly targeting healthcare in Q4 2024."
Audience: CISOs, executives

2. Tactical Intelligence

Adversary TTPs (Tactics, Techniques, Procedures).

Example: "APT29 uses PowerShell Empire for post-exploitation."
Audience: Security architects, threat hunters

3. Operational Intelligence

Details about specific campaigns.

Example: "Phishing campaign using fake Microsoft 365 login pages targeting EU companies."
Audience: SOC analysts, incident responders

4. Technical Intelligence

Concrete indicators of compromise (IoCs).

Example: "IP 192.0.2.123 is a known C2 server for the Emotet botnet."
Audience: SIEM systems, firewalls

Indicators of Compromise (IoCs)

IoCs are forensic artifacts that indicate a security incident:

Type	Example	Description
IP Address	192.0.2.123	Known malicious server
Domain	evil-phishing.com	Phishing or C2 domain
URL	http://malware.example/payload.exe	Direct link to malware
File Hash (MD5/SHA256)	a1b2c3d4e5...	Unique malware signature
Email Address	attacker@example.com	Source of phishing emails
User-Agent	"BadBot/1.0"	Suspicious HTTP client
File Path	/tmp/.hidden_backdoor	Unusual file location

Threat Intelligence Feeds

Organizations subscribe to feeds that provide real-time IoCs:

Free/Open-Source Feeds:

AbuseIPDB: Crowdsourced database of malicious IPs
URLhaus: Malware distribution URLs
PhishTank: Verified phishing sites
AlienVault OTX: Open threat exchange
Tor Exit Nodes: List of Tor anonymization endpoints

Commercial Feeds:

Recorded Future
Mandiant Threat Intelligence
CrowdStrike Falcon Intelligence
Palo Alto Networks Unit 42

How SIEM Uses Threat Intelligence

When integrated with a SIEM, threat intelligence enables automatic enrichment:

Log arrives: Connection from 198.51.100.42 to server
SIEM checks IoC database: Is this IP known to be malicious?
Match found: 198.51.100.42 is listed in AbuseIPDB (reported 47 times for brute force attacks)
Alert triggered: "Connection from Known Malicious IP (Confidence: High)"

This transforms raw logs into actionable security insights.

Real-World SIEM Use Cases

Use Case 1: Detecting a Ransomware Attack

Timeline of Events:

T+0 min: Phishing email arrives with malicious attachment
T+5 min: User opens attachment → Macro executes PowerShell
T+6 min: PowerShell downloads ransomware payload from C2 server
T+7 min: Ransomware starts encrypting files (1000+ file writes per minute)
T+10 min: Ransom note created on desktop

How SIEM Detects This:

Stage 1 - Initial Execution:

# Sigma Rule: Suspicious Macro Execution
detection:
  selection:
    process: WINWORD.EXE
    child_process: powershell.exe
  condition: selection

Stage 2 - C2 Communication:

# Sigma Rule: Outbound Connection to Suspicious Domain
detection:
  selection:
    destination_domain: known_c2_server.com  # From threat intel feed
  condition: selection

Stage 3 - Mass File Encryption:

# Sigma Rule: Rapid File Modifications
detection:
  selection:
    event_type: file_write
    file_extension: .encrypted
  condition: selection | count() > 100 within 1 minute

SIEM Action:

Alert security team via Slack (Critical severity)
Automatically isolate affected machine from network (if integrated with firewall)
Create incident ticket with full attack timeline

Use Case 2: Insider Threat Detection

Scenario: An employee planning to leave the company starts exfiltrating sensitive data.

Suspicious Behavior:

Downloads 50GB of customer data (unusual for their role)
Accesses competitor websites during work hours
Uses personal USB drive (against policy)
Logs in at 2 AM on Sunday (off-hours)

SIEM Detection:

# Composite rule for insider threat
detection:
  data_download:
    - large_file_transfer: > 10GB
    - destination: external_email OR usb_device

  anomalous_access:
    - access_time: between 22:00 and 06:00
    - day_of_week: Saturday OR Sunday

  policy_violation:
    - usb_device_used: true
    - encryption_disabled: true

  condition: data_download AND (anomalous_access OR policy_violation)

Use Case 3: Compliance & Audit Requirements

Many regulations mandate security monitoring:

GDPR (EU):

Article 32: Implement appropriate technical measures (logging)
Article 33: Detect and report breaches within 72 hours

PCI-DSS (Payment Card Industry):

Requirement 10: Track and monitor all access to network resources and cardholder data

SOC 2:

CC7.2: Monitor system components to detect anomalies

HIPAA (Healthcare):

§164.308(a)(1)(ii)(D): Implement procedures to detect security incidents

A SIEM provides the audit trail and incident response evidence required by these frameworks.

The Anatomy of a Security Operations Center (SOC)

A SOC (Security Operations Center) is a team of security analysts who use SIEM as their primary tool.

SOC Roles

Tier 1 Analyst (L1) - Triage

Monitors SIEM alerts
Performs initial investigation
Escalates true positives to Tier 2

Tier 2 Analyst (L2) - Investigation

Deep-dive forensic analysis
Correlates events across multiple sources
Recommends containment actions

Tier 3 Analyst (L3) - Threat Hunter

Proactively searches for threats (not just responding to alerts)
Develops new detection rules
Reverse-engineers malware

Incident Response Manager

Coordinates response to major incidents
Communicates with stakeholders
Documents lessons learned

A Day in the Life of a SOC Analyst

08:00 AM - Morning Briefing

Review overnight alerts (78 total, 3 high-priority)
Check threat intelligence feeds for new campaigns

09:15 AM - Alert Investigation

Alert: "Multiple failed SSH logins from 203.0.113.45"
Action: Check if IP is in threat feed → Not found
Result: Likely legitimate user with wrong password, closed as false positive

10:30 AM - Escalated Incident

Alert: "Possible data exfiltration - 25GB uploaded to unknown cloud service"
Investigation: User's laptop infected with info-stealer malware
Action: Isolate machine, reset user credentials, scan for similar infections

14:00 PM - Rule Tuning

Review false positive alerts from last week
Adjust thresholds (e.g., failed login alert: 5 attempts → 10 attempts)

16:00 PM - Threat Hunting

Search logs for PowerShell commands with encoded payloads (obfuscation technique)
Find 2 suspicious scripts → Further investigation needed

SIEM Challenges and Limitations

Challenge 1: Alert Fatigue

Problem: Most SIEMs generate thousands of alerts daily. 90-95% are false positives.

Example:

10,000 alerts per day
5 analysts × 8 hours = 40 analyst-hours
4 minutes per alert to investigate = Only 600 alerts investigated
9,400 alerts ignored (including potential real threats)

Solutions:

Risk-based alerting: Only alert on high-confidence detections
Automation: Use SOAR (Security Orchestration, Automation, and Response) to auto-triage low-priority alerts
Machine learning: Baseline normal behavior, alert only on anomalies

Challenge 2: High Resource Requirements

Traditional SIEMs are resource-intensive:

SIEM	RAM (Minimum)	Storage (Daily)	Complexity
Splunk	12GB+	50-100GB	High (proprietary SPL query language)
Elastic (ELK)	16GB+	40-80GB	Very High (requires Elasticsearch, Logstash, Kibana setup)
QRadar	32GB+	Varies	High (enterprise-focused, complex licensing)

For small businesses or home labs, this is prohibitively expensive.

Challenge 3: Skilled Analyst Shortage

The cybersecurity skills gap:

Global shortage of 4 million cybersecurity professionals
Average SOC analyst salary: $70,000-120,000/year
High burnout rate (shift work, alert fatigue)

This makes running a 24/7 SOC unaffordable for most organizations.

Challenge 4: Blind Spots

SIEM is only as good as the data it receives. Common blind spots:

Encrypted traffic: Can't inspect HTTPS without SSL interception
Cloud services: Many SaaS apps don't send logs to SIEM
Endpoint activities: Need EDR (Endpoint Detection and Response) integration
Physical security: Door access, CCTV not typically integrated

The Evolution of SIEM: Next-Generation Solutions

Traditional SIEM vs. Modern SIEM

Aspect	Traditional SIEM	Next-Gen SIEM
Detection	Rule-based (Sigma, regex)	AI/ML anomaly detection
Deployment	On-premise hardware	Cloud-native SaaS
Query Language	Proprietary (SPL, KQL)	SQL or natural language
Integration	Manual log parsing	Auto-discovery via APIs
Cost Model	Per-device or per-GB	Per-user or flat rate

Emerging Trends

1. Extended Detection and Response (XDR)

Combines SIEM + EDR + Network Detection + Cloud Security
Unified view across entire infrastructure

2. SOAR (Security Orchestration, Automation, Response)

Automates repetitive tasks (e.g., "Block this IP on all firewalls")
Playbooks for common incident types

3. User and Entity Behavior Analytics (UEBA)

Machine learning to detect anomalies
Example: "User normally accesses 10 files/day, today accessed 5,000 → Alert"

4. Cloud-Native SIEM

Born in the cloud, designed for cloud workloads
Automatic scaling, no infrastructure to manage

Building vs. Buying a SIEM

Enterprise SIEM Solutions

For Large Organizations (5,000+ employees):

Splunk Enterprise Security: $2,000-5,000/GB/year
IBM QRadar: Complex pricing, typically $100k+/year
Microsoft Sentinel: $2-5/GB ingested

Pros:

Mature product with extensive features
Enterprise support (SLAs)
Compliance certifications

Cons:

Very expensive at scale
Vendor lock-in
Complex to configure and maintain

Open-Source SIEM Solutions

For SMBs and Self-Hosters:

Wazuh: Open-source HIDS/SIEM (based on OSSEC)
Security Onion: Linux distro with ELK + Suricata + Zeek
Prelude: Open-source SIEM with commercial support option

Pros:

Free (or low-cost commercial support)
Full control over data
Customizable

Cons:

Requires security expertise to deploy
Limited out-of-the-box integrations
Community support only (unless you pay)

Lightweight SIEM Alternatives

For developers, home labs, and small teams, a full enterprise SIEM is overkill.

What you actually need:

Centralized log collection (syslog, API ingestion)
Basic threat detection (Sigma rules)
Alerting (Slack, email, webhooks)
Simple UI for searching logs

This is the gap many modern tools are filling—SIEM capabilities without the complexity and cost.

Conclusion: Why SIEM Matters in 2025

Cybersecurity is no longer optional. With the rise of:

Ransomware-as-a-Service (RaaS)
Supply chain attacks (SolarWinds, Log4Shell)
Nation-state threats (APTs targeting critical infrastructure)
Mandatory breach disclosure laws (GDPR, CCPA)

Organizations need visibility into their security posture.

SIEM provides that visibility—but traditional solutions remain expensive, complex, and resource-intensive.

The future belongs to lightweight, developer-friendly SIEM platforms that:

Deploy in minutes (not months)
Cost hundreds (not hundreds of thousands)
Use familiar technologies (SQL, Docker, open standards)
Respect data sovereignty (GDPR-compliant, self-hostable)

Whether you're running a home lab, a startup, or a mid-sized company, understanding SIEM concepts is no longer just for security professionals—it's essential for anyone responsible for infrastructure.

What is a SIEM?

The Core Problem SIEM Solves

How SIEM Works: The Four Pillars

1. Collection (Data Aggregation)

2. Correlation (Pattern Detection)

3. Alerting (Real-Time Notification)

4. Investigation (Forensics & Response)

MITRE ATT&CK: The Universal Language of Cyber Threats

What is MITRE ATT&CK?

The ATT&CK Matrix

Why MITRE ATT&CK Matters for SIEM

Sigma Rules: The Open Standard for Threat Detection

What is Sigma?

Anatomy of a Sigma Rule

Key Components Explained

The Power of Sigma

Threat Intelligence: Know Your Enemy

What is Threat Intelligence?

Types of Threat Intelligence

1. Strategic Intelligence

2. Tactical Intelligence

3. Operational Intelligence

4. Technical Intelligence

Indicators of Compromise (IoCs)

Threat Intelligence Feeds

How SIEM Uses Threat Intelligence

Real-World SIEM Use Cases

Use Case 1: Detecting a Ransomware Attack

Use Case 2: Insider Threat Detection

Use Case 3: Compliance & Audit Requirements

The Anatomy of a Security Operations Center (SOC)

SOC Roles

A Day in the Life of a SOC Analyst

SIEM Challenges and Limitations

Challenge 1: Alert Fatigue

Challenge 2: High Resource Requirements

Challenge 3: Skilled Analyst Shortage

Challenge 4: Blind Spots

The Evolution of SIEM: Next-Generation Solutions

Traditional SIEM vs. Modern SIEM

Emerging Trends

Building vs. Buying a SIEM

Enterprise SIEM Solutions

Open-Source SIEM Solutions

Lightweight SIEM Alternatives

Conclusion: Why SIEM Matters in 2025

Further Reading