DEV Community

Cover image for AWS GuardDuty vs OpenSearch Security Analytics: Building a Real SIEM Pipeline on AWS
Danh Hoang Hieu Nghi
Danh Hoang Hieu Nghi

Posted on

AWS GuardDuty vs OpenSearch Security Analytics: Building a Real SIEM Pipeline on AWS

Comparing AWS GuardDuty and OpenSearch Security Analytics for AWS SIEM architecture, and why centralizing AWS security findings usually means running both, not choosing one.

It's 2am and PagerDuty just fired. GuardDuty flagged an anomalous UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration finding in one account.

Meanwhile your WAF logs, three on-prem firewalls, and a fleet of custom application logs sit in a completely different system that has no idea the IAM finding exists. You need to answer one question — "has this IP touched anything else we own in the last 30 days?" — and you can't, because the answer is split across five tools with five different query languages, and none of them talk to each other.

This is the visibility problem every team running more than a handful of AWS accounts eventually hits. GuardDuty tells you something bad happened, but it wasn't built to be the place you go searching for everything that happened. That's a separate job, and AWS gives you a separate tool for it: Security Analytics on Amazon OpenSearch Service. The two get talked about as competitors more often than they should be — in practice, they're stages in the same pipeline.

What each service actually is

Amazon GuardDuty is a fully managed threat detection service — you turn it on and it starts analyzing immediately, with no infrastructure to provision or patch. It automatically ingests CloudTrail management events, VPC Flow Logs, and DNS logs, and you can layer on optional protection plans for S3 data events, EKS audit logs, RDS login activity, Lambda network activity, and Runtime Monitoring (OS, network, and file-level events on EC2, ECS on Fargate, and EKS). Detection comes from AWS-curated threat intelligence feeds combined with ML models trained across AWS's own telemetry. You don't write the rules; AWS does. It scales cleanly across an organization too — a delegated administrator account can auto-enable GuardDuty for every existing and new member account without anyone touching individual accounts by hand.

OpenSearch Security Analytics is a different animal. It's a feature layered on top of an OpenSearch Service domain that you provision, scale, and pay for like any other cluster. Nothing shows up automatically — you build the ingestion pipeline yourself, typically with Data Prepper, Kinesis Data Firehose, or Logstash, feeding in whatever you want: Windows event logs, NetFlow, CloudTrail, DNS logs, Active Directory/LDAP, custom application logs, even on-prem network gear. Detection rules are written in Sigma, an open and community-maintained format, which means you can write, tune, and share your own detections instead of relying on a fixed catalog. When a detector matches a rule against an incoming log event, it generates a finding, and alerting runs through OpenSearch's Alerting/Notifications plugin — which supports Amazon SNS as a channel alongside Slack, Chime, and custom webhooks. Underneath all of that, it's still a search engine, so you get real full-text and structured log search — the one thing GuardDuty categorically does not offer.

Where they genuinely differ

Amazon GuardDuty OpenSearch Security Analytics
Operational model Fully managed, zero infrastructure Self-managed OpenSearch domain — you size, scale, and patch it
Data sources Fixed set: CloudTrail, VPC Flow Logs, DNS, plus optional S3, EKS, RDS, Lambda, Runtime Monitoring Anything you pipe in — AWS logs, on-prem, custom apps, network devices
Detection logic AWS-curated threat intel + ML, closed finding types Sigma rules, open format, fully customizable
Log search None — findings only, no raw log query Native full-text/structured search across all ingested data
Multi-account Native AWS Organizations delegated admin, auto-enable Manual per-pipeline setup; no built-in org construct
Retention Findings retained ~90 days (via Security Hub) You control retention — as long as storage allows
Cost driver Pay per GB of data analyzed, no cluster to run Pay for the OpenSearch domain (instances, storage) plus ingestion pipeline

The pattern in that table is consistent: GuardDuty trades flexibility for zero operational overhead, and OpenSearch Security Analytics trades operational overhead for flexibility and depth. Neither trade is wrong — they're just optimized for different jobs. GuardDuty is built to catch the AWS-specific attack patterns AWS itself has the best visibility into (credential exfiltration, crypto-mining behavior, anomalous API calls) without you lifting a finger. OpenSearch Security Analytics is built for the investigation and correlation work that starts after something gets flagged, or for environments where the threats you care about live outside what GuardDuty can even see.

The real answer: they're pipeline stages, not competitors

AWS's own guidance is unusually direct about this. From the AWS Security Blog's SIEM guide: "Although Security Hub has some similarities to security information and event management (SIEM) tools, it is not designed as a standalone SIEM replacement." The same post adds the mirror-image caveat for OpenSearch: "Although the OpenSearch Service also is not a SIEM out-of-the-box tool, with some customization, you can use it for SIEM tool use cases."

Read together, those two lines describe the architecture most security teams actually land on. GuardDuty detects. Security Hub aggregates those findings — along with findings from other AWS security services — into one place. Because Security Hub only holds findings for around 90 days and isn't meant for cross-source log correlation, teams export findings out via S3 and a Lambda-based ETL step, or stream them through EventBridge, into an OpenSearch domain. From there, OpenSearch (or Kibana/OpenSearch Dashboards on top of it) handles long-term retention past that 90-day window, correlation against non-AWS log sources, and the dashboards analysts actually live in during an investigation.

A minimal version of the EventBridge routing rule that kicks this off looks like:

{
  "source": ["aws.guardduty"],
  "detail-type": ["GuardDuty Finding"]
}
Enter fullscreen mode Exit fullscreen mode

That rule targets a Lambda function that reshapes the finding and writes it into an OpenSearch index — or, more commonly today, targets Security Hub first, with a second export step moving aggregated findings from Security Hub into OpenSearch on a schedule. The point isn't the exact wiring; it's that GuardDuty findings become one input stream among several feeding a system built for the search and correlation GuardDuty was never designed to do.

guardduty-opensearch-pipeline

A decision framework, not a coin flip

If your workloads are entirely on AWS, your team is small, and what you need is "tell me when something bad happens so I can go look" — run GuardDuty organization-wide and stop there. It costs nothing to operate beyond the per-GB analysis fee, and it will catch the categories of AWS-native threats it's tuned for without anyone maintaining rules.

If you're already running GuardDuty but keep hitting the wall where you need to search raw logs, correlate a finding against something GuardDuty doesn't ingest, or retain data longer than 90 days for compliance — that's the signal to add OpenSearch Security Analytics on top, not to replace GuardDuty with it.

If you have log sources GuardDuty structurally cannot see — on-prem firewalls, custom application audit logs, non-AWS network appliances, Windows domain controllers — you need OpenSearch Security Analytics regardless of what you decide about GuardDuty, because no amount of GuardDuty configuration will ever ingest those.

And if you're running a mid-size environment — think a fintech spread across 40 AWS accounts with a mix of cloud-native and legacy on-prem systems — you'll likely end up needing both feeding a shared pipeline: GuardDuty as the always-on detection layer for AWS-native threats, Security Hub as the aggregation point, and OpenSearch as the durable, searchable, cross-source system of record underneath it all.

Decision framework: use GuardDuty alone for AWS-only workloads with basic alerting needs; add OpenSearch Security Analytics once you need raw log search, retention past 90 days, or non-AWS log sources

Where to start

Turn on GuardDuty across your organization today if it isn't already running — it's the highest-value, lowest-effort step available, and there's no good reason to delay it. Treat OpenSearch Security Analytics as the next investment, not a parallel one: reach for it once you can name the specific gap GuardDuty leaves open, whether that's log sources it can't reach, retention past 90 days, or the ability to search raw events instead of just reading findings. Building the Security Hub-to-OpenSearch pipeline before you have that gap just means paying to run and maintain a cluster you don't yet need.

Top comments (0)