DEV Community

Cover image for Why I Built My Cloud-Native DFIR Pipeline (And Ditched AWS/OpenSearch for BigQuery)
CrabCanneryShip
CrabCanneryShip

Posted on

Why I Built My Cloud-Native DFIR Pipeline (And Ditched AWS/OpenSearch for BigQuery)

#dfir #security #go #googlecloud

There are fantastic, battle-tested tools out there like KAPE and CDIR. But as a DFIR engineer, I wanted something more lightweight, highly customizable, and automated from the ground up to fit my own workflow. So, I built a fast forensics pipeline: Veloxamen.

The Pain Points: Why Not AWS?

I initially looked into building this on AWS. While Timesketch is hard to let go of, getting OpenSearch ingestion to feel just right on AWS always felt heavier and clunky than it should be.

Then, I shifted my focus to GCP. The scalability and sheer convenience of BigQuery for structured log analysis completely won me over.

What Veloxamen Does

Veloxamen is designed to automatically ingest and process forensic artifacts in GCP, transforming them into a structured, queryable timeline inside BigQuery.

  • Collector + Pipeline: Combined with a custom collector, it handles the heavy lifting.
  • Windows First (For Now): Right now, it focuses heavily on Windows artifacts (since attackers love targeting them), but the architecture is fully extensible. Drop your logs into the staging bucket with a clean prefix, and anything supported gets automatically transformed into a unified timeline.

Evolutional Improvements: What's Next?

BigQuery is just the baseline. Because all the parsed forensic timelines live cleanly in BigQuery, the path forward opens up to some serious evolution:

  • Microservices Transformation: While Log2Timeline/Plaso is undeniably powerful and sophisticated, managing and scaling its compute resources can be exhausting. It will be replaced with a highly concurrent microservices architecture for lightning-fast artifact processing.
  • Looker / Looker Studio Integration: Instant visual dashboards and interactive hunting views without wrestling with heavy legacy SIEM UIs.
  • Vertex AI-Powered Analysis: Leveraging LLMs and ML models directly over BigQuery data to automate anomaly detection, summarize event horizons, and speed up triage reporting.

You can check out the source code and architecture here:
🔗 GitHub: https://github.com/veloxamen

I'm currently opening it up to the global community. Feedback, issues, or thoughts from fellow DFIR/cloud security folks are more than welcome!

Top comments (0)