The latest articles on DEV Community by Archibald Duskbottle (@archibald_duskbottle_366f). https://dev.to/archibald_duskbottle_366f https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3802502%2F7925ea3d-1e75-4b35-a4c1-4da9951eb2ca.png https://dev.to/archibald_duskbottle_366f en Archibald Duskbottle Mon, 02 Mar 2026 22:27:50 +0000 https://dev.to/archibald_duskbottle_366f/how-we-built-a-ga4-compatible-analytics-pipeline-to-escape-us-tech-lock-in-a06 https://dev.to/archibald_duskbottle_366f/how-we-built-a-ga4-compatible-analytics-pipeline-to-escape-us-tech-lock-in-a06 <h1> How We Built a GA4-Compatible Analytics Pipeline to Escape US Tech Lock-in - d8a.tech </h1> <p>Google Analytics is everywhere. It's also a deal-breaker for a growing number of teams.</p> <p>Under GDPR and the post-Schrems II landscape, sending EU visitor data to Google's US infrastructure is legally murky at best. For healthcare organizations under HIPAA or government sites under FedRAMP, it's a non-starter.</p> <p>The usual answer is to switch to a privacy-friendly alternative. The problem: most of them require you to throw away your existing tracking plan and start over. If you've invested in a GA4 setup - event taxonomy, GTM configuration, custom dimensions - that's a real switching cost.</p> <h2> One hard requirement </h2> <p>We built d8a around a single constraint: it had to speak GA4's protocol natively. Same <code>/g/collect</code> endpoint, same parameters. If you're already sending data to Google, you're already sending it in the right format for d8a. No rewrites, no migration weekend.</p> <h2> How it's put together </h2> <p>The pipeline has three moving parts: a tracking component that turns HTTP requests into hits, a queue that buffers them, and a processing component that closes sessions and writes to your warehouse.</p> <p><strong>The transport is pluggable.</strong> By default the two components communicate through the filesystem - cheap, simple, works on a single VPS. For HA deployments (how our cloud runs it), you swap in object storage and the processes scale independently. A RabbitMQ driver would fit the same interface if you need minimum latency at high throughput. The tradeoffs are real: filesystem is fine for a single node but rules out HA; object storage is pennies and unlocks horizontal scaling; a message broker gets you minimum latency but comes with either maintenance overhead or a bigger cloud bill.</p> <p><strong>The session engine has a pluggable batched KV interface.</strong> Session state - grouping hits by visitor, tracking inactivity windows - lives behind an interface with a complete blackbox test suite. The default implementation uses BoltDB: embedded, no external process, runs anywhere. Swapping it for something distributed (Redis, Cassandra, whatever fits your infra) is a matter of passing a different implementation that satisfies the same contract.</p> <p><strong>The tracking component is protocol-agnostic.</strong> GA4 is the default, but the HTTP path-to-protocol mapping is an abstraction - adding a new ingest protocol is a matter of implementing an interface. Matomo, Amplitude, or anything else with a defined HTTP tracking format could be a drop-in. This also means, that it can act as a self-hosted mirror for teams already on those platforms, intercepting existing tracking calls without touching the client.</p> <p><strong>Warehouse destinations:</strong> ClickHouse (fully self-hosted), BigQuery, or CSV files written to S3/MinIO, GCS, or local disk. The file path works with Snowflake Snowpipe, Redshift Spectrum, Databricks Auto Loader, DuckDB - if you already have a warehouse, you can pipe into it.</p> <p>Deployment-wise: a single VPS is enough to get started. Our own cloud runs it on Kubernetes with the object storage transport between the two components.</p> <h2> Get started </h2> <p>d8a is open source, MIT licensed.</p> <ul> <li><a href="https://docs.d8a.tech/getting-started" rel="noopener noreferrer">Getting started guide</a></li> <li><a href="https://github.com/d8a-tech/d8a" rel="noopener noreferrer">GitHub</a></li> <li>We also have completely free (for now) <a href="https://app.d8a.tech" rel="noopener noreferrer">cloud - app.d8a.tech</a>.</li> </ul> analytics clickhouse eu bigquery