From Python to ClickHouse: Parquet ETL with Go

Hey Devs 👋,

If you're exploring modern data engineering stacks or want to try out ClickHouse with Go and Python — this post is for you!

I wanted to experiment with something lightweight but real:
Generating a Parquet file using Python and loading it into ClickHouse using Go.

Here’s what I built, how it works, and what I learned 👇

🔗 GitHub Repo

---

📦 What This Project Does

This is a beginner-friendly, containerized mini-project that:

🧪 Generates sample data using a Python script
📁 Converts it into a Parquet file
🔁 Loads the data into a ClickHouse table using a Go app
🐳 Runs locally using Docker Compose

---

🛠️ Tech Stack

• Python — to generate Parquet files
• Go — to read Parquet and insert into ClickHouse
• ClickHouse — lightning-fast OLAP DB
• Docker Compose — to simplify ClickHouse setup
• Parquet — for efficient columnar storage

---

⚙️ How To Run It Locally

Step 1. Clone the repo

   git clone https://github.com/mohhddhassan/go-clickhouse-parquet.git
   cd go-clickhouse-parquet

Step 2. Generate sample Parquet data

   cd python
   python3 generate_parquet.py

Step 3. Start ClickHouse using Docker Compose

   docker-compose up -d

Step 4. Run the Go app to ingest data

   cd go
   go run main.go

---

🗂️ Project Structure

go-clickhouse-parquet/
├── docker-compose.yml         # ClickHouse setup
├── parquet-files/
│   └── sample.parquet         # Auto-generated test file
├── python/
│   └── generate_parquet.py    # Script to create data
└── go/
    ├── go.mod
    ├── go.sum
    └── main.go                # Ingests Parquet into ClickHouse

---

🤯 What I Learned

💡 How to programmatically create Parquet files
💡 Connecting Go with ClickHouse and executing inserts
💡 Using Docker Compose to deploy ClickHouse quickly
💡 Structuring a mini ETL workflow with multiple languages

---

🔍 Why You Should Try This

If you're learning data engineering or systems programming:

• Try combining Python + Go for real-world data movement
• Practice building and using Parquet files — they're everywhere in analytics
• Explore ClickHouse and see how blazing fast OLAP can be
• Get used to wiring up different components in a real pipeline

---

📌 What’s Next?

📈 Build a ClickHouse dashboard on top of this data
⚙️ Try streaming Parquet data into ClickHouse
📂 Expand schema complexity for more realistic ingestion
🛠️ Benchmark Go vs Python for loading speed into ClickHouse

---

🙋‍♂️ About Me

Mohamed Hussain S
Associate Data Engineer
LinkedIn | GitHub

🧪 Building one mini project at a time to become a better data engineer.

---