DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Organizing How to Use AWS Lake Formation
datapenguin profile

Organizing How to Use AWS Lake Formation

#aws #dataengineering
3
Comments
9 min read
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
gathurum profile

ETL Pipeline: Fetching Real-Time News Data with Python and Postgres

#dataengineering #api #etl #beginners
1
Comments
4 min read
I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast
hajirufai profile

I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast

#dataengineering #python #database #performance
Comments
6 min read
Why I Don’t Let the LLM Decide Issue State
ahmadhumayun profile

Why I Don’t Let the LLM Decide Issue State

#ai #python #dataengineering #marketinganalytics
Comments
5 min read
The customer_id that isn't a customer
gaur_data_c6511af396c4bce profile

The customer_id that isn't a customer

#agents #analytics #dataengineering #sql
Comments
8 min read
Your Scraper Collected 50 Rows. There Were 4,000.
0012303 profile

Your Scraper Collected 50 Rows. There Were 4,000.

#webscraping #python #dataengineering #pagination
Comments
7 min read
Deeper into Dataform 3: Auditing Dataform
ben_watson_data profile

Deeper into Dataform 3: Auditing Dataform

#dataform #dataengineering #gcp #bigquery
Comments
2 min read
How I Broke Down My ETL Pipeline Project Into Smaller Engineering Exercises
tanmay_bhurkunde profile
Tanmay

How I Broke Down My ETL Pipeline Project Into Smaller Engineering Exercises

#dataengineering #database #etl #sql
Comments
2 min read
I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost
hajirufai profile

I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost

#python #dataengineering #datascience #opensource
Comments
6 min read
A practical pipeline for turning messy business documents into spreadsheets
alex_zheng_49d3089c0d3fdf profile

A practical pipeline for turning messy business documents into spreadsheets

#dataengineering #ai #productivity
Comments
2 min read
Day 9 of 100 Days of ClickHouse®: Mastering Data Aggregation from GROUP BY to CUBE
kanishga_subramani_49ad73 profile

Day 9 of 100 Days of ClickHouse®: Mastering Data Aggregation from GROUP BY to CUBE

#analytics #database #dataengineering #sql
1
Comments
4 min read
Your Scraper Died at Row 12,000. The Rerun Pattern.
0012303 profile

Your Scraper Died at Row 12,000. The Rerun Pattern.

#python #webscraping #dataengineering #reliability
Comments
13 min read
The Search Engine Renaissance: How Apache Lucene and Elasticsearch Are Reclaiming the AI-Native Future
vigneshh profile

The Search Engine Renaissance: How Apache Lucene and Elasticsearch Are Reclaiming the AI-Native Future

#dataengineering #lucene #elasticsearch #serverless
1
Comments
8 min read
If the warehouse already has the data, why are we copying it elsewhere?
gazerro profile

If the warehouse already has the data, why are we copying it elsewhere?

#architecture #data #dataengineering #systemdesign
Comments
5 min read
What Is Agentic Workflow Consulting? A Practical Guide for Data Leaders
labyrinthanalytics profile

What Is Agentic Workflow Consulting? A Practical Guide for Data Leaders

#ai #machinelearning #dataengineering #llm
Comments
7 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.