DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Day 10: Partitioning vs Bucketing - The Spark Optimization Guide Every Data Engineer Needs
sandeepk27 profile

Day 10: Partitioning vs Bucketing - The Spark Optimization Guide Every Data Engineer Needs

#python #dataengineering #spark #bigdata
Comments
2 min read
Deepening My Roots in the Data Ecosystem - Choosing Depth Over Breadth
jongwan93 profile

Deepening My Roots in the Data Ecosystem - Choosing Depth Over Breadth

#career #dataengineering #devjournal
Comments
2 min read
Automate Python Manual Extraction: Build End-to-End PDF -> LLM -> SQL Flows with CocoIndex, Ollama, and Postgres
badmonster0 profile

Automate Python Manual Extraction: Build End-to-End PDF -> LLM -> SQL Flows with CocoIndex, Ollama, and Postgres

#python #llm #dataengineering #etl
Comments
3 min read
The Boring Debug Checklist That Fixes Most “RAG Failures”
dowhatmatters profile

The Boring Debug Checklist That Fixes Most “RAG Failures”

#rag #dataengineering #llm #architecture
Comments
2 min read
Function Calling and Tool Use: Turning LLMs into Action-Taking Agents
qvfagundes profile

Function Calling and Tool Use: Turning LLMs into Action-Taking Agents

#ai #llm #mcp #dataengineering
Comments
18 min read
dremioframe & iceberg: Pythonic interfaces for Dremio and Apache Iceberg
alexmercedcoder profile

dremioframe & iceberg: Pythonic interfaces for Dremio and Apache Iceberg

#python #tooling #database #dataengineering
Comments
8 min read
AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL
datapenguin profile

AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL

#aws #dataengineering #etl
3
Comments
7 min read
SQL: Doing GROUP BY in CsvPath
david_kershaw_b6916404da6 profile

SQL: Doing GROUP BY in CsvPath

#sql #dataengineering #csv #csvpath
Comments
5 min read
đŸ”„ Day 3: RDDs - The Foundation of Spark
sandeepk27 profile

đŸ”„ Day 3: RDDs - The Foundation of Spark

#distributedsystems #interview #dataengineering #tutorial
Comments
2 min read
đŸ”„ Day 4: RDD Internals - Partitions, Shuffles & Repartitioning Demystified
sandeepk27 profile

đŸ”„ Day 4: RDD Internals - Partitions, Shuffles & Repartitioning Demystified

#spark #dataengineering #bigdata #python
Comments
2 min read
The Developer's Guide to Normalizing Historical Airline Flight Data for Machine Learning
rameshchauhan profile

The Developer's Guide to Normalizing Historical Airline Flight Data for Machine Learning

#dataengineering #datascience #machinelearning
Comments
6 min read
Overview of Real-Time Data Synchronization from MySQL to VeloDB
apachedoris profile

Overview of Real-Time Data Synchronization from MySQL to VeloDB

#database #aws #dataengineering #mysql
5
Comments
5 min read
Stop Writing df.describe(): Automate EDA with D-Tale (The Lazy Engineer's Way)
theaidataengineer1010 profile

Stop Writing df.describe(): Automate EDA with D-Tale (The Lazy Engineer's Way)

#python #dataengineering #productivity #beginners
Comments
3 min read
CHW Monthly Activity Aggregation: Turning Visit Logs into Insight
oliver_samuel_028c6f65ad6 profile

CHW Monthly Activity Aggregation: Turning Visit Logs into Insight

#analytics #sql #dataengineering #tooling
Comments
5 min read
đŸ”„ Day 2: Understanding Spark Architecture - How Spark Executes Your Code Internally
sandeepk27 profile

đŸ”„ Day 2: Understanding Spark Architecture - How Spark Executes Your Code Internally

#spark #python #dataengineering #bigdata
Comments
2 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.