DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
A cache key ate 99.9% of my records and the pipeline looked green
kyien profile

A cache key ate 99.9% of my records and the pipeline looked green

#dataengineering #python #architecture #sql
Comments
5 min read
SqlJam: Move Tables Across Databases with Schema, Data and All
paganini2008 profile

SqlJam: Move Tables Across Databases with Schema, Data and All

#database #java #sql #dataengineering
Comments
9 min read
Data Engineering Dropped DSA. What Replaced It Is Worse.
datadriven profile

Data Engineering Dropped DSA. What Replaced It Is Worse.

#dataengineering #interview #career #beginners
Comments
7 min read
Parsing financial PDFs without chasing 100% accuracy
strelok25dev profile

Parsing financial PDFs without chasing 100% accuracy

#python #pdf #dataengineering #pydantic
3
Comments
4 min read
Synthetic Time Series Data: A Practical Guide META
thiam_lee profile

Synthetic Time Series Data: A Practical Guide META

#ai #database #machinelearning #dataengineering
Comments
6 min read
CDC with Sling: When Log-Based Works and When Query-Based Is Enough
flarco profile

CDC with Sling: When Log-Based Works and When Query-Based Is Enough

#mysql #dataengineering #etl #database
Comments 1
8 min read
Data Pipelines Explained: Benefits, Types, and Use Cases
sergiiklius profile

Data Pipelines Explained: Benefits, Types, and Use Cases

#dataengineering #datascience #automation #tutorial
Comments
4 min read
The fastest Glue job is the one you never start: removing per-job overhead from a 191-job pipeline
kiran_gunturu_022085015ec profile

The fastest Glue job is the one you never start: removing per-job overhead from a 191-job pipeline

#aws #dataengineering #serverless #performance
Comments
8 min read
dbt Was Not Designed for Multi-Tenant SaaS. Here's How I Made It Work.
eu_ti_f127c5b5d7535b7174f profile

dbt Was Not Designed for Multi-Tenant SaaS. Here's How I Made It Work.

#dataengineering #sql #python #architecture
Comments
4 min read
Building a real-time price monitoring pipeline that does not poison your data
anakin_writers profile

Building a real-time price monitoring pipeline that does not poison your data

#webdev #dataengineering #scraping #analytics
Comments
4 min read
One Iceberg MCP Server, Seven Catalogs: What It Takes to Reach Each One
xbill profile

One Iceberg MCP Server, Seven Catalogs: What It Takes to Reach Each One

#mcp #iceberg #python #dataengineering
11
Comments
7 min read
Building an investing knowledge graph, part 3: same company, seven names
hannune profile

Building an investing knowledge graph, part 3: same company, seven names

#graphdb #knowledgegraph #dataengineering #entityresolution
2
Comments
4 min read
Why a Numaflow watermark wouldn’t move
akarazhev profile

Why a Numaflow watermark wouldn’t move

#dataengineering #distributedsystems #rust #opensource
Comments
7 min read
The Pipeline Worked. Then the Research Outgrew It.
daemonic01 profile

The Pipeline Worked. Then the Research Outgrew It.

#dataengineering #python #programming #productivity
Comments
4 min read
Subqueries vs CTEs: Query Optimizer Internals & Memory Spooling Explained
arpitmbangre profile

Subqueries vs CTEs: Query Optimizer Internals & Memory Spooling Explained

#database #sql #dataengineering #performance
Comments
2 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.