DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Your Extraction Pipeline Doesn't Need a Smarter Model. It Needs a Schema Contract Test.
datacpp_8185 profile

Your Extraction Pipeline Doesn't Need a Smarter Model. It Needs a Schema Contract Test.

#ai #python #llm #dataengineering
Comments
6 min read
The report says verified. Is the change safe to release?
jerry_king_d0ee94a9bca82f profile

The report says verified. Is the change safe to release?

#dataengineering #database #testing #agents
Comments
5 min read
Reproducible Benchmarks for Web-Access APIs: A Five-Layer Design
nativeport profile

Reproducible Benchmarks for Web-Access APIs: A Five-Layer Design

#abotwrotethis #ai #testing #dataengineering
Comments
6 min read
Why Apache Airflow Instead of Cron? A Deep Dive Into How Airflow Actually Schedules Your DAGs
maithreyan11 profile

Why Apache Airflow Instead of Cron? A Deep Dive Into How Airflow Actually Schedules Your DAGs

#airflow #dataengineering #python #etl
Comments
4 min read
56 fault-injection tests passed. The one that injected nothing failed.

Negative controls reveal flaky HLL estimators
ashg2099 profile

56 fault-injection tests passed. The one that injected nothing failed.

#dataengineering #python #testing #dbt
3
Comments 8
3 min read
Build a Data Lake on S3-Compatible Storage
ethan-carter profile

Build a Data Lake on S3-Compatible Storage

#datalake #s3 #objectstorage #dataengineering
2
Comments
7 min read
97% of Data Engineers Are Burned Out. Here's the Data.
datadriven profile

97% of Data Engineers Are Burned Out. Here's the Data.

#dataengineering #career #interview #beginners
1
Comments
7 min read
Blocking Key Design for Entity Resolution: Why Name Normalization Fails East Asian Corporate Data
hannune profile

Blocking Key Design for Entity Resolution: Why Name Normalization Fails East Asian Corporate Data

#python #dataengineering #graphs #machinelearning
1
Comments
4 min read
Nobody Warns You the Data Engineering Interview Isn't About Data Engineering
rahmanfrr profile

Nobody Warns You the Data Engineering Interview Isn't About Data Engineering

#dataengineering #interview #career #sql
Comments 3
2 min read
Two domains are 53% of the Databricks Data Engineer Associate exam
nicheelab profile

Two domains are 53% of the Databricks Data Engineer Associate exam

#databricks #dataengineering #certification #sql
Comments
4 min read
I built the "anti-Elementary" for dbt data quality and here's why
rbmuller profile

I built the "anti-Elementary" for dbt data quality and here's why

#python #dbt #opensource #dataengineering
Comments
5 min read
AI-Enhanced Data Pipelines: Integrating NVIDIA Triton Inference Server into Modern ETL Architectures
etloss profile

AI-Enhanced Data Pipelines: Integrating NVIDIA Triton Inference Server into Modern ETL Architectures

#ai #dataengineering #machinelearning #nvidia
1
Comments
4 min read
Why Spark Couldn't Read from Kafka: A Real Debugging Journey Across PySpark, Hadoop, Docker, and Kafka
lee_yao_cfeb14fb9b141b8c5 profile

Why Spark Couldn't Read from Kafka: A Real Debugging Journey Across PySpark, Hadoop, Docker, and Kafka

#dataengineering #apachekafka #apachespark #docker
Comments
10 min read
Agentic AI and big data: what your platform needs before agents are useful
zephico profile

Agentic AI and big data: what your platform needs before agents are useful

#ai #dataengineering #databricks
Comments
6 min read
My First GitHub Project: From a Local Folder to GitHub Using Git and SSH
muoks_102 profile

My First GitHub Project: From a Local Folder to GitHub Using Git and SSH

#github #dataengineering #git
Comments 1
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.