DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Power BI Service on ClickHouse, without an ODBC driver on your gateway
sergeisemenkov profile

Power BI Service on ClickHouse, without an ODBC driver on your gateway

#powerbi #clickhouse #semanticlayer #dataengineering
Comments
5 min read
Apache Airflow for Java Engineers: Orchestrating Data Pipelines Beyond Python
said_olano profile

Apache Airflow for Java Engineers: Orchestrating Data Pipelines Beyond Python

#java #airflow #dataengineering #architecture
Comments
8 min read
dbt snapshots are not an audit log
aniketsoni profile

dbt snapshots are not an audit log

#sql #analytics #dataengineering #dataquality
1
Comments
4 min read
Your Python Script Works on 1,000 Rows. Then the Real CSV Arrives.
emilywoodsnyc profile

Your Python Script Works on 1,000 Rows. Then the Real CSV Arrives.

#python #dataengineering #programming #interview
Comments 1
8 min read
Running CDC in Production: The WAL Slot Trap
pipelinewhiteboard profile

Running CDC in Production: The WAL Slot Trap

#postgres #dataengineering #devops #aws
Comments
7 min read
Zero-downtime partition swaps on Athena: build in a new prefix, flip with SET LOCATION
csotelo profile

Zero-downtime partition swaps on Athena: build in a new prefix, flip with SET LOCATION

#aws #dataengineering #python #athena
Comments
5 min read
dlt's Auto-Schema Pipeline: How Python Data Loaders Infer Structure Without Predefined Models
mech_app_ai profile

dlt's Auto-Schema Pipeline: How Python Data Loaders Infer Structure Without Predefined Models

#database #dataengineering #opensource #python
Comments
4 min read
A Green Spark Job Isn't Enough: Building a Fail-Closed Pipeline Health Monitor in Python
rupak_kulkarni_08b45c5846 profile

A Green Spark Job Isn't Enough: Building a Fail-Closed Pipeline Health Monitor in Python

#python #dataengineering #devops #testing
Comments
6 min read
Hadoop Ecosystem Overview
godofgeeks profile

Hadoop Ecosystem Overview

#bigdata #data #dataengineering
Comments
8 min read
Web Scraping vs Official API: Which Should You Use?
aqib_javaid profile

Web Scraping vs Official API: Which Should You Use?

#webscraping #apiintegration #dataengineering #architecture
Comments
6 min read
Equality deletes have no address, and that's why your Iceberg reads crawl
kyien profile

Equality deletes have no address, and that's why your Iceberg reads crawl

#dataengineering #iceberg #sql #flink
Comments
7 min read
Iceberg vs Parquet: do you need a table format?
mohammed_arshadansari_f2 profile

Iceberg vs Parquet: do you need a table format?

#dataengineering #database #data #analytics
Comments
3 min read
AI in Data Engineer Interviews: Banned, Allowed, or Graded?
datadriven profile

AI in Data Engineer Interviews: Banned, Allowed, or Graded?

#dataengineering #interview #career #ai
1
Comments 1
9 min read
Watermarks are a contract: validating 150+ PySpark streams
aayusbhat26 profile

Watermarks are a contract: validating 150+ PySpark streams

#pyspark #dataengineering #apachespark
1
Comments
4 min read
Proving exactly-once by killing the stream mid-backlog
tantn89 profile

Proving exactly-once by killing the stream mid-backlog

#dataengineering #spark #iceberg #kafka
Comments 3
8 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.