DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
DuckDB vs Spark: which workloads actually need a cluster
mohammed_arshadansari_f2 profile

DuckDB vs Spark: which workloads actually need a cluster

#dataengineering #database #python #data
Comments
3 min read
Pazarlama olaylarını sunucuda toplamak: Bir geliştiricinin kontrol listesi
korayyalcin profile

Pazarlama olaylarını sunucuda toplamak: Bir geliştiricinin kontrol listesi

#analytics #webdev #dataengineering #privacy
Comments
3 min read
I Registered Fresh Accounts on 5 Amazon Scraper APIs and Ran the Same Test on All of Them
pangolinfo profile

I Registered Fresh Accounts on 5 Amazon Scraper APIs and Ran the Same Test on All of Them

#webscraping #dataengineering #api #python
1
Comments
8 min read
Your BI Quote Is Really a Data Quote
seven_billion_ffd761d9574 profile

Your BI Quote Is Really a Data Quote

#businessintelligence #dataengineering #analytics #budgeting
Comments
2 min read
Faster Pip Installs in Python Docker Images With BuildKit Cache Mounts
aniketsoni profile

Faster Pip Installs in Python Docker Images With BuildKit Cache Mounts

#docker #python #cicd #dataengineering
2
Comments
4 min read
The real work in a data agent is the harness around it
jacob_baruch_2a2aec3f6e3c profile

The real work in a data agent is the harness around it

#ai #dataengineering #analytics #data
Comments
6 min read
How to assess AI data readiness for one business workflow
openlock1377 profile

How to assess AI data readiness for one business workflow

#ai #dataengineering #consult
Comments
3 min read
Data Engineering Log 2/3: Source Systems, Data Ingestion, and Pipelines
ayush_pangaonkar profile

Data Engineering Log 2/3: Source Systems, Data Ingestion, and Pipelines

#dataengineering #aws #etl #airflow
Comments
1 min read
Don't mock BigQuery, test on BigQuery: the rewrite of BigTesty
mazlum_tosun profile

Don't mock BigQuery, test on BigQuery: the rewrite of BigTesty

#bigquery #googlecloud #dataengineering #testing
2
Comments
7 min read
Eradicating AWS OOM Kills: 10M LLM Inferences on a $0 Cloud-Cost O(1) Memory Clamped Architecture
shotamatsubara profile

Eradicating AWS OOM Kills: 10M LLM Inferences on a $0 Cloud-Cost O(1) Memory Clamped Architecture

#dataengineering #devops #machinelearning #python
Comments
1 min read
Agentic Data Engineering: How AI Agents Build Data You Can Trust
revos_ai profile

Agentic Data Engineering: How AI Agents Build Data You Can Trust

#dataengineering #ai #dbt #agents
1
Comments
15 min read
Writing a Reconciliation Job in SQL and Python That Proves Two Systems Agree
jayant_chaudhary profile

Writing a Reconciliation Job in SQL and Python That Proves Two Systems Agree

#sql #dataengineering #integration #postgres
1
Comments 1
12 min read
Build a technology segment from page fingerprints and keep the evidence
timmyzinin profile

Build a technology segment from page fingerprints and keep the evidence

#apify #dataengineering #api #tutorial
Comments
3 min read
Delta Lake MERGE in PySpark: Upserts, SCD Type 2, and the Traps Interviewers Ask About
karanpulyala profile

Delta Lake MERGE in PySpark: Upserts, SCD Type 2, and the Traps Interviewers Ask About

#databricks #pyspark #dataengineering #deltalake
2
Comments
5 min read
PostHog Dropped ClickHouse for DuckDB: A Data Engineer's Take
datadriven profile

PostHog Dropped ClickHouse for DuckDB: A Data Engineer's Take

#dataengineering #duckdb #database #interview
Comments
9 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.