DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
We published our fundraising-prediction model and its misses. Here's what 219 rounds taught us.
data_nerd profile

We published our fundraising-prediction model and its misses. Here's what 219 rounds taught us.

#dataengineering #machinelearning #opensource #startup
Comments 1
3 min read
We timed HTTP-only scraping against a headless browser on the same page. It wasn't close.
fetchsmith profile

We timed HTTP-only scraping against a headless browser on the same page. It wasn't close.

#webscraping #dataengineering #api #javascript
Comments
4 min read
Clean a million rows with Hydra ETL — no database, no Docker
bejaouibechir profile

Clean a million rows with Hydra ETL — no database, no Docker

#dataengineering #python #rust #tutorial
Comments
6 min read
Databricks accounts, workspaces and metastores: which layer owns what
dino_david_c5d8f55f119b7f profile

Databricks accounts, workspaces and metastores: which layer owns what

#databrick #dataengineering #azure #beginners
Comments
2 min read
Stop Writing Custom File Listeners: Use Auto Loader Instead
aniketsoni profile

Stop Writing Custom File Listeners: Use Auto Loader Instead

#databricks #deltalake #dataengineering #spark
2
Comments 2
5 min read
Don't mock BigQuery, test on BigQuery: the rewrite of BigTesty
mazlum_tosun profile

Don't mock BigQuery, test on BigQuery: the rewrite of BigTesty

#bigquery #googlecloud #dataengineering #testing
7
Comments 1
7 min read
Turn a Lighthouse export into tickets with traceable threshold evidence
timmyzinin profile

Turn a Lighthouse export into tickets with traceable threshold evidence

#api #dataengineering #automation
Comments
3 min read
RustFS S3 Tables: A Built-In Iceberg REST Catalog in an Apache-2.0 Object Store
ethan-carter profile

RustFS S3 Tables: A Built-In Iceberg REST Catalog in an Apache-2.0 Object Store

#iceberg #storage #rust #dataengineering
1
Comments 1
12 min read
Your pipeline ran green ✅ and shipped last week’s data anyway
jenniferevangelistade profile

Your pipeline ran green ✅ and shipped last week’s data anyway

#python #databricks #dataengineering #dataquality
1
Comments 2
4 min read
DB Explorer: A Lightweight Database Viewer I Built for Everyday Development Work
ashish_srivastava_e6b8ecc profile

DB Explorer: A Lightweight Database Viewer I Built for Everyday Development Work

#ai #dataengineering #opensource #database
Comments
2 min read
A CSV Can Be Valid and Still Corrupt Your Import
silvio_greco profile

A CSV Can Be Valid and Still Corrupt Your Import

#dataengineering #sql #database #csv
Comments
2 min read
Where I got labeled training data for Korean-Japanese-Chinese entity resolution
hannune profile

Where I got labeled training data for Korean-Japanese-Chinese entity resolution

#entityresolution #dataengineering #python #nlp
1
Comments 1
2 min read
How to Validate Event Data in ClickHouse and Superset
sensorflow profile

How to Validate Event Data in ClickHouse and Superset

#clickhouse #analytics #opensource #dataengineering
Comments
9 min read
Generating Kafka ClickHouse pipelines from a JSON sample
maksim-gr profile

Generating Kafka ClickHouse pipelines from a JSON sample

#clickhouse #dataengineering #kafka #rust
Comments
7 min read
AWS Glue Concurrency Is Capped by One Subnet, Not Your Whole VPC
kiran_gunturu_022085015ec profile

AWS Glue Concurrency Is Capped by One Subnet, Not Your Whole VPC

#aws #awsglue #dataengineering #networking
Comments
4 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.