DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Snowflake to Databricks: what the migration actually costs you
zephico profile

Snowflake to Databricks: what the migration actually costs you

#databricks #dataengineering
Comments
5 min read
48% of Data Engineering Job Postings Are Ghost Jobs
datadriven profile

48% of Data Engineering Job Postings Are Ghost Jobs

#dataengineering #career #interview #beginners
Comments
7 min read
Governing AI Agents: Why Compile-Time Security is Mandatory
nilesh_kumar profile

Governing AI Agents: Why Compile-Time Security is Mandatory

#ai #dataengineering #architecture #datascience
1
Comments
2 min read
Building a Zero-Budget Eval Harness for LLM-Assisted SQL Generation
dataio_4921 profile

Building a Zero-Budget Eval Harness for LLM-Assisted SQL Generation

#dataengineering #ai #sql #testing
Comments
5 min read
You Forgot to Mention the 500 Million Rows
raghavsharma_ profile

You Forgot to Mention the 500 Million Rows

#ai #dataengineering #sql #performance
Comments
4 min read
Mastering Databricks Cost Optimisation Without Slowing Teams
vachkov profile

Mastering Databricks Cost Optimisation Without Slowing Teams

#databricks #finops #dataengineering #cloud
Comments
6 min read
One Bug Hit 11 Rows. Another Hit 36 Million.
arthurtorres profile

One Bug Hit 11 Rows. Another Hit 36 Million.

#dataengineering #python #pandas #datascience
Comments
4 min read
Why CTE vs Subquery Matters More on Redshift Than Anywhere Else
maithreyan11 profile

Why CTE vs Subquery Matters More on Redshift Than Anywhere Else

#redshift #sql #dataengineering #aws
Comments
2 min read
Storix: One Typed Python Filesystem API from Local Development to Cloud Storage
mghalix profile

Storix: One Typed Python Filesystem API from Local Development to Cloud Storage

#showdev #python #cloud #dataengineering
Comments
16 min read
RAG vs. Semantic Layer: Why AI Needs Deterministic Governance
harshit_colrows profile

RAG vs. Semantic Layer: Why AI Needs Deterministic Governance

#dataengineering #database #ai #datascience
Comments
2 min read
Monitoring Airbyte's Postgres CDC Replication Slots Before They Take Down Your Source Database
mukesh_13 profile
Mukesh

Monitoring Airbyte's Postgres CDC Replication Slots Before They Take Down Your Source Database

#airbyte #postgres #cdc #dataengineering
Comments
5 min read
Building Airbyte Connectors That Survive Flaky APIs: Cursor Checkpointing for Resilient Incremental Sync
mukesh_13 profile
Mukesh

Building Airbyte Connectors That Survive Flaky APIs: Cursor Checkpointing for Resilient Incremental Sync

#airbyte #dataengineering #elt #python
Comments
5 min read
Building an Airbyte Destination Connector That Embeds and Dedupes Records Before They Hit pgvector
mukesh_13 profile
Mukesh

Building an Airbyte Destination Connector That Embeds and Dedupes Records Before They Hit pgvector

#airbyte #dataengineering #pgvector #rag
Comments
4 min read
Stop Faking Your Test Data: Why `Faker.js` Breaks the Moment You Have Foreign Keys
jitendra_devabhaktuni_0f1 profile

Stop Faking Your Test Data: Why `Faker.js` Breaks the Moment You Have Foreign Keys

#api #ai #datascience #dataengineering
Comments
5 min read
How to Run Apache Iceberg & Trino Locally in 60 Seconds (Zero AWS Cost)
ofirnakdai profile

How to Run Apache Iceberg & Trino Locally in 60 Seconds (Zero AWS Cost)

#dataengineering #python #docker #architecture
Comments
2 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.