Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Pratyush Raghuwanshi

Jan 5

CSV Processing Gotchas: Don’t Let Invalid Data Slip Through the Cracks!!!

#python #dataengineering #softwaredevelopment #systemdesign

1 min read

Theodore P.

Jan 5

Streaming SQL Engine: Lightweight Cross-Data Source Integration for Resource-Constrained Environments.

#architecture #dataengineering #python #sql

1 min read

Christian Wagner

Jan 2

Stop Manually Tracing Azure Synapse Dependencies

#database #sql #opensource #dataengineering

1 min read

Alex Merced

Jan 2

Building Pangolin: My Holiday Break, an AI IDE, and a Lakehouse Catalog for the Curious

#ai #dataengineering #devjournal

6 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 8: Databricks Pipeline & Dashboard

#analytics #dataengineering #tutorial #sql

2 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

#architecture #dataengineering #tutorial #sql

2 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 5: Building a ZIP Code Dimension Table

#analytics #database #dataengineering #tutorial

2 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 2: Project Architecture

#systemdesign #data #architecture #dataengineering

2 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 1: Creating Databricks Workspace and Enabling Unity Catalog

#beginners #azure #dataengineering #tutorial

2 min read

Nithyalakshmi Kamalakkannan

Jan 2

Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

#dataengineering #python #tutorial

1 min read

Amine Laatfa

Jan 1

Automating Serverless Data Ingestion: How to Connect External APIs to BigQuery using Python and Cloud Functions

#googlecloud #bigquery #marketing #dataengineering

12 min read

Neo

Jan 1

The Data Liberation: Amazon Athena and the Architecting of a Serverless Future

#aws #dataengineering #serverless #cloudcomputing

3 min read

Tayfun Yalcinkaya

Jan 5

Why Apache Ozone is the Preferred Object Store for Big Data

#dataengineering #bigdata #datalakehouse #apacheozone

3 min read

Aris Georgatos

Jan 5

Be Essential or Be Optional: A Reality Check for Data Teams

#leadership #growth #dataengineering #datateams

1 min read

Yash Dubey

Feb 5

Markdown Is Not The Future of LLM Data Infrastructure

#ai #llm #webscraping #dataengineering

7 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# dataengineering

CSV Processing Gotchas: Don’t Let Invalid Data Slip Through the Cracks!!!

Streaming SQL Engine: Lightweight Cross-Data Source Integration for Resource-Constrained Environments.

Stop Manually Tracing Azure Synapse Dependencies

Building Pangolin: My Holiday Break, an AI IDE, and a Lakehouse Catalog for the Curious

Part 8: Databricks Pipeline & Dashboard

Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

Part 5: Building a ZIP Code Dimension Table

Part 2: Project Architecture

Part 1: Creating Databricks Workspace and Enabling Unity Catalog

Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

Automating Serverless Data Ingestion: How to Connect External APIs to BigQuery using Python and Cloud Functions

The Data Liberation: Amazon Athena and the Architecting of a Serverless Future

Why Apache Ozone is the Preferred Object Store for Big Data

Be Essential or Be Optional: A Reality Check for Data Teams

Markdown Is Not The Future of LLM Data Infrastructure