DEV Community: Dinesh

Stock Marketing Analytics

Dinesh — Mon, 31 Mar 2025 22:13:28 +0000

just built a complete #StockMarket analytics platform on #GCP! Real-time pipeline with #Kafka streaming, historical data processing via #Spark on Dataproc, transformations using #dbt orchestrated by #Airflow. built dashboards in Looker Studio track market trends .#dezoomcamp

Real-Time Data Processing with PyFlink and Redpanda

Dinesh — Mon, 17 Mar 2025 16:25:51 +0000

Just leveled up my #DataEngineering skills by building real-time data pipelines with PyFlink and Redpanda! 🚀 Discovered how session windows can reveal hidden patterns in NYC taxi data that batch processing would miss. #StreamingData #PyFlink #DEZOOMCAMP

Diving into bigdata

Dinesh — Thu, 06 Mar 2025 17:59:00 +0000

Just wrapped up Module 5 of #DEZOOMCAMP! Learned to work with Apache Spark for batch processing NYC taxi data. Analyzed over 125K taxi trips, Excited to dive into streaming data next! #DataEngineering #BigData

Data Transformation

Dinesh — Wed, 26 Feb 2025 15:37:41 +0000

#DEZoomcamp! Learnt dbt fundamentals - from model resolution to complex data lineage. Built pipelines analyzing 160M+ NYC taxi rides with advanced SQL and visualized them in dataloader. dbtcloud was cool , the ide and the lineage was nice #dbt #BigQuery

ETL with DLTHUB

Dinesh — Sun, 16 Feb 2025 17:16:56 +0000

Just wrapped up the Data Engineering Zoomcamp workshop on dlt! Learned how to build data pipelines the easy way - extracted NYC Taxi data using dlt's REST client, handled pagination like a pro, and loaded it all into DuckDB. Love how dlt makes ETL feel like a breeze!

Module 3 of DataTalks club Data Engg bootcamp.

Dinesh — Wed, 12 Feb 2025 20:39:02 +0000

📊 Just wrapped up Module 3 of #DEZOOMCAMP! Dove deep into BigQuery and learned:

External vs Materialized tables
Smart partitioning & clustering strategies
Column-based storage benefits
Query cost optimization techniques
Love how BQ handles those NYC taxi records efficiently! 🚕✨

DataEngineering Bootcamp Module 2

Dinesh — Wed, 05 Feb 2025 22:44:24 +0000

🔄 Module 2 of #DEZOOMCAMP: Diving into workflow orchestration with Kestra! Successfully processed NYC taxi data using GCP & BigQuery. Learned about task scheduling, data pipelines, and proper ETL practices. Cool stuff: automated data loading from CSV to BigQuery and implemented timezone-aware scheduling! 🚖💻 #DataEngineering

Data Engg Bootcamp module 1

Dinesh — Mon, 27 Jan 2025 17:54:35 +0000

🚀 Just completed Module 1 of #DEZoomcamp! Built a data pipeline analyzing NYC taxi data using Docker, PostgreSQL, and Terraform. Check out my solutions here: https://github.com/Deathslayer89/DTC_dataEngg/tree/main/module1-hw

🚀 Just completed Module 1 of #DEZoomcamp! Built a data pipeline analyzing NYC taxi data using Docker, PostgreSQL, and Terraform. Check out my solutions here: https://github.com/Deathslayer89/DTC_dataEngg/tree/main/module1-hw #DataEngineering #Docker #SQL

Dinesh — Mon, 27 Jan 2025 17:53:57 +0000