Dataengineering

Robin Moffatt

Mar 30 '21

Quick profiling of data in Apache Kafka using kafkacat and visidata

#dataviz #apachekafka #dataengineering #datascience

2

1

2 min read

Robin Moffatt

Feb 18 '21

📼 ksqlDB HOWTO - A mini video series 📼

#ksqldb #dataengineering #apachekafka #streamprocessing

10

4 min read

Adi Polak for Microsoft Azure

Feb 2 '21

Tech Exceptions Show: Accelerating Data Engineering with Azure

#techtalks #dataengineering #cloudskills #startup

12

2 min read

Adi Polak

Jan 14 '21

Apache Spark Ecosystem, Jan 2021 Highlights

#opensource #datascience #dataengineering

11

4 min read

Robin Moffatt

Jan 18 '21

Running a self-managed Kafka Connect worker for Confluent Cloud

#apachekafka #kafkaconnect #confluentcloud #dataengineering

8

11 min read

Mayara Machado

Jan 4 '21

ETL com Apache Airflow, Web Scraping, AWS S3, Apache Spark e Redshift | Parte 1

#dataengineering #datascience #aws #etl

19

1

7 min read

Robin Moffatt

Jan 4 '21

Kafka Connect - Deep Dive into Single Message Transforms

#apachekafka #kafkaconnect #dataengineering #streamprocessing

4

3 min read

Rich Dudley for AWS Community Builders

Dec 29 '20

First Look: AWS Glue DataBrew

#glue #databrew #dataengineering #etl

10

7 min read

Peter Hanssens #BlackLivesMatter for AWS Heroes

Dec 17 '20

My favourite re:Invent data announcements

#aws #dataengineering #redshift #reinvent2020

8

5 min read

Robin Moffatt

Dec 15 '20

🎄 Twelve Days of SMT 🎄 - Day 6: InsertField II

#apachekafka #kafkaconnect #dataengineering #twelvedaysofsmt

6

3 min read

Anand

Dec 15 '20

New Features in Amazon DynamoDB - PartiQL, Export to S3, Integration with Kinesis Data Streams

#aws #dynamodb #database #dataengineering

11

12 min read

Robin Moffatt

Dec 8 '20

🎄 Twelve Days of SMT 🎄 - Day 1: InsertField (timestamp)

#apachekafka #kafkaconnect #dataengineering #twelvedaysofsmt

5

3 min read

Anniina Sallinen for Ompeluseura LevelUP Koodarit

Nov 22 '20

Datetimes Are Hard: Part 1 - Incoming data and formats

#dataengineering #data #series

4

1

4 min read

Sephi Berry

Nov 16 '20

Tidying up Pipelines with DataClasses

#pipeline #dataengineering #python #scikit

5

5 min read

Irtiza Ali

Nov 12 '20

Uniform Data Distribution Among Kinesis Data Stream Shards

#kinesis #dataengineering #python3 #aws

2

3 min read

BenBirt for Dataform

Sep 24 '20

Cut data warehouse costs with run caching

#elt #dataengineering #pipeline #etl

5

3 min read

Eshban Suleman for Traindex

Oct 26 '20

Introduction to Data Pipelines

#datascience #bigdata #dataengineering #pipelines

2

1

4 min read

Michiel Ghyselinck

Oct 15 '20

Dagster with User Code Deployments (gRPC)

#dataengineering #etl #dagster #kubernetes

14

2

6 min read

Satish Chandra Gupta

Oct 10 '20

12 Ways of Applying a Function to Python Pandas DataFrame

#python #datascience #machinelearning #dataengineering

4

1 min read

Aman Ranjan Verma

Oct 3 '20

Data engineering essentials

#hacktoberfest #python #devops #dataengineering

4

1 min read

Robin Moffatt

Sep 25 '20

Some of my favourite public data sets

#opendata #data #dataengineering #datascience

8

3

2 min read

Adi Polak

Sep 21 '20

Becoming a Data Engineer

#database #career #beginners #dataengineering

64

2

1 min read

Anand

Sep 20 '20

Transform AWS CloudTrail data using AWS Data Wrangler

#aws #bigdata #cloud #dataengineering

3

8 min read

Bartosz Gajda

Sep 19 '20

5 Essential skills for becoming a Data Engineer

#beginners #datascience #career #dataengineering

8

6 min read

Greg

Sep 17 '20

The Most Popular Data Science Newsletters

#datascience #machinelearning #dataengineering

11

9 min read

Simon Yu

Sep 17 '20

Build a monitored code-based pipeline to move data from Postgres to Snowflake

#aws #postgres #snowflake #dataengineering

7

9 min read

Wai Yan

Sep 14 '20

Handling upstream data changes via Change Data Capture

#dataengineering #database #datapipeline

8

8 min read

maninekkalapudi

Sep 14 '20

Intoduction to Apache Spark

#dataengineering #apachespark #bigdata #spark

10

6 min read

01:00

Robin Moffatt

Sep 11 '20

Kafka Connect in 60 seconds

#apachekafka #dataengineering #bigdata #dataintegration

4

2 min read

Simon Yu

Aug 25 '20

Deploying data pipelines to AWS Fargate - with monitoring and alerts built-in

#aws #dataengineering #serverless #docker

6

3 min read

Nazli Ander

Aug 14 '20

Windowing in Streaming Data: Theory and a Scikit-Multiflow Example

#python #datascience #dataengineering #datastreams

2

4 min read

Darsh Shukla

Aug 11 '20

Data Warehouse - The Minimal Architectural Approach

#dataengineering #datascience #datawarehouse #cloud

3

1

2 min read

Darsh Shukla

Aug 11 '20

Data Lake - 5 Major Principles

#dataengineering #datascience #datalake #cloud

2

2 min read

Todd Birchard for Hackers And Slackers

Aug 8 '20

Scrape Structured Data with Python and Extruct

#python #scraping #dataengineering #scraper

10

16 min read

Josh Holbrook

Jul 12 '20

How To Run Airflow on Windows (with Docker)

#dataengineering #etl #airflow

15

3

8 min read

Sephi Berry

Jul 9 '20

Implementing a graph network pipeline with Dagster

#dagster #dataengineering #pipeline #graph

22

1

12 min read

Krithika

Jun 21 '20

What differentiates schema on read from schema on write?

#database #data #dataengineering #schema

3

2

3 min read

Robin Moffatt for Confluent

Jun 23 '20

Loading CSV data into Kafka - video walkthrough

#apachekafka #tutorial #csv #dataengineering

5

10 min read

Todd Birchard for Hackers And Slackers

Jun 11 '20

Scraping Data on the Web with BeautifulSoup

#python #scrapers #data #dataengineering

33

12 min read

BenBirt for Dataform

Jun 8 '20

CI/CD for ETL/ELT pipelines

#cicd #etl #elt #dataengineering

18

3 min read

Joseph

Jun 3 '20

A proven approach to land a Data Engineering job

#database #beginners #career #dataengineering

6

5 min read

Joseph

May 28 '20

Data Engineering Project for Beginners - Batch edition

#dataengineering #tutorial #beginners #aws

25

19 min read

Joseph

May 11 '20

10 Key skills, to help you become a data engineer

#dataengineering #beginners #database #etl

9

3 min read

CitizenK

Apr 23 '20

Airflow UI with Role-Based Access Control

#python #apacheairflow #dataengineering

5

1 min read

CitizenK

Apr 22 '20

Apache Airflow Installation - mysql+celery

#python #apacheairflow #dataengineering

7

1 min read

Todd Birchard for Hackers And Slackers

Mar 26 '20

Extract Nested Data From Complex JSON

#python #restapis #dataengineering

9

6 min read

JoLo

Mar 3 '20

DataOps - A Made-Up Term or Actual Practice

#dataops #dataengineering #datascience #devops

13

7 min read

Waylon Walker

Mar 2 '20

🛢Create New Kedro Pipeline (kedro new)

#data #dataengineering #kedro #datascience

5

4 min read

Waylon Walker

Feb 24 '20

🤷‍♀️ What is Kedro (The Parts)

#data #dataengineering #kedro #datascience

18

4

3 min read

Josh Yap

Feb 12 '20

Data engineering portfolio projects?

#dataengineering #data #portfolio

29

1

1 min read

Zahidul Islam

Jan 3 '20

Apache Airflow Core Concepts

#airflow #workflow #dataengineering

26

4 min read

Luciano Strika

Oct 19 '19

Coding MapReduce in C from Scratch using Threads: Map

#programming #beginners #c #dataengineering

7

9 min read

Caleb Ariel

Sep 30 '19

I am a junior data engineer without a senior engineer. What should I do?

#startup #dataengineering

7

1

1 min read

Cong

Sep 26 '19

Toward GCP Data Engineer certification

#gcp #machinelearning #bigdata #dataengineering

9

1 min read

00:31

Yan Parker

Sep 14 '19

Data Engineering Skills

#dataengineering #datascience

14

1 min read

Flo P

Aug 9 '19

Intro to Data Ingestion and Data Lakes

#dataengineering #datalake #dataingestion

8

1

3 min read

Yan Parker

Aug 7 '19

Data Engineering — Complete Reference Guide From A-Z [2019]

#dataengineering #datascience #bigdata

30

16 min read

Julien Kervizic

Aug 6 '19

Overview of the different approaches to putting Machine Learning (ML) models in production

#data #dataengineering #machinelearning #software

9

14 min read

Julien Kervizic

Jul 30 '19

ON the evolution of Data Engineering

#data #dataengineering #etl #sql

15

4 min read

Jeff Hale

Jun 22 '19

10 Days to Become a Google Cloud Certified Professional Data Engineer

#cloud #database #dataengineering #google

26

2

11 min read