Deep in the heart of data // Carl Steinbach // MLOps Coffee Sessions #22

Coffee Sessions #22 with Carl Steinbach of LinkedIn, Deep in the Heart of Data.

//Bio
Carl is a Senior Staff Software Engineer and currently the Tech Lead for LinkedIn's Grid Development Team. He is a contributor to Emerging Architectures for Modern Data Infrastructure

//Other links referenced by Carl:
https://rise.cs.berkeley.edu/wp-content/uploads/2017/03/CIDR17.pdf
https://www.youtube.com/watch?v=-xIai_FvcSk&ab_channel=WePayEngineering
https://softwareengineeringdaily.com/2019/10/23/linkedin-data-platform-with-carl-steinbach/
https://www.slideshare.net/linkedin/carl-steinbach-open-source
https://dreamsongs.com/RiseOfWorseIsBetter.html
https://engineering.linkedin.com/blog/2017/03/a-checkup-with-dr--elephant--one-year-later
https://engineering.linkedin.com/
https://engineering.linkedin.com/blog/2018/11/using-translatable-portable-UDFs
https://a16z.com/2020/10/15/the-emerging-architectures-for-modern-data-infrastructure/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/
Connect with Carl on LinkedIn: https://www.linkedin.com/in/carlsteinbach/

Timestamps:
[00:00] Introduction to Carl Steinbach
[00:44] Carl's background
[04:51] Breakdown of Transpiler
[10:55] Advantages of Decoupling the Execution Layer
[15:25] Differences between UDF (user-defined function) Functions and Views
[18:45] How do you ensure the reproducibility of these Views?
[23:58] Data structure evolution
[27:55] Are Data Lakes and Data Warehouse fundamentally different things or are they on a path towards conversion?
[33:37] It's inevitable that people will start doing machine learning on databases
[36:01] Who gets permission on what, especially when it comes to data and how sensitive things can be?
[41:27] Security aspect of data  
[43:40] Does it require a level of obstruction on top of the data of the file system?
[45:48] Why do we go back and go forward which sets this trend?

Episode source