Also props for mentioning Luigi. Have you looked into Kedro? McKinsey's new open sourced thing. It's kind of neat. I've been using it for personal projects recently.
Oh man! No. It's so hard to keep up with all the new frameworks. But I will take a look.
The tutorial explains the features pretty well, it makes a sample regression pipeline that pre-process data and puts them into raw, processed, etc. folders.
I'm trying to see if I can combine it with tox and then maybe MLFlow for parameter saving.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.