I reviewed a data pipeline project that compares Louisville Metro expenditure data across fiscal years. The project utilized Terraform, a Prefect Docker container, and Google Cloud Storage/Big Query. I was impressed with the partitioning of data by fiscal year, fulfilling project requirements despite the dataset being less than 1GB. Clear instructions and a well-designed dashboard made it easy to navigate. Suggested future improvement is to include a script for downloading data directly from the original source to avoid excessive report generation. Kudos to everyone involved in DataTalksClub Data Engineering Zoomcamp 2023 and Louisville Open Data portal for public access to the data.
Muhammed Jimoh
Posted on
Review of Louisville Metro Expenditure Data Pipeline
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)