DEV Community

Ayush S Pangaonkar
Ayush S Pangaonkar

Posted on

Data Engineering Log 3/3: Data Storage and Queries

I finished Data Storage and Queries, the third course in the DeepLearning.AI + AWS Data Engineering Professional Certificate, on October 3, 2026. The labs were hands-on on AWS, and I cleared the final exam.

Course

Data Storage and Queries, DeepLearning.AI and Amazon Web Services, on Coursera.

Verify certificate: https://coursera.org/verify/6AY7BEQK4RBQ

What I did

  • Studied how data is physically stored. That covers storage on disk and in memory, then object, block and file storage, and the databases built on top of those.
  • Followed the evolution of storage architectures. From data warehouses to data lakes to data lakehouses, comparing the advantages and drawbacks of each.
  • Designed a simple data lake with AWS Glue, then built a data lakehouse with AWS Lake Formation and Apache Iceberg.
  • Looked at how queries work behind the scenes. I wrote more advanced SQL, compared query performance on row-oriented versus column-oriented storage, and ran streaming queries with Apache Flink.

What I took from it

Storage and queries are connected. How data is laid out on disk, row or column, decides how fast a query can run on it, so the storage choice has to be made with the queries in mind.


Series: Data Engineering Log, part 3 of 3, from the DeepLearning.AI + AWS Data Engineering Professional Certificate.

More of my work is on GitHub.

Top comments (0)