DEV Community

Discussion on: What Even Is Data Engineering?

Collapse
 
sloan profile image
Sloan the DEV Moderator • Edited

This is an anonymous comment sent in by a member who does not want their name disclosed.

I'm not sure how valid these questions are or if I'm articulating them properly.

  • What is a data warehouse and when do you need it? This is what I've been assuming: a gigantor database that extracts data from other sources, transforms them into the format you want and consolidates them all in one place so you can query all you want. i.e. if you don't need data from a db to be combined with other data, you wouldn't need to include it your data warehouse. Is that in the ballpark?

  • Do apps pull data from the data warehouse? I would assume not because they'd pull directly from the app database ^ if my above notion is correct.

  • Do data warehouses typically get updated in real time?

Collapse
 
istratos00 profile image
david Huaman Ñ • Edited

Hi! I've been working in DWH some years, in my expierence i could say about question 1&2: a data warehouse is not only for querying data, it allows you to get information(next level of data) and explode them with anothers tools like PowerBI, Microstrategy, etc. I didn't know about an app the reads a data warehouse.

Datawarehouses get updated in batch, it's not common a streamming updating, this is an app database behaviour.

:)

Collapse
 
thejessleigh profile image
jess unrein

Totally valid questions! I will definitely start working on an explainer post about data warehouses - what they are, what commercial products are available, and common use cases! Thank you for reaching out!