Thanks for the post, I learned a lot of new things! I'd also like to mention that OLAP style queries can be rough with lots of data.
I am also thinking about this. RDBMS generally are good at most types of workloads but mixed workloads like OLTP and OLAP on the same system will interfere with each other.
How do people do embedded reporting AND transactions in modern web apps?
NoSQL solutions like ElasticSearch are mentioned but they seem more appropriate for search. Data warehouse solutions like Snowflake, BigQuery, Redshift are good for internal analytics and reporting but they just don’t have the concurrency to support direct queries from customer facing apps.
How else to do this without complex data pipelines or complex infrastructure involving Kafka?
I do it with a complex data pipeline and infrastructure involving Kafka!
I've got my eye on Apache Druid, though I haven't spent any real time (sorry, pun totally intended) with it.
I’ve looked at Druid, also considering MemSQL , ClickHouse and others
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.