re: DEV Community Is Open-Sourcing Its Performance Metrics via Skylight VIEW POST

VIEW PARENT COMMENT VIEW FULL DISCUSSION
 

I believe includes(:user) is a huge memory hog because all the user associations get loaded into memory in their entirety when we typically only need the username, profile pic, etc.

I believe you, it's probably one of those cases where the DB behaves perfectly fine until you have lots and lots of data. It would be nice if we could create a script, separate from seeds.rb, that creates a DB with a size comparable to the real one. This way one could measure performance locally and see the query plan to understand what's going on. The DB can change how it executes a query depending on the estimated cost.

Another thing that could help is using the latest versions of PostgreSQL. PostgreSQL 10 and 11 execute more queries and joins in parallel, which could help a bit. But I would start by "explaining" the queries if that's the bottleneck (added to what you hinted at as Rails taking too much memory transforming rows into objects). Upping PostgreSQL's version, unless there's a specific reason to do it, won't make a huge difference

You can run explain like this in the console:

query.explain

This is a cool guide to interpret the info explain outputs: Reading a Postgres EXPLAIN ANALYZE Query Plan

We also have some lingering n+1 issues and other types of repeated queries.

Shouldn't n+1 be notified by the gem bullet? Is it because they are triggered only in production?

Anyhow, I'm optimistic about the fact that devto can be optimized!

Within 2019 we'd love to get to a place where we can let other folks run the application as their own community app and a part of that is improving efficiency so people can run the app more cheaply and reliably.

Yes, having an idea of how the app scales as the user base grows could be helpful for other communities.

code of conduct - report abuse