dev.to has a fairly scalable architecture in that we've built it so most traffic is cached statically on the edge via our CDN Fastly. In this sense we scale like a static site might, but any time content changes (like a new comment, or an update to the article, or any other freshness change), we start over and go straight to the origin. We also serve some dynamic content asynchronously with most requests. So there's still a lot of load on the server.
Like any Rails app, we hit different resource constraints and work through them as we scale.
In addressing fluctuations in resource bottlenecks we basically need to know what resources are being used up and what levers to pull when they get used up. Because of highly successful tech companies that have scaled this stack in the past, we rarely need to be all that clever, we just need to know the tools.
But despite this, we still have problems, of course. Code is tough. I want to share a few mental blocks I've needed to overcome to help lead our team in this regard. On to the lesson:
When problems arise, go back to old material
I've failed to re-read good material when we hit rough patches, erroneously thinking the problem is something new. Often it's the same basic problem in a new light. The next time we hit a particular rough patch, I will re-read a lot of great material. Read lots of stuff with an open mind, rather than purely reading to problem solve. You don't want to miss the forest for the trees when thinking this stuff through.
In this regard, the Speedshop blog by Nate Berkopec and other content from him has been very practical. The specific problems we faced don't need to be re-hashed here, but the advice not to think you need new material is the lesson here.
A couple DEV members who's stuff has also been helpful either on platform or on their personal blogs of years past:
![[deleted user] image](https://res.cloudinary.com/practicaldev/image/fetch/s--xtH13c6f--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://res.cloudinary.com/practicaldev/image/fetch/s--diPD5F8K--/c_fill%2Cf_auto%2Cfl_progressive%2Ch_150%2Cq_auto%2Cw_150/https://thepracticaldev.s3.amazonaws.com/i/99mvlsfu5tfj9m7ku25d.png)
[Deleted User]
Both definitely worth following, as are many more folks who post in #rails
Recently this old post from Mike was particularly helpful in terms of thinking through Heroku resource issues. Re-reading was critical because I'd gotten some of the important conclusions mixed up in my memory.
Going forward
The future holds many challenges. For example, we have not had Postgres as a bottleneck in a while, but in the future it will become one again. When that happens, rather than scramble to learn new things in that moment, the best immediate action will be to thoroughly re-read old material that has helped us get over a few hiccups in the past. From here we can go on to learning new things, but we cannot have the fallacy of having fully learned and retained old material for indefinite new problems.
Happy coding!
Top comments (10)
Sigh. Every time I make a post like this, we hit new issues.
Such is life, but still feeling like the lesson is working 😄
Maybe they respond to that. Try making posts until you run out of issues to handle. 🤔
Use the Cloud, Benakin! 😯
It’s funny how our problems are mostly config related. As cloud users, we have such first world problems of just setting the right config and having autoscaling configures.
I can only imagine our problems if we weren’t using the cloud.
Hi Ben, very interesting post. I’ve been looking for information about the Dev.to current stack (hardware resources and services) but I didn’t find. Could you make a post about it? I think this site is the best example (by far) about scaling Rails in the right way.
😄
Yeah, Heroku's prices are definitely not great but still hasn't been a big % of our costs of running the business. Examining moving to DO or otherwise definitely a possibility.
From my experience, DO does offer value for one's money with stellar documentation for most things and good support (in case of issues)
While other people were migrating to Elixir or Go I just dug deeper into Rails to work at scale with less friction. One such thing that breathed new life into Rails was serving json directly from Postgres database. I saw endpoints go from 500ms to 20ms with no caching. It was unbelievable.
Though this required writing raw queries and so I then created my own gem that made it easier to managed queries and also some helpers for json functions.
I probably would have a popular gem on my hands if I bothered to put some marketing effort and documentation behind it.
Monster Queries
I have on my todo list to get rid of using handlebars and writing my own templating language for SQL.
Well said Ben! Some content, especially about performance and optimization, is evergreen.
Speedshop is my jam 🤣
Do you have any resources on how you went about porting your app from Heroku to DO? egghead runs Rails on Heroku and that cost benefits sounds really appealing.