DEV Community

How we decreased our memory usage with jemalloc

rhymes on June 05, 2020

DEV is a Ruby on Rails application deployed on Heroku servers. The other day Molly Struve, our resident SRE sorceress, noticed we were having some...

Read full post

Yaser Al-Najjar • Jun 5 '20 • Edited

Great job Rhymes and Molly!

I would also suggest few things to track down the issue further:

Creating a rails app with just one endpoint and all the gems used in DEV, to determine the normal from the abnormal memory usage.
Using a memory profiler and determining which endpoints account for the majority of the memory usage.
After pinpointing the bottlenecks, you might consider switching their implementations into a faster one like C/Go/Rust (I suspect the serialization and deserialization are the culprits).
All that above matters if there is some real financial pain, otherwise it's not worth pursuing 😁

rhymes • Jun 5 '20

Thanks a lot for the suggestions!

The gem derailed_benchmarks also helps:

schneems / derailed_benchmarks

Go faster, off the Rails - Benchmarks for your whole Rails app

Derailed Benchmarks

A series of things you can use to benchmark a Rails or Ruby app.

Compatibility/Requirements

This gem has been tested and is known to work with Rails 3.2+ using Ruby 2.1+. Some commands may work on older versions of Ruby, but not all commands are supported.

For some benchmarks, not all, you'll need to verify you have a working version of curl on your OS:

$ which curl
/usr/bin/curl
$ curl -V
curl 7.37.1 #

Install

Put this in your gemfile:

gem 'derailed_benchmarks', group: :development

Then run $ bundle install.

While executing your commands you may need to use bundle exec before typing the command.

To use all profiling methods available also add:

gem 'stackprof', group: :development

You must be using Ruby 2.1+ to install these libraries. If you're on an older version of Ruby, what are you waiting for?

Use

There are…

View on GitHub

Zubair Mohsin • Jun 5 '20

the memory management API looks very much like libc's malloc implementation

Looks like PHP internally uses malloc implementation?

php.net/manual/en/internals2.memor...

rhymes • Jun 5 '20

I'm not 100% sure about it, as I'm unfamiliar with PHP. I went as far as this file github.com/php/php-src/blob/master... which contains the declaration of the list of allocators you linked.