DEV Community: Dhaval Singh

How Truly Random are Random Numbers?

Dhaval Singh — Sun, 07 Jan 2024 09:27:43 +0000

We have all generated random numbers at one point or another, be it for calculations or HTTP cookies. If you haven’t manually generated one and you are working on a real-life project, there is a high chance somewhere it is being generated for you in "Sumdi" ('Sumdi' is a Marathi word which means to do your work quietly and not let anyone take notice).

In this post, I want to dive into how these random numbers are generated, how random are they actually and what goes behind the scenes. It is really interesting and even though you won’t be "using" this knowledge irl, it will generate a lot of interesting conversations and deepen your understanding of the technology you interact with daily

Most of you might have heard of Cloudflare's Lava Lamps. It's probably the most famous way to generate "truly" random bytes.

It is very well-documented how it works and why they use this, you can read about it here.

In this post, I want to explore how random numbers/bytes are generated in programming languages. I will use Ruby as an example, but pretty much all languages have a similar implementation.

class Random

Okay, before we jump into what happens, it's important to understand the terminology and a few small things to understand and appreciate this subject better.

Let's take the class Random of Ruby and start from there, the introduction covers most of the terms necessary for us to jump-start this journey. Here is what the docs say, read it once, and I will break down the more uncommon terms that we require.

PRNG

"PRNG" means "Pseudorandom Number Generator" which means that a sequence of numbers (bits, bytes...) is produced from an algorithm that looks random, but is in fact deterministic (the sequence is generated from some unknown internal state), hence pseudorandom.

There is another term CSPRNG: Cryptographically Secure Pseudo-Random Number Generator. We will get into this later.

In simple terms, all you need to understand is PRNG is not truly random, and here is where the right question comes in, if it's not truly random, then how random is it?

Mersenne Twister with a period of 2¹⁹⁹³⁷-1

A Mersenne prime is a prime number that can be written in the form of 2ⁿ - 1, where n is an integer. Not all numbers of this form are prime, but when they are, they're called Mersenne primes.

The Mersenne Twister is a type of pseudorandom number generator (PRNG). It's known for providing fast and high-quality random numbers. Its name comes from the fact that its period length is a Mersenne prime. The most common version of the Mersenne Twister, MT19937, has a period of 2¹⁹⁹³⁷-1.

The period of a PRNG is the length of the sequence of numbers it produces before it starts repeating. The Mersenne Twister with a period of 2¹⁹⁹³⁷-1 means it can generate 2¹⁹⁹³⁷-1 numbers before the sequence repeats, an extraordinarily large number.

How Mersenne Twister works is pretty complex and it's above my paygrade. Very interesting nonetheless.

Let's move ahead with the part I want to talk more about!

SecureRandom

As this algorithm is not for cryptographical use, you must use SecureRandom for security purpose, instead of this PRNG.

Okay, this makes sense. For security purposes, we need numbers that are completely unpredictable and Ruby recommends we don't use this class instead use SecureRandom. Let's see what that class definition says.

Here I will focus on /dev/urandom but all three offer more or less the same result.

Okay, so what is /dev/urandom ? How is it more "secure" than the complicated algorithm using large prime numbers?

Now keep in mind we are going to use SecureRandom for generating password tokens and whatnot, we need it to be completely unpredictable! It has to be so unpredictable that even the folks with knowledge of its inner workings are not able to "guess" it.

Computers are very deterministic machines (even if they don’t feel like they are)

Now if you think about it, computers are basically built to always sequentially execute the same set of instructions which we call 'code'. We want the same set of instructions to run in the same way across different computers.

Duh! Seems obvious right?

But here is where we get in trouble, when generating random stuff we want two different computers to do it completely differently. Otherwise, it's not random anymore!

So we understand computers are not unpredictable by nature. So, how or where will they find this "randomness" that we require?

Well, the answer is right there! In the real world, the messy physical world!

Unpredictable events

Every time you type on your keyboard or move your mouse, the disk spins a bit to read some data, it happens in the best case just a little differently.

Now all these events are visible to the kernel. It's the kernel's job to manage these interactions so it keeps track of these events. For example, You move your mouse from (0,1) to (0,3.6).
How wide this range(how spread apart the values are) is what we call entropy.

The more spread apart they are, the harder they are to predict. What they are not is uniform! You are going to move your cursor in a very non-uniform way and the same goes for other events.

Now this is good. We have some unpredictability on our hands, but these are not enough to satisfy our random bytes needs. Enter a CSPRNG.

A CSPRNG is a cryptographic tool that generates a stream of random bytes using all and only the input(our random events) you give to it.

A very simple way to understand how this works is, that we start with a pool just with 0s. Now anytime an event happens

we convert it into binary
Hash it with the pool
Now we have an updated pool.

Rinse and repeat for all events!

We keep doing this and our entropy pool will keep changing. Now for anyone to figure out what my entropy pool is at any point in time, they will have to replicate each and every event of my system, right from mouse movements to hard disk reads in the exact same order with the exact same values! This is pretty much an impossible task!

Now anytime you want say 1000 random bits, we take the entropy pool, hash with 0, 1, 2, 3... (basically a counter). We get some outputs, join them, and voila! We have 1000 bits as random and unpredictable as the pool itself!

Now there are some basic properties of the hash function itself like

you cant get the input from the output
all the bits in the input affect all the bits of the output which ensures it is almost impossible to reverse-engineer this!

So back to our question, What is /dev/urandom?

It is exactly this! It's a CSPRNG in the (Linux) Kernel. It looks like a file, you read it like a file. Every time you read some bytes, it will run the CSPRNG and stir the contents of the file. Other OS's have something similar to /dev/urandom.

There is something called /dev/random too. I'll leave it to you to find the difference and what it's use-case is.

Back to SecureRandom

Now the Ruby implementation is pretty much what we talked about. It just reads the file and massages the output in the desired format. Something like...

def self.random_bytes(n=nil)
  File.open("/dev/urandom", File::RDONLY) do|f|
    f.readpartial(n)
  end
end

The current implementation is in C, but you get the idea...

And that is why SecureRandom is considered secure. It still relies on a pseudorandom number generator as we would do in Ruby Land, but it has the benefit of using environmental noise, making it astronomically much more difficult to crack, and technically impossible without gathering the exact same environmental data.

Footnote: There are cases where /dev/urandom has been exploited during the boot sequence. At that time, the system may not have enough entropy available to provide safe random numbers. When the start sequence is being initiated, the system starts from a known state, always the same (particularly true for virtual images), and mostly the same also on any other identical environment. This is also a pretty interesting rabbit hole if you wish to visit it.

To raise or not to raise?

Dhaval Singh — Sun, 26 Nov 2023 18:04:12 +0000

Did you know they have ashtrays in Airplane bathrooms even tho smoking is banned inside!? It turns out there is a good reason for them, and you’ll be glad they’re there.
In case a passenger does illegally sneak a ciggy, planes must have a safe place to put out the butts.

This intriguing detail has significantly influenced my perspective on system design. Having that ashtray even though smoking is banned just shows how critical it is to ensure safety in Airplanes and I've always wanted the systems I build to be similar in this aspect. Resilient and Robust in the face of a bad input or an error.

So I started reading about this stuff and boy is it confusing! Rightly so, because exception handling is pretty context-heavy. Right from what the business logic dictates to how "confidently" the code has been written, it can vary quite a bit.

If you too are tired of seeing nested being/rescue blocks or abused nil checks in your codebase, this series is for you!

So, in this and the following articles as I dive deeper(and implement them) we will be able to gain a much better understanding of how to deal with exceptions or the spaghetti code that surrounds it.

Let's start with some first-principle thinking.

My code works as it is, why should I care?

In school you might have heard the expression, "Garbage in, garbage out." That expression is essentially software development's version of caveat emptor: let the user beware.
For production software, garbage in, garbage out isn't good enough. A good program never puts out garbage, regardless of what it takes in. A good program uses "garbage in, nothing out," "garbage in, error message out," or "no garbage allowed in" instead. By today's standards, "garbage in, garbage out" is the mark of a sloppy, nonsecure program.

code complete

From the above quote and airplane example, it's evident that handling rogue or incorrect inputs is crucial for system integrity. This is particularly vital in critical systems like banking or payments, where explicit error handling is essential. In less critical systems, logging and moving on may suffice. However, in all cases, mastering failure-handling techniques is key to ensuring our systems are both reliable and robust.

Understanding Failures in Programming

Every element in a codebase serves a purpose – if it doesn't, it's probably safe to let it go. When something in that element doesn't do what it's supposed to, that's what we call a failure.

So how or when can a code element fail? Let's see a few common reasons

# Implementation issue
 h = {:age=> 23}; h["age"] += 1

# External issue
HTTP.get_response(URL).code # => 500

# System/Hardware issue, eg: ran out of memory 
# NoMemoryError

Okay, so failure can be caused by us or the hardware or due to someone else’s mistake, but it's our responsibility to handle this and ensure this doesn't break our system.

Now we understand what is meant by failure, the question that arises is;
"How does our program tell us there is a failure?"

This question has always confused me since some terms are used interchangeably in software, ie: Exception and Error.

Exception vs Error

These terms are commonly used in software so often that it is not my cup of tea to be able to define them(without creating more confusion). So I will just quote sources that made it actually clear for me!

A failure is the inability of a software element to satisfy its purpose.

An exception is the occurrence of an abnormal condition during the execution of a software element.

An error is the presence in the software of some element not satisfying its specification.

from Object Oriented Software Construction

So the answer to the question, how does a program tell us something has failed?
It does so by raising an Exception.
And one might say, these exceptions are caused due to errors.

Still not clear enough? Steve McConnell explains it beautifully in Code Complete:

Exceptions are a specific means by which code can pass along errors or exceptional events to the code that called it. If code in one routine encounters an unexpected condition that it doesn't know how to handle, it throws an exception, essentially throwing up its hands and yelling, "I don't know what to do about this—I sure hope somebody else knows how to handle it!" Code that has no sense of the context of an error can return control to other parts of the system that might have a better ability to interpret the error and do something useful about it.

code complete

The Exception Tree

Okay now let's talk in terms of Ruby. Exceptions are simply classes that the Ruby library has predefined for us.

Below is a list of exception classes that ship with Ruby’s standard library. Third-party gems like Rails will add additional exception classes to this chart. Every additional exception from Rails will inherit from some class on this list.

Exception
 NoMemoryError
 ScriptError
   LoadError
   NotImplementedError
   SyntaxError
 SignalException
   Interrupt
 StandardError
   ArgumentError
   IOError
     EOFError
   IndexError
   LocalJumpError
   NameError
     NoMethodError
   RangeError
     FloatDomainError
   RegexpError
   RuntimeError
   SecurityError
   SystemCallError
   SystemStackError
   ThreadError
   TypeError
   ZeroDivisionError
 SystemExit
 fatal

So any error that can happen in Ruby will be a part of one of the above classes. Let's see a few of them just to make things clear.

NoMemoryError - This is raised when memory allocation fails and the application must halt.
SignalException::Interrupt - This is raised when you press ctrl + c to stop your program.
ScriptError::SyntaxError - Syntax errors mean that when your program comes across things like p "this quote is not closed, it will raise this.
__ StandardError__ - As the name implies, StandardError is the most common or standard type of exception Ruby will raise. Most of the usual errors you see are part of this.

Raising these exceptions is Ruby's way of telling us something is wrong and what that is.

What does Raise actually mean?

When an exception is raised (either explicitly with raise or implicitly by the Ruby runtime), an instance of an Exception class (or one of its subclasses) is created. This object contains information about the exception, such as an error message and a backtrace.
It also means that an exception object has been created and the normal flow of program execution has been interrupted.

NOTE: This is not like return ExceptionObj

This process involves more than just returning an object; it's a specific type of control flow mechanism, in which the normal flow of program execution has been interrupted.

Let's understand with a simple example:

def divide_numbers(x, y)
  result = x / y
  puts "Result of division is #{result}"
end

puts "Program starts"
divide_numbers(10, 0)
puts "Program ends"

The program prints "Program starts".
The divide_numbers method is called with 10 and 0 as arguments.
Inside divide_numbers, the division x / y is attempted. Since y is 0, Ruby raises a ZeroDivisionError.
The exception interrupts the normal flow of the program. The lineputs "Result of division is #{result}" is never executed because the exception has been raised before this line.
Since there's no rescue block to catch the ZeroDivisionError, the exception propagates up the call stack. In this case, it propagates back to the top level of the script.
No part of the script handles the exception, so it causes the program to terminate.

The final puts "Program ends" line is never executed because the program has already been interrupted and terminated by the unhandled exception.
When you run this script, you'll see the "Program starts" message, followed by an error message indicating a ZeroDivisionError, and the "Program ends" message will not be displayed.

So, returning an exception object is fundamentally different from raising one. Returning an exception object treats it like any other value, requiring explicit checks and handling by the caller. Hence, an exception is always raised and not returned.

If you want to manually raise an exception, you can do so using raise or fail in Ruby like this

raise "This will raise an exception!"

By default, if you don't specify anything, this will assume the exception is a RuntimeError.

The raise method accepts arguments in this format
raise [EXCEPTION_CLASS], [MESSAGE], [BACKTRACE]
There is enough documentation on how these methods work, so I won't dive into them.

An Exception is raised, what now?

Now that we understand an exception will break the flow and is just thrown out there for someone to "catch" it, we must mindfully handle these.

For eg: If in a Rails controller flow, an exception is raised and you don't handle it manually, Rails middleware will handle it and give a 500 Internal Server Error.

Ruby gives you the begin..rescue..ensure..end block to handle these exceptions.

Note: Ruby also has a BEGIN...END block too, which is different from the begin..end block.

This is how the being..rescue block looks like

begin
  raise 'This exception will be rescued!'
rescue StandardError => e
  puts "Rescued: #{e.inspect}"
end

If you don't mention anything in rescue, by default Ruby rescues the StandardError class. This is very much intended. As to why the default is not the Exception class, this will give you a clear idea. https://www.honeybadger.io/blog/ruby-exception-vs-standarderror-whats-the-difference/

NOTE: You should never rescue the Exception class.

Now if we want to write the division code shown before with proper error handling, it will be

def divide(a, b)
  begin
    result = a / b
  rescue ZeroDivisionError => e
    puts "Error: #{e.message}"
    result = nil
  end
  return result
end

puts divide(10, 2)   # Output: 5
puts divide(10, 0)   # Output: Error: divided by 0

Great! What next?

Okay, so we now understand failures, exceptions, errors and how to address them. The big question is what are the best practices around this? When should I raise an exception manually? How should I write code so it is easy to manage errors?

Understanding these things is crucial to being confident that your code works as it was intended to. We will cover these things in the next article. But before I go, I will leave you with this quote from Code Complete

Throw an exception only for conditions that are truly exceptional. Exceptions should be reserved for conditions that are truly exceptional—in other words, for conditions that cannot be addressed by other coding practices.

What the pluck?

Dhaval Singh — Fri, 17 Nov 2023 21:01:40 +0000

ActiveRecord is a great ORM, with a ton of nifty little things that make your life easy when dealing with DB queries. pluck is one such method. I've been using pluck for as long as I can remember, and even my second oldest article is on it!

Even after using it for so long, recently I found myself debugging a relatively simple query that used pluck and realized I missed some key points regarding this sweet little AR method.

The Rails Guides on pluck is pretty neat, but there are still some things you might miss so I'll try to sum it all up here. If you're just starting out or might have overlooked the documentation, I highly recommend going through this. However, if you're already versed in AR/pluck, feel free to jump to the concluding sections.

What is pluck and how does it work?

Pluck is an AR query method that selects one or more attributes(or columns) and returns an Array, without loading the corresponding records. It uses select internally and triggers a query for the values.

Customer.pluck(:id)
# SELECT customers.id FROM customers
=> [1,2,3]
# or
Customer.pluck(:id, :first_name)
# SELECT customers.id, customers.first_name FROM customers
=> [[1, "David"], [2, "Fran"], [3, "Jose"]]

There are multiple ways to do the above in Rails, pluck is the most performant out of all in almost all cases.
Basically, you are replacing code that looks like this

Customer.select(:id).map { |c| c.id }
# or
Customer.select(:id, :first_name).map { |c| [c.id, c.first_name] }

to be cleaner and more efficient.

So why is it faster?

pluck directly converts the data into a Ruby array skipping the construction of AR objects, thus giving it an edge in performance. Here's an article benchmarking pluck vs select vs collect/map.

There are some more points that are well-documented in the guides, so I'll just list them here.

You are not limited to querying fields from a single table, you can query multiple tables as well.

Order.joins(:customer, :books).pluck("orders.created_at, customers.email, books.title")

pluck triggers an immediate query, and thus cannot be chained with any further scopes, although it can work with scopes already constructed earlier:

Customer.pluck(:first_name).limit(1)
=> NoMethodError: undefined method `limit' for #<Array:0x007ff34d3ad6d8>

Customer.limit(1).pluck(:first_name)
=> ["David"]

pluck will trigger eager loading if the relation object contains include values, even if the eager loading is not necessary for the query. As you can see in the example, we don't need the join for customer ids, but pluck will still go ahead and do it! So be careful when you have eager-loaded associations!

irb> assoc = Customer.includes(:reviews)
irb> assoc.pluck(:id)
SELECT "customers"."id" FROM "customers" LEFT OUTER JOIN "reviews" ON "reviews"."id" = "customers"."review_id"

# to avoid this you can do
assoc.unscope(:includes).pluck(:id)

Now we have seen what's in the guides, let's see some cases that might trip you up!

Plucking jsonb fields in Rails

If you have a jsonb field in your AR object and you want some nested value you can still use pluck!

user.metadata = {
  "location": {
    "city": "New York",
    # ... additional data
  }
  # ... other metadata attributes
}

# Using 'pluck' to retrieve the 'city' values from the 'location' key in 'metadata'
cities = user.pluck("metadata -> 'location' ->> 'city'")
=> ["New York"]

Pluck's Overriding Power

Since pluck uses select internally. it will override any other selects you do before this. This seems obvious but when writing a complex query one might miss out on this. I'll try to illustrate with a relatively simple example

# Scenario: You want to fetch distinct cities from a user table

# Step 1: Using 'select' with 'distinct' to get unique cities
distinct_cities_query = User.select(:city).distinct

# Step 2: Attempting to use 'pluck' to retrieve the city names
cities = distinct_cities_query.pluck(:city)
# Cities might contain duplicate city names

Plucking with where

Another common issue can arise when using pluck with where (remember, pluck fires an immediate query). Hence, it can fire some extra queries where it's not required. Thankfully there’s a PluckInWhere Rubocop you can use to warn you.

# bad
# will trigger 2 queries
Post.where(user_id: User.active.pluck(:id))
# SELECT id FROM users WHERE /* conditions for active users */;
# SELECT * FROM posts WHERE user_id IN (1, 2, 3, ...);  -- Array of IDs from the first query

# good
# triggers 1 query only
Post.where(user_id: User.active.select(:id))
# SELECT * FROM posts WHERE user_id IN (SELECT id FROM users WHERE /* conditions for active users */);

When to think twice before using pluck?

When you have already loaded the AR objects into memory there is no need to use pluck as it will trigger another query. Rather just use map or collect.
When dealing with large datasets be careful with pluck as it will load everything into your memory.

# Instead of using pluck which can consume a lot of memory
# User.pluck(:email).each do |email|
#   # Process email
# end

# Use find_each or find_in_batches for batch processing
User.find_each(batch_size: 1000) do |user|
  email = user.email
  # Process email
end

In conclusion, pluck is a versatile and powerful method in Rails for efficiently retrieving specific columns from a database, but it's crucial to understand its nuances and limitations.
Happy Plucking :D

Understanding Rails Callbacks & Common Pitfalls

Dhaval Singh — Mon, 30 Oct 2023 19:53:27 +0000

Context

I recently spent a good chunk of time debugging a bug and in turn, did a lot of research on how Transactions and Touch work with callbacks like after_commit. Below is the debugging "story" I shared on X, if you like reading those, give it a shot but it's not required for this reading.

Story

Foreword

Callbacks are Evil. There, I said it. Whew, feels good to get this off my chest in the first line itself. Okay, you may ask why or just think I am not a seasoned RoR dev yet to be making this statement(which, by the way, I’m not). But hear me out.

One of the crowning achievements of Rails is its ability to facilitate rapid development
enabling businesses and ideas to scale quickly. But with this speed comes a potential pitfall.
What I mean is that you do something in one place of the codebase and soon that pattern will get replicated everywhere. The "intent" of doing something gets lost pretty quickly if someone doesn't keep an eye on it and more so as the codebase gets bigger and complexity goes through the roof. This can make or break an application in the long run.

There will always be examples of "Oh we were able to do XYZ, if you can't that means you are not doing it right". But here’s the thing: in a framework as powerful as Rails, which often feels magical in its abstraction of complexities, it's really easy to misuse features beyond their intended purpose.

Callbacks are one of those abused things.

Now, the point of writing this next part is that I want to share a few edge cases I've come across. Think of this as a compilation of Callback gotchas that one might easily miss.

Callback Gotachs

Callbacks are easy and very convenient to use and that is the worst part.

Now the Rails Docs doesn't state anywhere what should ideally go in a callback. It shows how many different types are there and what they do, but what goes into them is still a blur. So you really need to be sure of what you put inside your callbacks before it's too late. But this post is not about what goes in a callback. This is about how it can surprise you once you are already dealing with it.

Some of the things I will mention are either in the API Docs or the Rails guide, but a few aren't mentioned anywhere(and can actually trip you up). Here are a few things you should keep in mind when dealing with callbacks.

1. Callback ordering:

The non-transactional callbacks run in the order they are defined.

But..

From Rails 7.1 this can be altered via configuration.

config.active_record.run_after_transaction_callbacks_in_order_defined = false

The default value from 7.1 is true.

2. Multiple transactional callbacks with the same method name override each other:

This means only the last one is executed. This goes for after_commit, after_*_commit, after_rollback, etc)

The above 2 are mentioned in the Rails Guide, thankfully. The next one is not!

3. Multiple instances of the same record in a transaction:

In the context of a single transaction, if you interact with multiple loaded objects that represent the same record in the database, there's a crucial behavior in the after_commit and after_rollback callbacks to note. These callbacks are triggered only for the first object of the specific record that undergoes a change within the transaction. Other loaded objects, despite representing the same database record, will not have their respective after_commit or after_rollback callbacks triggered.

As you can see in the above example, only 2 callbacks are executed. This behavior is not documented anywhere. I've raised a PR to add this in the guides, let's see what happens. Anyway, you need to be careful about this.

Some additional gotchas

There are a few more surprises in terms of when a rollback happens, and how Rails handles it, especially with nested transactions.
Using nested transactions is considered an anti-pattern in a lot of cases, so think twice before using it.

Bonus: Confusing behavior of Touch with Transaction and `after_commit`

Now this is not a bug, but for me, it was the lack of clarity/docs on this that was confusing.
Let's understand this with an example

In the above example, the touch SQL queries are coalesced at the end to be more efficient but the Car model is actually touched after each touch hence triggering the after_commits in a specific order. This can be especially confusing when you are debugging stuck in something I like to call Callback Hell. This is also not mentioned anywhere, hence I raised another PR to add this to the API docs.

Overall, callbacks just seem like a bad design, because they are really powerful but so easy to use. It's more of a bad design based on how a human brain works rather than a programming one, but if your codebase already has a lot of these and you need to work with them a general rule to keep in mind when dealing with it would be

Try without a callback, if you really think it belongs there then move it into a callback. But don't start with a callback.

From Bloat to Sleek: A Rails Memory Tale With Jemalloc

Dhaval Singh — Fri, 27 Oct 2023 11:31:15 +0000

Prelude

In my previous blog we tried to analyze what could be the reason behind a Rails app hogging so much memory. Now there are multiple ways to approach this issue from fixing N+1 queries to profiling shady APIs and debugging them, my most fruitful discovery was a shift to a memory allocator named jemalloc.

This term kept popping up, and, initially, it was a complete enigma to me.

Memory Surges in Multi-Threaded Environment

As mentioned in my previous post, we also migrated to Puma from Unicorn during the K8s migration. Now Unicorn is single-threaded while we started running Puma with 5 threads in production and boom, the memory usage skyrocketed.

This happens due to something called per-thread memory arenas in the default glibc malloc. While I won't delve deep into the intricacies this post by Nate Berkopec does it beautifully.

For those who just want an ELI5 version, continue on. Otherwise, feel free to jump to the next section.

ELI5: Per-thread memory arenas

Playroom & Rules: Imagine a playroom with kids (threads). There's a rule: only one kid can tell a story (execute Ruby code) at a time. This is because of the "Storytelling Rule" (akin to the GVL).

Kids & Toy Boxes: Each kid has their own toy box (per-thread memory arena). When they play, they first look for toys in their personal box.

Main Toy Chest: There's also a big shared toy chest in the middle (main memory arena). If a kid can't find the toy they want in their personal box, they might go to this big chest to get one.

Messy Toy Situations: Sometimes, kids don't put toys back neatly. Toys spread out, and soon there's wasted space in the toy boxes and the main toy chest. This is like memory fragmentation.

Balancing Toys & Storytime: Having many toy boxes means kids can quickly find toys, but it might lead to more mess (fragmentation). Fewer boxes mean tidier storage but might make playtime less spontaneous.

This is oversimplified but you get the idea. In short, fewer memory areans mean less fragmentation but slower programs.

Addressing the Memory Quandary

Reducing the memory areans seems like a straightforward way which you can do by setting the environment variable MALLOC_ARENA_MAX anywhere from 2-4 (optmial for Ruby).

But changing the memory allocator seemed to be more effective in other cases. Enter jemalloc. While jemalloc mirrors malloc in its use of per-thread arenas, it manages them more efficiently.

Configuring Ruby to harness jemalloc is a breeze. Here's a snapshot of my setup process:

I've adopted jemalloc version 5.3.0. A quick peek revealed that Gitlab utilizes this same version, which cemented my choice.

A word of caution as you embark on this: jemalloc has had its share of hiccups when paired with Ruby in Alpine-based Docker images, thankfully we are not using it.

The Outcome

To my surprise, it didn’t have any negative side effects(at least none I could see) and it performed way better than I expected.

As you can see, each pod pre-jemalloc was gobbling up an average of 1.6GB. Post-jemalloc, this consumption plummeted to a mere 550 MB.

In aggregate terms, total memory usage took a nosedive from 16GB down to 4GB, marking a staggering 75% decrease!

We saw similar effects on our other services also, not as pronounced but still significant.

The biggest winner, no doubt were our Sidekiq workers(as they can utilize up to 32 threads, I think this is way too high, will work on this soon enough)

Moving Forward

While I personally haven't encountered a scenario where jemalloc underperformed, the Ruby core team have their specific reasons. Nevertheless, implementing jemalloc appears to be a worthwhile endeavor. I'd strongly recommend giving it a shot. If not jemalloc, tweaking the MALLOC_ARENA_MAX might also be beneficial.

The outcome? More efficient pods in terms of size and a noticeable reduction in AWS costs. It's definitely worth a try to determine what works best for your setup.