Nearly a year ago, we noticed that the server’s memory was being used up much quickly than before. At first, it was not a big problem because the server was restarted during releases, and the memory was cleared. However, over time, the situation worsened: the memory ran out in just a few days, then in two days, and eventually, the server would only work for a few hours before crashing.
Therefore, we needed a thorough analysis of memory usage. For this, we installed the stackprof gem on production and let it dump the memory report usage for a couple of days. After analyzing the reports, we found out that the cause of the leaks was large text strings (hundreds of thousands of lines) generated every time the server responded to a request.
Before that, we used the active_model_serializers gem, which was converting data into JSON using the as_json method. While converting to JSON, a large number of string objects were allocated in memory and were not garbage-collected. As a result, memory usage increased rapidly from about 500 KB to 10-20 MB per request. Since our server had only 2 GB of RAM, all the memory was quickly used up, leading to restarts and system freezes.
So, we identified that the main problems were inefficient serialization, large amounts of created unreleased string objects, and using old gems that were not designed for handling large amounts of data well. What should we do next? Read more for detailed insights.
Top comments (0)