The latest articles on DEV Community by Felix (@felixlast). https://dev.to/felixlast https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F61102%2Fdb89b22a-8f6c-4958-a4fc-ca10584a3689.png https://dev.to/felixlast en Felix Thu, 02 Jul 2020 08:13:38 +0000 https://dev.to/felixlast/django-migrations-leak-memory-in-debug-mode-59m https://dev.to/felixlast/django-migrations-leak-memory-in-debug-mode-59m <p>While optimizing a django database migration which iterated over batches of a large table (~40 Mio rows), I noticed that the memory usage would consistently grow. Memory consumption kept growing even after I <code>del</code>'ed all inner-loop variables and asked the garbage collector to do their work.</p> <p>I finally figured out that Django's <code>DEBUG</code> mode, which is <a href="https://www.dimagi.com/blog/django-orm-memory-leaks-in-debug-mode-73877/">well known to use up memory</a> for traceability, has the same effect in migrations, too. Adding the following two lines to your migration ensures smooth, memory-efficient processing, no matter the environment:<br> </p> <div class="highlight"><pre class="highlight python"><code><span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span> <span class="n">settings</span><span class="p">.</span><span class="n">DEBUG</span> <span class="o">=</span> <span class="bp">False</span> </code></pre></div> django migration memory