Writing my first module in Python was a confusing experience. As it usually happens, when I was testing it in the interactive Python REPL, the first version turned out to have some bugs (the second and third ones also did đ).
Thatâs fine - I thought - I will just fix the module and reimport it.
But it turned out that calling from my_module import my_function
didnât update the code! my_function
still had the bug that I just fixed! I double-checked if I modified the correct file, reimported it again and still nothing. It turns out, as StackOverflow kindly explained, that you canât just reimport a module. If you already imported a module (import a_module
) or a function (from a_module import a_function
) in your Python session and you try to import it again, nothing will happen. It doesnât matter if you use the standard Python REPL or IPython.
How does importing in Python work?
Turns out that, for efficiency reasons, when you import a module in an interactive Python session, Python interpreter does two steps:
- First, it checks if the module is already cached in the
sys.module
dictionary. - And only if itâs not there, it actually imports the module.
Which means that, if you already imported the module (or imported a different module that references this one) and you try to import it again, Python will ignore this request. You can read more about how importing works in the documentation.
So, if I canât reimport a module, does it mean that I have to restart Python each time? Not really, that would be very inconvenient.
How to reimport a module?
The easiest way is to quit your interactive session and start it again. It works fine if you donât care about preserving the data that you already have in your session, like the functions that you wrote and the variables that you calculated. But usually you donât want to restart the REPL, so there are better ways.
Since we know that the interpreter will first look for the module in the sys.modules
dictionary, we can just delete our module from this dictionary. And it will work in most cases, but there are some caveats. If your module is referenced from another module, there is a chance that you still wonât be able to reimport it. So donât do this. There is a better way.
The recommended solution is to use the importlib.reload
function. This function is designed exactly for reimporting modules that have already been imported before. To reload your module, you need to run:
import importlib
importlib.reload(my_module)
So thatâs how you can reimport a module in Python. And if you are not using IPython, this is where your options end. But IPython users have some other interesting solutions to this problem.
%run
If you donât care about actually âimportingâ your module and all you need is to run some functions defined in a file, you can execute that file instead. It will run all the commands as if you would copy and paste them in your IPython session. You can rerun a file as many times as you want and it will always update all the functions. Running a file in IPython is extremely easy:
%run my_file.py
# You can even skip the ".py" extension:
%run my_file
I cheated a bit when I said that this option is not available in standard Python REPL. It is, but it requires more typing:
exec(open("./my_file.py").read())
To be honest, if I had to type all this, I might as well just use the importlib.reload
instead.
All those options are great, but if you are as bad as me when it comes to writing code and you make a lot of mistakes, then it means a lot of reloading. And typing this importlib.reload
/ %run
/ exec...
is annoying. Wouldnât it be great if there was a way to automatically reload a module? Well, IPython can actually do that!
%autoreload to the rescue
Another one of the magic methods in IPython is related to reloading modules. Itâs called %autoreload. Itâs not enabled by default, so you have to load it as an extension:
%load_ext autoreload
Now, you can turn on auto-reloading:
%autoreload 2
And each time you execute some code, IPython will reimport all the modules to make sure that you are using the latest possible versions.
There are 3 configuration options that you can set:
-
%autoreload 0
- disables the auto-reloading. This is the default setting. -
%autoreload 1
- it will only auto-reload modules that were imported using the%aimport
function (e.g%aimport my_module
). Itâs a good option if you want to specifically auto-reload only a selected module. -
%autoreload 2
- auto-reload all the modules. Great way to make writing and testing your modules much easier.
Great, any caveats? I found 3 minor ones:
- IPython with %autoreload enabled will be slightly slower. IPython is quite smart about what to reload. It will check the modification timestamps of the modules and compare them with the time when they are imported. But this checking (and eventually reimporting of the modified modules) will still take some time. It wonât be so slow that you will feel it (unless you have modules that take seconds to import), but it will obviously run faster if you disable the auto-reloading.
- As pointed out in the documentation, %autoreload is not 100% reliable and there might be some unexpected behaviors. I never noticed any problems, but you might, so Iâm just warning you.
- You need to make sure that you donât have syntax errors in your modules when you are running IPython commands. I often start writing some code in a file and, in the middle of the command, I switch to IPython to quickly test something. And when I execute some code in IPython, it will try to reimport the file that I just modified (the one with the half-written command) and throw a SyntaxError. The good thing is - after the error, you will still get the output of the command that you ran. So for me, itâs a minor annoyance, not a real problem. You can easily solve it by running two IPython sessions - one for testing the module (with %autoreload enabled) and the other for running some random commands and looking up things in the documentation.
Here is how %autoreload
works in practice:
So if you donât know %autoreload
yet, give it a try the next time you will be working on a module in Python!
Image from: Unsplash
Top comments (0)