Modules are cached

Share
Copied to clipboard.
Series: Modules
Trey Hunner smiling in a t-shirt against a yellow wall
Trey Hunner
3 min. read Watch as video Python 3.8—3.12
Python Morsels
Watch as video
02:36

Python caches modules.

Re-importing modules doesn't update them

We have a points module here (a file called points.py) that contains a Point class:

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

Let's say we're testing out this code at the Python REPL and we decide that the current string representation for a Point object isn't friendly enough:

>>> from points import Point
>>> p = Point(1, 2)
>>> p
<points.Point object at 0x7fc5ee136970>

So we modify our Point class to add a __repr__ method:

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __repr__(self):
        return f"Point({self.x}, {self.y})"

Then we save the points.py file that this Point class lives in. And then we re-import this module:

>>> from points import Point

And make a new instance of this class:

>>> p = Point(1, 2)

And then look at its string representation (expecting it to have changed):

>>> p
<points.Point object at 0x7fc5ef33f220>

But then we notice that the string representation hasn't actually changed! It's the same as it was before.

Restarting the Python REPL to force-update our module

In a desperate attempt to figure out why our Point class's string representation won't update, we exit the Python REPL, restart the Python REPL, and do everything all over again.

>>> exit()
$ python3
Python 3.10.0
Type "help", "copyright", "credits" or "license" for more information.
>>>

We import our Point class from our module, make a new instance of it, and then look at its string representation to see that it changed this time!

>>> from points import Point
>>> p = Point(1, 2)
>>> p
Point(1, 2)

What's going on here? Why did importing the module a second time not work but starting a new Python interpreter worked?

Python caches all imported modules

This all happened because Python caches modules.

In Python, every module that is imported is stored in a dictionary called sys.modules.

This dictionary maps the name of each module to the module object it represents:

>>> import sys
>>> sys.modules['points']
<module 'points' from '/home/trey/points.py'>

Each time the same module is imported again, Python doesn't actually reevaluate the code for that module: it just gives us back the same module object as before.

>>> import points
>>> points
<module 'points' from '/home/trey/points.py'>

How to clear the cache for a module

This module caching that Python does is a feature, not a bug: Python does this for performance reasons. But this feature does start to feel like a bug when we're testing our code from the Python REPL while also making changes to our code.

The easiest way to fix this problem is to exit the REPL and start a new REPL, starting a brand-new Python process (which has a brand new sys.modules dictionary). But there's other ways to do this as well.

You could also try to modify sys.modules, deleting things from it to clear the cache for that module:

>>> del sys.modules['points']

Or you could use the reload function from Python's importlib module, which reloads a module object:

>>> from importlib import reload

In either case, you might not actually fully fix your problem though. Because reloading a module doesn't delete references to old versions of classes as well as instances to old versions of classes. So the easiest way to fix this really is to exit the REPL and restart it.

Exit the REPL to refresh your cached Python modules

Python caches modules. So while developing your code as you're testing your code in a Python REPL, keep in mind that when you re-import a module, Python will used the cached version of your module instead of reevaluating all the code in your module.

To really refresh a module, you should exit the Python REPL and start a new REPL.

Series: Modules

Modules are the tool we use for breaking up our code into multiple files in Python. When you write a .py file, you're making a Python module. You can import your own modules, modules included in the Python standard library, or modules in third-party packages.

To track your progress on this Python Morsels topic trail, sign in or sign up.

0%
Concepts Beyond Intro to Python

Intro to Python courses often skip over some fundamental Python concepts.

Sign up below and I'll share ideas new Pythonistas often overlook.

Python Morsels
Watch as video
02:36