shayan holakouee

Posted on Apr 29

Python Metaclasses, What's Actually Happening When You Write class

#python #django #computerscience #webdev

Every Python developer knows that everything is an object. What fewer know is that classes are objects too, and like every other object, they are instances of something. That something is a metaclass. Understanding metaclasses means understanding how Python builds classes in the first place, which turns out to explain a lot of behavior that otherwise looks like magic.

Classes Are Instances of type

Start here:

class MyClass:
    pass

print(type(MyClass))    # <class 'type'>
print(type(int))        # <class 'type'>
print(type(str))        # <class 'type'>
print(type(type))       # <class 'type'>

type is the metaclass of every class you have ever written unless you said otherwise. It is the class of classes. It is also an instance of itself, which is one of those things that is true and also slightly uncomfortable to think about for too long.

Since type is a class, you can call it to create new classes at runtime. The three-argument form of type does exactly this:

# type(name, bases, namespace)
MyClass = type("MyClass", (object,), {"x": 42, "greet": lambda self: "hello"})

obj = MyClass()
print(obj.x)        # 42
print(obj.greet())  # hello

This is not a trick or an edge case. This is literally what happens when Python processes a class statement. The class syntax is a shorthand for calling type (or a custom metaclass) with the class name, base classes, and namespace.

The Class Creation Protocol, Step by Step

When Python encounters a class block, it does not just evaluate the body and hand it to type. It follows a specific protocol.

Step 1: Determine the metaclass.

Python looks for a metaclass keyword argument in the class definition. If not found, it checks the base classes for their metaclass. If no bases specify one, it defaults to type.

class Meta(type):
    pass

class MyClass(metaclass=Meta):
    pass

print(type(MyClass))  # <class '__main__.Meta'>

Step 2: Prepare the namespace.

Python calls metaclass.__prepare__(name, bases, **kwargs). This returns the dictionary that will be used as the class namespace while the body executes. By default it returns a plain dict, but you can return an OrderedDict or a custom mapping to do things like track definition order or intercept attribute assignments.

class OrderedMeta(type):
    @classmethod
    def __prepare__(mcs, name, bases, **kwargs):
        return {}  # default, but you can return anything dict-like

Step 3: Execute the class body.

The class body runs as a function, with the namespace from step 2 as its local scope. Every name defined in the body becomes a key in that namespace. This is why you can do things like:

class MyClass:
    items = []
    for i in range(3):
        items.append(i)  # this runs at class definition time

Step 4: Call the metaclass.

Python calls metaclass(name, bases, namespace) with the populated namespace. This is where the class object is actually created. This is type.__call__, which internally calls type.__new__ to create the class object and type.__init__ to initialize it.

The full chain: Meta(name, bases, ns) → Meta.__call__ → Meta.__new__ + Meta.__init__.

Writing a Metaclass That Does Something Useful

The canonical example of a metaclass that earns its complexity is one that enforces an interface. Suppose you want an abstract base pattern without the abc module, where subclasses that forget to implement required methods fail at class definition time rather than at instantiation.

class RequiredMethodsMeta(type):
    required = []

    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)

        # skip the check for the base class itself
        if bases:
            missing = [
                method for method in mcs.required
                if method not in namespace
            ]
            if missing:
                raise TypeError(
                    f"{name} must implement: {', '.join(missing)}"
                )
        return cls


class Shape(metaclass=RequiredMethodsMeta):
    RequiredMethodsMeta.required = ["area", "perimeter"]


class Circle(Shape):
    def area(self):
        return 3.14 * self.radius ** 2

    def perimeter(self):
        return 2 * 3.14 * self.radius

# This class raises TypeError at definition time, not at use time
class Square(Shape):
    def area(self):
        return self.side ** 2
    # missing perimeter
# => TypeError: Square must implement: perimeter

The error happens when the class statement for Square is processed, before any instance is ever created. This is the key advantage of metaclass-level enforcement over runtime checks.

new vs init in a Metaclass

When you subclass type, the two methods you override are __new__ and __init__. The distinction matters.

__new__ creates and returns the class object. This is where you modify the class before it exists as a complete object. You can alter the namespace, add or remove attributes, change the bases. The class has not been fully constructed yet.

__init__ receives the already-created class object and can perform post-construction setup. The class exists at this point but has not been returned to the caller yet.

class TrackedMeta(type):
    registry = []

    def __new__(mcs, name, bases, namespace):
        # Modify namespace before class is created
        namespace["_created_by"] = "TrackedMeta"
        cls = super().__new__(mcs, name, bases, namespace)
        return cls

    def __init__(cls, name, bases, namespace):
        # Class exists, register it
        TrackedMeta.registry.append(cls)
        super().__init__(name, bases, namespace)


class Base(metaclass=TrackedMeta):
    pass

class Child(Base):
    pass

print(TrackedMeta.registry)  # [<class 'Base'>, <class 'Child'>]
print(Base._created_by)      # TrackedMeta

The practical rule: use __new__ when you need to alter what the class is. Use __init__ when you just need to register or observe it after it is created.

prepare and Definition Order

__prepare__ is underused and often the cleaner solution to problems people solve with __new__. Because it controls the namespace used during class body execution, it can observe the order in which things are defined.

A plain dict preserves insertion order in Python 3.7+, so you get definition order for free now. But __prepare__ lets you go further: you can use a custom mapping that intercepts assignments, rejects duplicates, or transforms values as they are added.

class NoDuplicatesMeta(type):
    @classmethod
    def __prepare__(mcs, name, bases, **kwargs):
        return NoDuplicatesDict()

    def __new__(mcs, name, bases, namespace):
        return super().__new__(mcs, name, bases, dict(namespace))


class NoDuplicatesDict(dict):
    def __setitem__(self, key, value):
        if key in self and not key.startswith("_"):
            raise TypeError(f"Duplicate attribute: {key}")
        super().__setitem__(key, value)


class MyClass(metaclass=NoDuplicatesMeta):
    x = 1
    y = 2
    x = 3  # raises TypeError: Duplicate attribute: x

This catches the duplicate at class body execution time, not after. The error message points at the right place.

Metaclass Conflict and the MRO Problem

If you use multiple inheritance and the base classes have different metaclasses, Python raises a TypeError unless one metaclass is a subclass of all the others.

class MetaA(type): pass
class MetaB(type): pass

class A(metaclass=MetaA): pass
class B(metaclass=MetaB): pass

class C(A, B): pass
# TypeError: metaclass conflict: the metaclass of a derived class
# must be a subclass of the metaclasses of all its bases

The fix is to create a combined metaclass that inherits from both:

class MetaC(MetaA, MetaB): pass

class C(A, B, metaclass=MetaC): pass  # works

In practice, metaclass conflicts are a strong signal that the design has gotten complicated. If you are writing three metaclasses that need to be composed, consider whether __init_subclass__ or a class decorator solves the same problem more simply.

init_subclass: The Metaclass You Did Not Have to Write

Python 3.6 added __init_subclass__, which handles the most common metaclass use case without requiring a metaclass at all. It is called on the base class whenever a subclass is created.

class Plugin:
    _registry = {}

    def __init_subclass__(cls, plugin_name=None, **kwargs):
        super().__init_subclass__(**kwargs)
        if plugin_name:
            Plugin._registry[plugin_name] = cls


class JSONPlugin(Plugin, plugin_name="json"):
    def process(self, data): ...

class XMLPlugin(Plugin, plugin_name="xml"):
    def process(self, data): ...

print(Plugin._registry)
# {'json': <class 'JSONPlugin'>, 'xml': <class 'XMLPlugin'>}

No metaclass, no __new__, no type subclass. __init_subclass__ is called automatically at class creation time with whatever keyword arguments were passed in the class definition. For registration patterns, interface enforcement, and configuration at definition time, this handles most cases cleanly.

When a Metaclass Is Actually the Right Tool

Metaclasses are the right tool in a narrow set of situations. They make sense when you need to transform or validate the class namespace before the class object exists, when you need behavior that __init_subclass__ cannot express because it does not run at all for the base class itself, and when you are building framework infrastructure that other developers will subclass without knowing the mechanics.

Django's ORM uses metaclasses. SQLAlchemy uses metaclasses. The abc module uses a metaclass. These are all framework-level concerns where the metaclass is invisible to the end user.

If you are writing application code and reaching for a metaclass, stop and ask whether a class decorator or __init_subclass__ solves the problem. They almost always do, and they are significantly easier to understand, test, and debug. A metaclass that the next developer has to spend an hour understanding is a liability. The best metaclass is usually the one you did not have to write.

DEV Community