Python's Descriptor Protocol, The Feature Behind Everything You Use Daily

#python #django #fastapi #backend

You use descriptors every day. Every time you write @property, every time you call a method, every time you use @staticmethod or @classmethod, you are using the descriptor protocol. Most Python developers have no idea it exists as a unified mechanism. Once you see it, you cannot unsee it.

What a Descriptor Is

A descriptor is any object that defines at least one of three methods: __get__, __set__, or __delete__. That is it. When an attribute lookup finds one of these objects sitting in a class's __dict__, Python hands control over to the descriptor instead of returning the object directly.

The minimal descriptor looks like this:

class MyDescriptor:
    def __get__(self, obj, objtype=None):
        print(f"__get__ called: obj={obj}, objtype={objtype}")
        return 42

class MyClass:
    attr = MyDescriptor()

instance = MyClass()
instance.attr   # prints: __get__ called: obj=<MyClass ...>, objtype=<class 'MyClass'>
MyClass.attr    # prints: __get__ called: obj=None, objtype=<class 'MyClass'>

When obj is None, the descriptor is being accessed from the class itself, not an instance. This distinction matters and you will use it constantly when writing real descriptors.

Data Descriptors vs Non-Data Descriptors

This split is the most important thing to understand about how descriptors interact with instance __dict__.

A data descriptor defines both __get__ and __set__ (or __delete__). It takes priority over the instance's own __dict__. You cannot shadow it by doing instance.attr = something because __set__ intercepts the assignment.

A non-data descriptor defines only __get__. The instance's __dict__ takes priority over it. If you assign to instance.attr, Python just stores the value in the instance dictionary and the descriptor is no longer called for that instance.

The lookup order for instance.attr is:

Data descriptors from the class (and its MRO)
Instance __dict__
Non-data descriptors and other class attributes

This ordering is not arbitrary. It is what makes @property work correctly. A property is a data descriptor because it defines __set__ (even if you only define a getter, the setter raises AttributeError), so it cannot be overridden by instance assignment. Functions, on the other hand, are non-data descriptors, which is why you can shadow a method by assigning to an instance attribute of the same name.

How Functions Become Methods

This is the part that clicks everything together. In Python, a plain function is a non-data descriptor. It only defines __get__. When you access a function through an instance, __get__ is called, and it returns a bound method object that already has the instance baked in as the first argument.

def greet(self):
    return f"Hello from {self}"

class Person:
    greet = greet

p = Person()

# These are equivalent
p.greet()
Person.greet(p)

# What actually happens
bound_method = Person.__dict__['greet'].__get__(p, Person)
bound_method()

function.__get__(instance, owner) returns a MethodType object that wraps the function and the instance together. There is no magic beyond this. The entire method binding mechanism in Python is the descriptor protocol applied to plain functions.

You can verify this:

print(type(greet))           # <class 'function'>
print(greet.__get__)         # <method-wrapper '__get__' of function object at ...>
print(type(p.greet))         # <class 'method'>

Writing a Real Descriptor: Typed Attributes

Here is where descriptors become practically useful. Suppose you want class attributes that enforce a type at assignment time without cluttering __init__ with validation logic.

class Typed:
    def __set_name__(self, owner, name):
        self.name = name
        self.private_name = f"_{name}"

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return getattr(obj, self.private_name, None)

    def __set__(self, obj, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(
                f"{self.name} must be {self.expected_type.__name__}, "
                f"got {type(value).__name__}"
            )
        setattr(obj, self.private_name, value)

class Integer(Typed):
    expected_type = int

class String(Typed):
    expected_type = str

class Person:
    name = String()
    age = Integer()

    def __init__(self, name, age):
        self.name = name
        self.age = age

p = Person("Alice", 30)   # works
p.age = "thirty"          # raises TypeError: age must be int, got str

Notice __set_name__. This is a hook Python calls automatically when the class is created, passing the descriptor the owner class and the attribute name it was assigned to. Without it, the descriptor would not know its own name and you would have to pass it manually. It was added in Python 3.6 and makes descriptors significantly cleaner to write.

classmethod and staticmethod Are Just Descriptors

Once you understand the protocol, classmethod and staticmethod stop being mysterious decorators and become straightforward descriptor implementations.

staticmethod wraps a function and its __get__ simply returns the function unchanged, with no binding:

class StaticMethod:
    def __init__(self, f):
        self.f = f

    def __get__(self, obj, objtype=None):
        return self.f  # no binding, just return the raw function

classmethod wraps a function and its __get__ binds the class instead of the instance:

class ClassMethod:
    def __init__(self, f):
        self.f = f

    def __get__(self, obj, objtype=None):
        if objtype is None:
            objtype = type(obj)
        return self.f.__get__(objtype)  # bind the class, not the instance

These are simplified but functionally accurate. The actual CPython implementations are in C for performance, but the logic is identical. You could replace @classmethod with your own descriptor and everything would work the same way.

The Caching Descriptor Pattern

One underused pattern is using descriptors to compute expensive values once and then cache them on the instance, bypassing the descriptor entirely on subsequent access.

class cached_property:
    def __init__(self, func):
        self.func = func
        self.attrname = None

    def __set_name__(self, owner, name):
        self.attrname = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        val = self.func(obj)
        obj.__dict__[self.attrname] = val  # store directly on instance
        return val

This works because cached_property is a non-data descriptor (it only defines __get__). On first access, it calls the function and stores the result in the instance __dict__ under the same name. On second access, the instance __dict__ lookup wins and the descriptor is never called again.

Python ships functools.cached_property since 3.8 using exactly this technique. It is elegant because the caching mechanism falls directly out of the descriptor lookup order rather than requiring any explicit flag or sentinel.

When to Actually Use This

Descriptors are the right tool in specific situations. If you find yourself writing the same property logic across multiple attributes or classes (validation, unit conversion, access logging), a descriptor lets you define that logic once and attach it declaratively.

ORMs lean on this heavily. Django's Field classes and SQLAlchemy's Column objects are descriptors. When you write user.name, the ORM intercepts it through __get__ and can return a SQL expression rather than a plain value, depending on context.

Frameworks that do attribute access tracking for reactive programming (like Traitlets in Jupyter or any observable pattern) use descriptors to intercept reads and writes and fire callbacks.

If you are not building framework-level infrastructure, plain properties cover most cases. But understanding descriptors tells you exactly what @property is doing and why it behaves the way it does, which makes the edge cases much easier to reason about.