DEV Community

Prashant Sharma
Prashant Sharma

Posted on • Updated on • Originally published at sharmapacific.in

Metaclasses in Python

#productivity #metaclasses #python #tutorial

In most programming languages, classes are just pieces of code that define the rules for an object, but in Python, as you must hear that everything is an object: it turns out that this is true of classes themselves. classes are actually first-class objects, they can be created at runtime, passed as parameters, returned from functions, and assigned to variables.

let's look at the below example -

class Tutorial:
    pass

print(Tutorial())

# Output - 
# <__main__.Tutorial object at 0x7fd92c1500f0>
Enter fullscreen mode Exit fullscreen mode

as we can see the instance of Tutorial class tells us that this is an object of the main Tutorial object. at some location.

Now just print the class itself -

print(Tutorial)

# Output - 
# <class '__main__.Tutorial'>
Enter fullscreen mode Exit fullscreen mode

The reason we're able to do this because the Tutorial class is an object, just like any other object. When you use the class keyword, Python creates this object automatically. It's an instance of a metaclass - type.

Python metaclasses

A metaclass is the class of a class; it defines how a class behaves.

Now for making it easier, let go into depth, Maybe you have come across the type keyword in Python? Which used to finding the Type of an Object -

print(type(1))

# Output - 
# <class 'int'>

print(type('Hey'))

# Output - 
# <class 'str'>
Enter fullscreen mode Exit fullscreen mode

As expected 1 is the type of int class, Hey is the type of str class, let's find out the Type of our Class -

print(type(Tutorial))

# Output - 
# <class 'type'>
Enter fullscreen mode Exit fullscreen mode

This time, we get a printout that Tutorial is of type of class type. But what about the type itself? What is the type's type?

print(type(type))

# Output - 
# <class 'type'>
Enter fullscreen mode Exit fullscreen mode

the type of type is class type, you maybe find this weird. Thus we find that type is also its own metaclass!

Understanding How Metaclasses Work

type is a built-in metaclass in python. it is used to construct classes just like a class is used to construct the objects. So Whenever we create any class then the default metaclass(type) gets called and giving us an option to use it as an object.
That means every class in python is also an object of type, Therefore We can use type directly to make a class, without any class syntax. The type() function can be used to directly define classes by using the following three arguments -

type(<name>, <bases>, <dct>)
Enter fullscreen mode Exit fullscreen mode

  • name - This is the internal representation of the class. This is the name of the class.

  • bases - This specifies anything that we inherit from a superclass or a parent class. This is a tuple of the parent class.

  • dct - This specifies a namespace dictionary containing definitions of class's methods and variables.

To make it more clear -

Test = type('Test', (), {})
Enter fullscreen mode Exit fullscreen mode

so the above line of code is completely equivalent to this below code -

class Test:
    pass
Enter fullscreen mode Exit fullscreen mode

there's absolutely nothing different about them.

Hence, if we want to modify the behavior of classes, we will need to write our own custom metaclass.

To create our own custom metaclass, we first have to inherit the default metaclass type, and implement the metaclass's __new__ method and/or __init__ method.

  • __new__: This dunder method is usually overridden type's __new__, to modify some properties of the class to be created,
    before calling the original __new__ which creates the class.

  • __init__: This method is called when you want to control the initialization after the instance/object has been created.

class MyMeta(type):
    def __new__(self, name, bases, attr):
        print(attr)
        return type(name, bases, attr)
Enter fullscreen mode Exit fullscreen mode

So here I have defined a simple metaclass MyMeta, and we're just going to print out the attributes so we can see how they look like. after that, I have defined another class Sample and it's having the metaclass MyMeta, with name and age variable -

class Sample(metaclass=MyMeta):
    name = 'bob'
    age = 24
Enter fullscreen mode Exit fullscreen mode

Now without creating an instance this stills runs as you can see the output below -

{'__module__': '__main__', '__qualname__': 'Sample', 'name': 'bob', 'age': 24}
Enter fullscreen mode Exit fullscreen mode

So how this class Sample has been created -

Interpreter sees metaclass=MyMeta defined in Sample, So now the interpreter has information that default metaclass type must not be used to create Sample class, Instead, MyMeta must be used to create Sample class.

So, the interpreter makes a call to MyMeta to create class Sample, When MyMeta is got called, __new__ of MyMeta is invoked and it prints out the attributes and using type its construct the instance of MyMeta which is Sample and returns to us the object.

This metaclass only overrides object creation. All other aspects of class and object behavior are still handled by type.

Now We've covered enough theory to understand what metaclasses are and how to write custom metaclasses. Now let's look a simple real case scenario -

let's suppose we have a requirement that all attributes of your class should be in upper case, There are multiple ways to achieve this functionality, but here we are going to achieve this using metaclass at the module level, So if in namespace dictionary(attributes) if a key doesn't start with a double underscore we need to change it to be uppercase -

class MyMeta(type):
    def __new__(self, name, bases, atts):

        print(f'current_attributes - {atts}\n')
        new_atts = {}

        for key, val in atts.items():
            if key.startswith('__'):
                new_atts[key] = val
            else:
                new_atts[key.upper()] = val

        print(f'modified_attributes - {new_atts}')        
        return type(name, bases, new_atts)


class Sample(metaclass=MyMeta):
    x = 'bob'
    y = 24

    def say_hi(self):
        print('hii')
Enter fullscreen mode Exit fullscreen mode

Output -

current_attributes - {'__module__': '__main__', '__qualname__': 'Sample', 'x': 'bob', 'y': 24, 'say_hi': <function Sample.say_hi at 0x7fd92c10d048>}

modified_attributes - {'__module__': '__main__', '__qualname__': 'Sample', 'X': 'bob', 'Y': 24, 'SAY_HI': <function Sample.say_hi at 0x7fd92c10d048>}
Enter fullscreen mode Exit fullscreen mode

As you can see above we have printed the current attributes and created a dictionary that we used to represent our modified attributes.

Here we are just checking if the key starts with a double underscore then just adds the proper value otherwise we are adding the uppercase attribute with that corresponding value.

Now to make sure that this is working let's try, I have created an instance of Sample, and just print one of the old attributes.

s = Sample()
s.x
Enter fullscreen mode Exit fullscreen mode

Output -

AttributeError Traceback (most recent call last)
<ipython-input-18-87b2922593a9> in <module>
      1 s = Sample()
----> 2 s.x

AttributeError: 'Sample' object has no attribute 'x'
Enter fullscreen mode Exit fullscreen mode

As expected we have got an error AttributeError, which is totally fine because we have just modified the construction of the object, let's try to access by modified attributes -

print(s.X)

s.SAY_HI()
Enter fullscreen mode Exit fullscreen mode

Output -

bob
hii
Enter fullscreen mode Exit fullscreen mode

This is why they call it magic because with this kind of hook into the creation of classes you can really enforce quite a bit of constraint on how classes are created

So for example, if you want every single class in a specific module to never be allowed to use a certain attribute or to follow a specific pattern you could set metaclasses for those specific modules.

Conclusion

The purpose of metaclasses isn't to replace the class/object distinction with metaclass/class - it's to change the behavior of class definitions (and thus their instances) in some way.
A reasonable pattern of metaclass use is doing something once when a class is defined rather than repeatedly whenever the same class is instantiated. When multiple classes share the same special behavior, repeating metaclass=X is obviously better than repeating the special purpose code and/or introducing ad-hoc shared superclasses.

Reference

https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python

https://stackoverflow.com/questions/392160/what-are-some-concrete-use-cases-for-metaclasses/393368

Top comments (6)

Collapse
 
waylonwalker profile image
Waylon Walker •

The way that you modified all of the attributes of the Sample was really great. It was simple enough to see what is going on and feel like I can grasp it just a bit better.

This might be a full article in itself, but can you explain the difference between MetaClasses and AbstractBaseClasses?

Collapse
 
sharmapacific profile image
Prashant Sharma •

@Waylon, of course, it could be a full article in itself, but for now, in short -
abstract base classes cannot be instantiated. But we can pass as parent class whose child class should have all the abstract methods defined in the abstract class.

The metaClass is a class whose instances are classes.

For more, you can go through here- stackoverflow.com/questions/357079...

Collapse
 
stealthmusic profile image
Jan Wedel •

It goes even further.

A long time ago, I wrote a Python interpreter in Java based on some existing interpreter written in C.

And I was amazed how everything was internally an object, strings, numbers, classes but even modules and code itself!

In some interviews where candidates have knowledge in both languages, I ask the vicious question „which one is more object-oriented, Java or Python?“ 😈

Collapse
 
mahmoudajawad profile image
Mahmoud Abduljawad •

One of the best aspects of Python. Metaprogramming is just an easy access having "everything is object" as standard in the language.

Great article 👍

Collapse
 
rhymes profile image
rhymes •

Nice overview Prashant!

Collapse
 
sharmapacific profile image
Prashant Sharma •

Thank You Rhymes :)

Read next

luqmanbello profile image

Building Production-Grade ECS Anywhere Infrastructure with Custom Capacity Providers

Luqman Bello -

silvio_riveros_49b5505866 profile image

Migrar de SQLite a MySQL.

Silvio Riveros -

dchaley profile image

Performance trap: general libraries & helper objects

David Haley -

hakeem profile image

Concurrency and Parallelism in Python

Hakeem Abbas -