Python: __init__() is not the only constructor

Many beginners and even Pythonic Programmers think that __init__() is a constructor as it initializes values when it is called, but __init__() is a special method which initializes objects and it is immediately called when an object is created and initialized and does not allocate memory. So it is not the only method for the constructor.

class Foo:
  def __init__(self, value_a, value_b):
    self.a = value_a
    self.b = value_b

foo = Foo(7, 9) # __init__ is called
print(foo.a , foo.b) # 7 , 9

__new__() method is called before __init__() acts. It is because__new__() allocates memory before __init__() is called.

def __new__(cls, *args, **kwargs)

Things which are need to be kept in mind about __new__():

• __new__() is always called before __init__().
• First argument is the class itself which is passed implicitly.
• Always return a valid object from new(). Not mandatory, but that's the whole point.
• __new__() controls object creation. That means, in an extreme case, you can return a completely different type of object at the end. I will give an example soon.

class Point():
    def __new__(cls,*args,**kwargs):
        print("From new")
        print(cls)
        print(args)
        print(kwargs)
        # create our object and return it
        obj = super().__new__(cls)
        return obj
    def __init__(self, x = 0, y = 0):
        print("From init")
        self.x = x
        self.y = y

Output:

>>> p2 = Point(3,4)
From new
<class '__main__.Point'>
(3, 4)
{}
From init

We see that __new__() is called before __init__() when an object is initialized and can also see that the parameter cls in __new__() is the class itself (Point). Finally, the object is created by calling the __new__() method on object base class. In Python, object is the base class from which all other classes are derived. In the above example, we have done this using super().

When to use __init__() and __new__()?

You would have seen __init__() being used more frequently than __new__() as we mostly need to initialize objects not how the object is controlled though we can use __new__() to initialize objects but it will make more sense to initialize them in __init__().

Some practical use of __new__() can be to restrict the number of objects created from a class, make a singleton and return a completely different object at the end.

Restrict the number of objects created

Suppose we want to make a class RectPoint for creating instances to represent the four vertices of a square. We can inherit from our previous class Point (first example in this article) and use new() to implement this restriction. Here is an example to restrict a class to have only four instances.

class RectPoint(Point):
    MAX_Inst = 4
    Inst_created = 0
    def __new__(cls,*args,**kwargs):
        if (cls.Inst_created >= cls.MAX_Inst):
            raise ValueError("Cannot create more objects")
        cls.Inst_created += 1
        return super().__new__(cls)

A sample run.

>>> p1 = RectPoint(0,0)
>>> p2 = RectPoint(1,0)
>>> p3 = RectPoint(1,1)
>>> p4 = RectPoint(0,1)
>>> 
>>> p5 = RectPoint(2,2)
Traceback (most recent call last):
...
ValueError: Cannot create more objects

A Simple Singleton

In this you have to be careful in __init__(), because it is called every time you “instantiate” the object (ie. you do a = Example()), even though they return the same object.

_singleton = None

class Example:
    def __new__(cls):
        global _singleton

        if _singleton is None:
            _singleton = super(Example, cls).__new__(cls)

        return _singleton

a = Example()
b = Example()
a is b
# output: True

Returning completely a new Object

In this example int.__init__() will be called, not Example.__init__(), as the returned object is not of type Example but int.

class Example:
    def __new__(cls):
        return 3

type(Example())
# output: int

Thus, both __new__() and __init__() work and function as a constructor from other OOP language like C++ as it allocates and initializes objects. But as __init__() is mostly used a stated above. You can call __init__() a "constructor" as we don't care about allocation. Python is a very abstracted language and we don't need to care about memory much.

I hope my post clarifies this well and if there's any problem then please don't forget to comment.

Comments

[Gergely Polonkai]

In all honesty, this post confused me a lot. And my $job is Python for about 6 years now.

After reading it several times, i finally see your point, but the whole thing should be edited, if not rewritten, to make it easier to understand, especially for beginners.

The most important thing is that __new__ controls object creation. That means, in an extreme case, you can return a completely different type of object at the end:

class Example:
    def __new__(cls):
        return 3

type(Example())
# output: int

Also, it can be used to create a singleton:

_singleton = None

class Example:
    def __new__(cls):
        global _singleton

        if _singleton is None:
            _singleton = super(Example, cls).__new__(cls)

        return _singleton

a = Example()
b = Example()
a is b
# output: True

After the object is created, the object’s __init__ method is called. This one is especially tricky; for example, in my first example, int.__init__ will be called, not Example.__init__, as the returned object is not of type Example but int. In the second example you have to be careful in __init__, because it is called every time you “instantiate” the object (ie. you do a = Example()), even though they return the same object.

[Swastik Baranwal]

Yeah I have missed that and I will probably re-write this post again. Thanks for enlightening that for me!

Also can I use your code which you have given to explain? Would be awesome if you aloowed.

[Gergely Polonkai]

Sure, go ahead and use it! I’m glad i could help.

[Swastik Baranwal]

I just don't want to mislead people especially beginners.

[Mohi]

Amazing article. Thanks for highlighting the big difference between new() and init() methods. Even great Python 3 books does not mention that!
Awaiting more posts from you if possible.

[Swastik Baranwal]

This post also came in a tweet of Python LibHunt.

Thanks to everyone who read and found the possible mistakes.

[Swastik Baranwal]

Opps thanks!

[robin]

Hi,
I'm relatively new to python and was using it more as scripting language (without bothering to create classes) since my projects are rather small.
But I was wondering what the clse in this line
if (cls.Inst_created >= clse.MAX_Inst)
Meant, I googled that, but it seems, that it's most likely a typo, but your program did work, so
.. ?

[Swastik Baranwal]

It's like self reference that points to current instance of the class. I used cls because it needs to be like that.

[robin]

No, I explicitly meant the cls*e*

[Swastik Baranwal]

That seems to be a typo. I have updated the post so please check again.

[robin]

Okay, thanks a lot
Really nice and easy to understand article then :)

[ariefatkhur]

This raises a new question for me. When I daclare a variable (or name) in class scope, say, MAX_Inst = 4, usually I would call this from an instance of that class (self.MAX_Inst). This example shows that it is also a class/static variable, hence could be called from cls context. My question is, does python allocate 2 types of variable, one for class one for instance?

Thank you for bringing up new. new creates a new instance of object (thus returning an obj is the point), and init initialize the created instance (since initialization could only be done to something that exists). That's what I get from the naming and this article. It's not named const (like in php) for some reason. Python split the construction process in 2 steps, before and after it's created.

[Nested Software]

I think Python will look up variables in __class__ if it does not find them associated with the instance. So there should only be one variable bound to the class.

[Ben Sinclair]

I've learnt a feair bit from reading the discussion in the comments here.

[Nika]

Same here :)

[Brian]

I found this article enlightining.
I understand the argument to define new() as a constructor but not used in everyday life.
I started my programming life with Java, where the method with the same name as the class will create the object instance and you use the same method to initial them.
Coming into python, my understanding was init() was doing the same thing(create and initialise).
But now I know better. Don't know when I will use new(), but
the "ahaa" moments (@ 2am) come from small details like this...

[Rob Fletcher]

In a statically typed language, the default state of an object is defined by its object definition. When we instantiate that object memory is allocated into this default state, a constructor is then run to customise the instance.

Python mimics this pattern in that __new__ produces a default instance of the class and __init__ customises it.

__init__ is fulfilling the constructor part of the pattern.

If you start customising the object in new you risk having those changes overwritten by init as it is always executed on the instance returned by new.

From the python docs

from new

new() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.

from init

Because new() and init() work together in constructing objects (new() to create it, and init() to customize it), no non-None value may be returned by init(); doing so will cause a TypeError to be raised at runtime.

[Carl Okunubi]

Very insightful

[solstice333]

The official python doc (docs.python.org/3/reference/datamo...) makes it clear that __new__ is meant to be used as a static factory method that returns an instance of the target type. Then __init__ does some further editing to the instance from there. Ok, whatever.... programming is hard enough. No need to be pedantic about all of this. IMO it's almost always better to just lie to yourself and say __init__ is the constructor and you'll be just fine whether you're a beginner or not