Python: __init__() is not the only constructor

Many beginners and even Pythonic Programmers think that __init__() is a constructor as it initializes values when it is called, but __init__() is a special method which initializes objects and it is immediately called when an object is created and initialized and does not allocate memory. So it is not the only method for the constructor.

class Foo:
  def __init__(self, value_a, value_b):
    self.a = value_a
    self.b = value_b

foo = Foo(7, 9) # __init__ is called
print(foo.a , foo.b) # 7 , 9

__new__() method is called before __init__() acts. It is because__new__() allocates memory before __init__() is called.

def __new__(cls, *args, **kwargs)

Things which are need to be kept in mind about __new__():

• __new__() is always called before __init__().
• First argument is the class itself which is passed implicitly.
• Always return a valid object from new(). Not mandatory, but that's the whole point.
• __new__() controls object creation. That means, in an extreme case, you can return a completely different type of object at the end. I will give an example soon.

class Point():
    def __new__(cls,*args,**kwargs):
        print("From new")
        print(cls)
        print(args)
        print(kwargs)
        # create our object and return it
        obj = super().__new__(cls)
        return obj
    def __init__(self, x = 0, y = 0):
        print("From init")
        self.x = x
        self.y = y

Output:

>>> p2 = Point(3,4)
From new
<class '__main__.Point'>
(3, 4)
{}
From init

We see that __new__() is called before __init__() when an object is initialized and can also see that the parameter cls in __new__() is the class itself (Point). Finally, the object is created by calling the __new__() method on object base class. In Python, object is the base class from which all other classes are derived. In the above example, we have done this using super().

When to use __init__() and __new__()?

You would have seen __init__() being used more frequently than __new__() as we mostly need to initialize objects not how the object is controlled though we can use __new__() to initialize objects but it will make more sense to initialize them in __init__().

Some practical use of __new__() can be to restrict the number of objects created from a class, make a singleton and return a completely different object at the end.

Restrict the number of objects created

Suppose we want to make a class RectPoint for creating instances to represent the four vertices of a square. We can inherit from our previous class Point (first example in this article) and use new() to implement this restriction. Here is an example to restrict a class to have only four instances.

class RectPoint(Point):
    MAX_Inst = 4
    Inst_created = 0
    def __new__(cls,*args,**kwargs):
        if (cls.Inst_created >= cls.MAX_Inst):
            raise ValueError("Cannot create more objects")
        cls.Inst_created += 1
        return super().__new__(cls)

A sample run.

>>> p1 = RectPoint(0,0)
>>> p2 = RectPoint(1,0)
>>> p3 = RectPoint(1,1)
>>> p4 = RectPoint(0,1)
>>> 
>>> p5 = RectPoint(2,2)
Traceback (most recent call last):
...
ValueError: Cannot create more objects

A Simple Singleton

In this you have to be careful in __init__(), because it is called every time you “instantiate” the object (ie. you do a = Example()), even though they return the same object.

_singleton = None

class Example:
    def __new__(cls):
        global _singleton

        if _singleton is None:
            _singleton = super(Example, cls).__new__(cls)

        return _singleton

a = Example()
b = Example()
a is b
# output: True

Returning completely a new Object

In this example int.__init__() will be called, not Example.__init__(), as the returned object is not of type Example but int.

class Example:
    def __new__(cls):
        return 3

type(Example())
# output: int

Thus, both __new__() and __init__() work and function as a constructor from other OOP language like C++ as it allocates and initializes objects. But as __init__() is mostly used a stated above. You can call __init__() a "constructor" as we don't care about allocation. Python is a very abstracted language and we don't need to care about memory much.

I hope my post clarifies this well and if there's any problem then please don't forget to comment.

Comments

[Veselin Karaganev]

self is normally used when referring to an instance of a class, not the class itself. Also, in your example usage for __new__ you have used cls instead of self, which is not present. You might want to correct that.

[Gabriel Amram]

__new__ should use cls and not self as it is a "static" method that belongs to the class, not the instance.
See docs.python.org/3/reference/datamo...

[Tim Conroy]

Absolutely correct @gabriel
per the Python documentation:
" The new(cls[,...]) method is called first to create a new instance of class, cls. It is a static
method that takes the class of which an instance was requested as its first argument.
The return value of new( ) should be the new object instance(usually an instance of class(cls)). "
Semantics of interpretation of the programming language: It is a class 'static' method. :)>

[Swastik Baranwal]

Yes, its right and I had changed it a long time back!

[Lea Maya]

It's actually a classmethod, not a static method but cls is still the recommended name

[Han You]

By this logic many languages would have no "constructors" even though people have used this term for decades. C++/Java, for example. This article could use a better and less clickbaity title.

[Swastik Baranwal]

Can you define constructor

[Han You]

Exactly. Coming from C++ I'd say the function that initializes new objects is the constructor (this term is also used by the ISO specification). What this article is saying is that the Python equivalence of allocator in C++, or the implicit object factory in Java, is the "true" constructor. I mean yeah perhaps it suits the English definition of the verb "construct" better, but people have used the term "constructor" to call the "initializer" function for decades. Also both programming and computer science are full of inappropriate jargons, so why wasting time saying this yet another jargon is imprecise?

[Rob Fletcher]

yep. It worries me that this article is going to mislead and confuse people learning the language.

[Swastik Baranwal]

After reading all the arguments. I have edited the post a bit so that it does not looks misleading. If there's anything still unclear then please tell.

[Rob Fletcher]

what do you define as a "true" constructor?

take a look at C++ as an example. once you are in a constructor the classes memory has already been allocated, even uninitialised members have memory assigned to them. The constructor is customising the instance.

A c++ class with a zero constructor is like a python class with no init. you get a default instance.

[Swastik Baranwal]

I changed the title again. Does it looks fine now?

[Bradley Grant]

I thought you changed the title, it definitely wasn't what I remembered seeing.

I think it would be useful to understand exactly what internal functions are called every time a new instance is created, for instance __new__ and __init__, are there any others?

[Swastik Baranwal]

There is also __del__ which is called but it is not needed to make because it de-allocates the object. I think it functions like a destructor.

[Gergely Polonkai]

In all honesty, this post confused me a lot. And my $job is Python for about 6 years now.

After reading it several times, i finally see your point, but the whole thing should be edited, if not rewritten, to make it easier to understand, especially for beginners.

The most important thing is that __new__ controls object creation. That means, in an extreme case, you can return a completely different type of object at the end:

class Example:
    def __new__(cls):
        return 3

type(Example())
# output: int

Also, it can be used to create a singleton:

_singleton = None

class Example:
    def __new__(cls):
        global _singleton

        if _singleton is None:
            _singleton = super(Example, cls).__new__(cls)

        return _singleton

a = Example()
b = Example()
a is b
# output: True

After the object is created, the object’s __init__ method is called. This one is especially tricky; for example, in my first example, int.__init__ will be called, not Example.__init__, as the returned object is not of type Example but int. In the second example you have to be careful in __init__, because it is called every time you “instantiate” the object (ie. you do a = Example()), even though they return the same object.

[Swastik Baranwal]

Yeah I have missed that and I will probably re-write this post again. Thanks for enlightening that for me!

Also can I use your code which you have given to explain? Would be awesome if you aloowed.

[Gergely Polonkai]

Sure, go ahead and use it! I’m glad i could help.

[Swastik Baranwal]

I just don't want to mislead people especially beginners.

[solstice333]

The official python doc (docs.python.org/3/reference/datamo...) makes it clear that __new__ is meant to be used as a static factory method that returns an instance of the target type. Then __init__ does some further editing to the instance from there. Ok, whatever.... programming is hard enough. No need to be pedantic about all of this. IMO it's almost always better to just lie to yourself and say __init__ is the constructor and you'll be just fine whether you're a beginner or not

[robin]

Hi,
I'm relatively new to python and was using it more as scripting language (without bothering to create classes) since my projects are rather small.
But I was wondering what the clse in this line
if (cls.Inst_created >= clse.MAX_Inst)
Meant, I googled that, but it seems, that it's most likely a typo, but your program did work, so
.. ?

[Swastik Baranwal]

It's like self reference that points to current instance of the class. I used cls because it needs to be like that.

[robin]

No, I explicitly meant the cls*e*

[Swastik Baranwal]

That seems to be a typo. I have updated the post so please check again.

[robin]

Okay, thanks a lot
Really nice and easy to understand article then :)

[Nurettin Onur TUĞCU]

new is like a static factory method, not a constructor.

[Lee-Lilly-FIR]

Depends on how you use "cls", when you use "cls".
"init(self)" : instantiate a class.
"new(cls)": could be equal to a decorative method, that is allowed to be used/called before initiating a class self.

[Rob Fletcher]

In a statically typed language, the default state of an object is defined by its object definition. When we instantiate that object memory is allocated into this default state, a constructor is then run to customise the instance.

Python mimics this pattern in that __new__ produces a default instance of the class and __init__ customises it.

__init__ is fulfilling the constructor part of the pattern.

If you start customising the object in new you risk having those changes overwritten by init as it is always executed on the instance returned by new.

From the python docs

from new

new() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.

from init

Because new() and init() work together in constructing objects (new() to create it, and init() to customize it), no non-None value may be returned by init(); doing so will cause a TypeError to be raised at runtime.

[Carl Okunubi]

Very insightful

[eranelbaz]

So I still didn't understand, if __new__ is the actual constructor, why we see __init__ as a constructor in almost any Python code?

[Nested Software]

Technically the __new__ method takes a class and generates an instance of that class. __init__ is used to then initialize the properties of the instance that was created by __new__. So __new__ is the constructor: Its job is to build a new instance object using the class as a template. I think it is very rare to override __new__ because its default behaviour, which is to create an empty instance of the particular class, is usually what we want.

In everyday Python programming, we just override the __init__ method to assign values to the object that was already created earlier by the default __new__ behaviour. __init__ is not the constructor, since it receives a pre-created instance object as a parameter - usually we call this parameter self. Instead, it's an initializer - it initializes an instance object.

As an example, and there are probably more ways to do this, you could override __new__ to limit the number of instances of an object that can be created, or to obtain them from a pool, that sort of idea. So there are probably use-cases for overriding __new__ but I doubt that you'd need to worry about it for everyday usage.

Update: I looked into the nomenclature, and it does not seem to me that Python has the concept of "constructor" used in the documentation in the same way that it's used for languages like Java or C++. Instead the docs mention that the call used to create an object, like say Foo(), is a constructor expression:

docs.python.org/3/reference/datamo...

This constructor expression calls __new__ and then __init__ internally. I guess neither of the latter would officially be called a constructor. I think it's okay to call __init__ the constructor informally though, since it does pretty much the same kind of thing that constructors in languages like Java, C#, C++ do.

[Jorge Cañizales Díaz]

__new__ is what Objective-C and C++ would call an allocator: Its responsibility is to decide what "chunk of memory" to use for a new object.

Calling super().__new__ is the default behavior of allocating new memory in the heap and using that.

But for performance-critical code, you could implement a memory pool (or "arena") by selecting objects out of a contiguous array.

Or, if your class should have value semantics, you can cache new objects by value and have __new__ return already-existing objects. This technique is useful in other contexts too: Java does it automatically for small numerical objects and small strings (which are immutable, so caching them this way improves runtime and memory). Another context you might want to do it is for an object that represents a remote resource: Having only one local object per remote instance simplifies its code considerably when it comes to locking and caching remote state.

Should you call __new__ a constructor and say __init__ isn't a constructor? It doesn't matter. Although I think it's confusing for people familiar with other languages to say __init__ isn't the constructor, and kinda moot if one doesn't offer a better description for it.

[Brian]

I found this article enlightining.
I understand the argument to define new() as a constructor but not used in everyday life.
I started my programming life with Java, where the method with the same name as the class will create the object instance and you use the same method to initial them.
Coming into python, my understanding was init() was doing the same thing(create and initialise).
But now I know better. Don't know when I will use new(), but
the "ahaa" moments (@ 2am) come from small details like this...