Python: __init__() is not the only constructor

Many beginners and even Pythonic Programmers think that __init__() is a constructor as it initializes values when it is called, but __init__() is a special method which initializes objects and it is immediately called when an object is created and initialized and does not allocate memory. So it is not the only method for the constructor.

class Foo:
  def __init__(self, value_a, value_b):
    self.a = value_a
    self.b = value_b

foo = Foo(7, 9) # __init__ is called
print(foo.a , foo.b) # 7 , 9

__new__() method is called before __init__() acts. It is because__new__() allocates memory before __init__() is called.

def __new__(cls, *args, **kwargs)

Things which are need to be kept in mind about __new__():

• __new__() is always called before __init__().
• First argument is the class itself which is passed implicitly.
• Always return a valid object from new(). Not mandatory, but that's the whole point.
• __new__() controls object creation. That means, in an extreme case, you can return a completely different type of object at the end. I will give an example soon.

class Point():
    def __new__(cls,*args,**kwargs):
        print("From new")
        print(cls)
        print(args)
        print(kwargs)
        # create our object and return it
        obj = super().__new__(cls)
        return obj
    def __init__(self, x = 0, y = 0):
        print("From init")
        self.x = x
        self.y = y

Output:

>>> p2 = Point(3,4)
From new
<class '__main__.Point'>
(3, 4)
{}
From init

We see that __new__() is called before __init__() when an object is initialized and can also see that the parameter cls in __new__() is the class itself (Point). Finally, the object is created by calling the __new__() method on object base class. In Python, object is the base class from which all other classes are derived. In the above example, we have done this using super().

When to use __init__() and __new__()?

You would have seen __init__() being used more frequently than __new__() as we mostly need to initialize objects not how the object is controlled though we can use __new__() to initialize objects but it will make more sense to initialize them in __init__().

Some practical use of __new__() can be to restrict the number of objects created from a class, make a singleton and return a completely different object at the end.

Restrict the number of objects created

Suppose we want to make a class RectPoint for creating instances to represent the four vertices of a square. We can inherit from our previous class Point (first example in this article) and use new() to implement this restriction. Here is an example to restrict a class to have only four instances.

class RectPoint(Point):
    MAX_Inst = 4
    Inst_created = 0
    def __new__(cls,*args,**kwargs):
        if (cls.Inst_created >= cls.MAX_Inst):
            raise ValueError("Cannot create more objects")
        cls.Inst_created += 1
        return super().__new__(cls)

A sample run.

>>> p1 = RectPoint(0,0)
>>> p2 = RectPoint(1,0)
>>> p3 = RectPoint(1,1)
>>> p4 = RectPoint(0,1)
>>> 
>>> p5 = RectPoint(2,2)
Traceback (most recent call last):
...
ValueError: Cannot create more objects

A Simple Singleton

In this you have to be careful in __init__(), because it is called every time you “instantiate” the object (ie. you do a = Example()), even though they return the same object.

_singleton = None

class Example:
    def __new__(cls):
        global _singleton

        if _singleton is None:
            _singleton = super(Example, cls).__new__(cls)

        return _singleton

a = Example()
b = Example()
a is b
# output: True

Returning completely a new Object

In this example int.__init__() will be called, not Example.__init__(), as the returned object is not of type Example but int.

class Example:
    def __new__(cls):
        return 3

type(Example())
# output: int

Thus, both __new__() and __init__() work and function as a constructor from other OOP language like C++ as it allocates and initializes objects. But as __init__() is mostly used a stated above. You can call __init__() a "constructor" as we don't care about allocation. Python is a very abstracted language and we don't need to care about memory much.

I hope my post clarifies this well and if there's any problem then please don't forget to comment.

Comments

[eranelbaz]

So I still didn't understand, if __new__ is the actual constructor, why we see __init__ as a constructor in almost any Python code?

[Nested Software]

Technically the __new__ method takes a class and generates an instance of that class. __init__ is used to then initialize the properties of the instance that was created by __new__. So __new__ is the constructor: Its job is to build a new instance object using the class as a template. I think it is very rare to override __new__ because its default behaviour, which is to create an empty instance of the particular class, is usually what we want.

In everyday Python programming, we just override the __init__ method to assign values to the object that was already created earlier by the default __new__ behaviour. __init__ is not the constructor, since it receives a pre-created instance object as a parameter - usually we call this parameter self. Instead, it's an initializer - it initializes an instance object.

As an example, and there are probably more ways to do this, you could override __new__ to limit the number of instances of an object that can be created, or to obtain them from a pool, that sort of idea. So there are probably use-cases for overriding __new__ but I doubt that you'd need to worry about it for everyday usage.

Update: I looked into the nomenclature, and it does not seem to me that Python has the concept of "constructor" used in the documentation in the same way that it's used for languages like Java or C++. Instead the docs mention that the call used to create an object, like say Foo(), is a constructor expression:

docs.python.org/3/reference/datamo...

This constructor expression calls __new__ and then __init__ internally. I guess neither of the latter would officially be called a constructor. I think it's okay to call __init__ the constructor informally though, since it does pretty much the same kind of thing that constructors in languages like Java, C#, C++ do.

[Jorge Cañizales Díaz]

__new__ is what Objective-C and C++ would call an allocator: Its responsibility is to decide what "chunk of memory" to use for a new object.

Calling super().__new__ is the default behavior of allocating new memory in the heap and using that.

But for performance-critical code, you could implement a memory pool (or "arena") by selecting objects out of a contiguous array.

Or, if your class should have value semantics, you can cache new objects by value and have __new__ return already-existing objects. This technique is useful in other contexts too: Java does it automatically for small numerical objects and small strings (which are immutable, so caching them this way improves runtime and memory). Another context you might want to do it is for an object that represents a remote resource: Having only one local object per remote instance simplifies its code considerably when it comes to locking and caching remote state.

Should you call __new__ a constructor and say __init__ isn't a constructor? It doesn't matter. Although I think it's confusing for people familiar with other languages to say __init__ isn't the constructor, and kinda moot if one doesn't offer a better description for it.

[Veselin Karaganev]

self is normally used when referring to an instance of a class, not the class itself. Also, in your example usage for __new__ you have used cls instead of self, which is not present. You might want to correct that.

[Gabriel Amram]

__new__ should use cls and not self as it is a "static" method that belongs to the class, not the instance.
See docs.python.org/3/reference/datamo...

[Lea Maya]

It's actually a classmethod, not a static method but cls is still the recommended name

[Tim Conroy]

Absolutely correct @gabriel
per the Python documentation:
" The new(cls[,...]) method is called first to create a new instance of class, cls. It is a static
method that takes the class of which an instance was requested as its first argument.
The return value of new( ) should be the new object instance(usually an instance of class(cls)). "
Semantics of interpretation of the programming language: It is a class 'static' method. :)>

[Swastik Baranwal]

Yes, its right and I had changed it a long time back!

[Lee-Lilly-FIR]

Depends on how you use "cls", when you use "cls".
"init(self)" : instantiate a class.
"new(cls)": could be equal to a decorative method, that is allowed to be used/called before initiating a class self.

[Han You]

By this logic many languages would have no "constructors" even though people have used this term for decades. C++/Java, for example. This article could use a better and less clickbaity title.

[Swastik Baranwal]

Can you define constructor

[Han You]

Exactly. Coming from C++ I'd say the function that initializes new objects is the constructor (this term is also used by the ISO specification). What this article is saying is that the Python equivalence of allocator in C++, or the implicit object factory in Java, is the "true" constructor. I mean yeah perhaps it suits the English definition of the verb "construct" better, but people have used the term "constructor" to call the "initializer" function for decades. Also both programming and computer science are full of inappropriate jargons, so why wasting time saying this yet another jargon is imprecise?

[Rob Fletcher]

yep. It worries me that this article is going to mislead and confuse people learning the language.

[Swastik Baranwal]

After reading all the arguments. I have edited the post a bit so that it does not looks misleading. If there's anything still unclear then please tell.

[Rob Fletcher]

what do you define as a "true" constructor?

take a look at C++ as an example. once you are in a constructor the classes memory has already been allocated, even uninitialised members have memory assigned to them. The constructor is customising the instance.

A c++ class with a zero constructor is like a python class with no init. you get a default instance.

[Swastik Baranwal]

I changed the title again. Does it looks fine now?

[Bradley Grant]

I thought you changed the title, it definitely wasn't what I remembered seeing.

I think it would be useful to understand exactly what internal functions are called every time a new instance is created, for instance __new__ and __init__, are there any others?

[Swastik Baranwal]

There is also __del__ which is called but it is not needed to make because it de-allocates the object. I think it functions like a destructor.

[Nurettin Onur TUĞCU]

new is like a static factory method, not a constructor.

[solstice333]

The official python doc (docs.python.org/3/reference/datamo...) makes it clear that __new__ is meant to be used as a static factory method that returns an instance of the target type. Then __init__ does some further editing to the instance from there. Ok, whatever.... programming is hard enough. No need to be pedantic about all of this. IMO it's almost always better to just lie to yourself and say __init__ is the constructor and you'll be just fine whether you're a beginner or not

[Rob Fletcher]

In a statically typed language, the default state of an object is defined by its object definition. When we instantiate that object memory is allocated into this default state, a constructor is then run to customise the instance.

Python mimics this pattern in that __new__ produces a default instance of the class and __init__ customises it.

__init__ is fulfilling the constructor part of the pattern.

If you start customising the object in new you risk having those changes overwritten by init as it is always executed on the instance returned by new.

From the python docs

from new

new() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.

from init

Because new() and init() work together in constructing objects (new() to create it, and init() to customize it), no non-None value may be returned by init(); doing so will cause a TypeError to be raised at runtime.

[Carl Okunubi]

Very insightful

[Brian]

I found this article enlightining.
I understand the argument to define new() as a constructor but not used in everyday life.
I started my programming life with Java, where the method with the same name as the class will create the object instance and you use the same method to initial them.
Coming into python, my understanding was init() was doing the same thing(create and initialise).
But now I know better. Don't know when I will use new(), but
the "ahaa" moments (@ 2am) come from small details like this...

[Ben Sinclair]

I've learnt a feair bit from reading the discussion in the comments here.

[Nika]

Same here :)

[ariefatkhur]

This raises a new question for me. When I daclare a variable (or name) in class scope, say, MAX_Inst = 4, usually I would call this from an instance of that class (self.MAX_Inst). This example shows that it is also a class/static variable, hence could be called from cls context. My question is, does python allocate 2 types of variable, one for class one for instance?

Thank you for bringing up new. new creates a new instance of object (thus returning an obj is the point), and init initialize the created instance (since initialization could only be done to something that exists). That's what I get from the naming and this article. It's not named const (like in php) for some reason. Python split the construction process in 2 steps, before and after it's created.

[Nested Software]

I think Python will look up variables in __class__ if it does not find them associated with the instance. So there should only be one variable bound to the class.