Python magic method __new__

By: Varghese Chacko 3 years, 3 months ago

Many developers believe that __init__ is the first method invoked in the life-cycle of an object. When we create an object or instantiate a class, the first method that gets called __new__, which actually creates an objects and then passed the arguments to to __init__, the initializer. __new__ gets invoked before __init__.

The first argument to __new__ is the class and followed by other arguments to be passed to __init__. __new__ is very rarely called, but is used when sub classing immutable types like tuple. The responsibility of __new__ is to return the instance of the class. There are cases where instances are created without calling __init__ for example, loading from pickle where as there is no way to create an instance without calling __new__. Don't get me wrong, we can skip calling __new__ of base class in some cases, but that is not in the scope of this article.

Let us say, we want a type to convert meter to inch. In this case, we can subclass float and override the __new__ method as below.

class meter(float):
    "Convert from inch to meter"
    def __new__(cls, arg=0.0):
        return float.__new__(cls, arg*0.0254)

Though this is not the right method for unit conversion, let us use it.

>>> print inch(12)

There are some points regarding __new__

  1. __new__ is a static method. When defining it, you don't need to use the phrase "__new__ = staticmethod(__new__)"
  2. The first argument to __new__ must be a class; the remaining arguments are the arguments as seen by the constructor call.
  3. A __new__ method that overrides a base class's __new__ method may call that base class's __new__ method. The first argument to the base class's __new__ method call should be the class argument to the overriding __new__ method, not the base class; if you were to pass in the base class, you would get an instance of the base class.
  4. A __new__ method must call its base class's __new__ method; that's the only way to create an instance of your object. The subclass __new__ can do two things to affect the resulting object: pass different arguments to the base class __new__, and modify the resulting object after it's been created
  5. __new__ must return an object. There's nothing that requires that it return a new object that is an instance of its class argument, although that is the convention. If you return an existing object, the constructor call will still call its __init__ method. If you return an object of a different class, its __init__ method will be called. If you forget to return something, Python will unhelpfully return None, and your caller will probably be very confused.
  6. For immutable classes, your __new__ may return a cached reference to an existing object with the same value; this is what the int, str and tuple types do for small values. This is one of the reasons why their __init__ does nothing: cached objects would be re-initialized over and over. (The other reason is that there's nothing left for __init__ to initialize: __new__ returns a fully initialized object.)
  7. If you subclass a built-in immutable type and want to add some mutable state (maybe you add a default conversion to a string type), it's best to initialize the mutable state in the __init__ method and leave __new__ alone.
  8. If you want to change the constructor's signature, you often have to override both __new__ and __init__ to accept the new signature. However, most built-in types ignore the arguments to the method they don't use; in particular, the immutable types (int, long, float, complex, str, unicode, and tuple) have a dummy __init__, while the mutable types (dict, list, file, and also super, classmethod, staticmethod, and property) have a dummy __new__. The built-in type 'object' has a dummy __new__ and a dummy __init__ (which the others inherit). The built-in type 'type' is special in many respects; see the section on metaclasses.
  9. (This has nothing to do to __new__, but is handy to know anyway.) If you subclass a built-in type, extra space is automatically added to the instances to accomodate __dict__ and __weakrefs__. (The __dict__ is not initialized until you use it though, so you shouldn't worry about the space occupied by an empty dictionary for each instance you create.) If you don't need this extra space, you can add the phrase "__slots__ = []" to your class. (See above for more about __slots__.)
  10. Factoid: __new__ is a static method, not a class method. I initially thought it would have to be a class method, and that's why I added the classmethod primitive. Unfortunately, with class methods, upcalls don't work right in this case, so I had to make it a static method with an explicit class as its first argument. Ironically, there are now no known uses for class methods in the Python distribution (other than in the test suite). I might even get rid of classmethod in a future release if no good use for it can be found!

References: https://www.python.org/download/releases/2.2/descrintro/#__new__

Let us talk!

We take the vision which comes from dreams and apply the magic of science and mathematics, adding the heritage of our profession and our knowledge to create a design.