Python Special/Magic Methods

· Part 1: Introduction
· Part 2: Construction and Initialization
· Part 3: Comparison
· Part 4: Numeric Magic Method
· Part 5: Type Conversion
· Part 6: Class Representation
· Part 7: Attribute Controlling
· Part 8: Container
· Part 9: Callable
· Part 10: Context Managers
· Part 11: Descriptor Object
· Part 12: Copying
· Part 13: Pickling
· Reference

Part 1: Introduction

  • The biggest advantages of using Python’s magic method is that they provide a simple way to make objects behave like built-in types.
  • Python’s magic methods are incredibly powerful, and with great power comes great responsibility. It’s important to know the proper way to use magic methods so you don’t break any code.
  • Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they’re more like guidelines.

Part 2: Construction and Initialization

  • _new_ is the first method to get called in an object’s instantiation. However, it is rarely used. Like __del__, which is also rarely used, is mainly used for garbage collection.

Part 3: Comparison

  • __cmp__ is the most basic comparison magic method. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.). It’s usually best to define each comparison you need rather than define them all at once, but __cmp__ can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria.
  • The other comparison methods include __eq__, __ne__, __lt__, __gt__, __le__, __ge__.

Part 4: Numeric Magic Method

  • Unary operators include the following:
  • arithmetic operators
  • reflected arithmetic operators : So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around.
  • augmented assignment: Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for a += b, __iadd__ might return a + b, which would be assigned to a

Part 5: Type Conversion

Python also has an array of magic methods designed to implement behavior for built in type conversion functions like float()

Part 6: Class Representation

  • The major difference between __str__ and __repr__ is intended audience. repr() is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereas str() is intended to be human-readable.

The output is:

Two special methods, __repr__ and __str__, are essential for creating proper string representations of your custom class, which will give the code readers more intuitive information about your classes. Between them, the major difference is that the __repr__ method defines the string, using which you can re-create the object by calling eval(repr(“the repr”)), while the __str__ method defines the string that is more descriptive and allows more customization. In other words, you can think that the string defined in the __repr__ method is to be viewed by developers while that used in the __str__ method is to be viewed by regular users. The following shows you an example.

  • __unicode__(self)Defines behavior for when unicode() is called on an instance of your class. unicode() is like str(), but it returns a unicode string. Be wary: if a client calls str() on an instance of your class and you've only defined __unicode__(), it won't work. You should always try to define __str__() as well in case someone doesn't have the luxury of using unicode.
  • __format__(self, formatstr)Defines behavior for when an instance of your class is used in new-style string formatting. For instance, "Hello, {0:abc}!".format(a) would lead to the call a.__format__("abc"). This can be useful for defining your own numerical or string types that you might like to give special formatting options.
  • __hash__(self)Defines behavior for when hash() is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing __eq__ as well. Live by the following rule: a == b implies hash(a) == hash(b).
  • __nonzero__(self)Defines behavior for when bool() is called on an instance of your class. Should return True or False, depending on whether you would want to consider the instance to be True or False.

Part 7: Attribute Controlling

  • __getattr__(self, name)You can define behavior for when a user attempts to access an attribute that doesn't exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing an AttributeError. It only gets called when a nonexistent attribute is accessed, however, so it isn't a true encapsulation solution.
  • __setattr__(self, name, value)Unlike __getattr__, __setattr__ is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes.
  • __delattr__(self, name)This is the exact same as __setattr__, but for deleting attributes instead of setting them. The same precautions need to be taken as with __setattr__ as well in order to prevent infinite recursion.
  • __getattribute__(self, name)After all this, __getattribute__ fits in pretty well with its companions __setattr__ and __delattr__. However, I don't recommend you use it. __getattribute__ can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassing object. It allows you to define rules for whenever an attribute's value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class's __getattribute__ method to prevent this). It also mainly obviates the need for __getattr__, which, when __getattribute__ is implemented, only gets called if it is called explicitly or an AttributeError is raised. This method can be used (after all, it's your choice), but I don't recommend it because it has a small use case (it's far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free.

Part 8: Container

  • __len__(self)Returns the length of the container. Part of the protocol for both immutable and mutable containers.
  • __getitem__(self, key)Defines behavior for when an item is accessed, using the notation self[key]. This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions: TypeError if the type of the key is wrong and KeyError if there is no corresponding value for the key.
  • __setitem__(self, key, value)Defines behavior for when an item is assigned to, using the notation self[nkey] = value. This is part of the mutable container protocol. Again, you should raise KeyError and TypeError where appropriate.
  • __delitem__(self, key)Defines behavior for when an item is deleted (e.g. del self[key]). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used.
  • __iter__(self)Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by the iter() built in function and when a container is looped over using the form for x in container:. Iterators are their own objects, and they also must define an __iter__ method that returns self.
  • __reversed__(self)Called to implement behavior for the reversed() built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple.
  • __contains__(self, item)__contains__ defines behavior for membership tests using in and not in. Why isn't this part of a sequence protocol, you ask? Because when __contains__ isn't defined, Python just iterates over the sequence and returns True if it comes across the item it's looking for.__missing__(self, key)
  • __missing__ is used in subclasses of dict. It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionary d and said d["george"] when "george" is not a key in the dict, d.__missing__("george") would be called).

Example 1 for __setitem__ and __getitem__:

Example 2 for __iter__

__iter__ is tried first before falling back to the __getitem__ approach.

Part 9: Callable

__call__(self, [args...])Allows an instance of a class to be called as a function. Essentially, this means that x() is the same as x.__call__(). Note that __call__ takes a variable number of arguments; this means that you define __call__ as you would any other function, taking however many arguments you'd like it to.

__call__ can be particularly useful in classes with instances that need to often change state. "Calling" the instance can be an intuitive and elegant way to change the object's state. An example might be a class representing an entity's position on a plane:

Another example is to use __call__ to invoke a function:

Part 10: Context Managers

Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a with statement. The behavior of the context manager is determined by two magic methods:

__enter__(self)Defines what the context manager should do at the beginning of the block created by the with statement. Note that the return value of __enter__ is bound to the target of the with statement, or the name after the as.__exit__(self, exception_type, exception_value, traceback)Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, exception_type, exception_value, and traceback will be None. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure __exit__ returns True after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen.

__enter__ and __exit__ can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects.

Part 11: Descriptor Object

To be a descriptor, a class must have at least one of __get__, __set__, and __delete__ implemented. Let's take a look at those magic methods:

__get__(self, instance, owner)Define behavior for when the descriptor's value is retrieved. instance is the instance of the owner object. owner is the owner class itself.

__set__(self, instance, value)Define behavior for when the descriptor's value is changed. instance is the instance of the owner class and value is the value to set the descriptor to.

__delete__(self, instance)Define behavior for when the descriptor's value is deleted. instance is the instance of the owner object.

Part 12: Copying

Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python’s copy comes into play. However (fortunately), Python modules are not sentient, so we don't have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things.

__copy__(self)Defines behavior for copy.copy() for instances of your class. copy.copy() returns a shallow copy of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original).

__deepcopy__(self, memodict={})Defines behavior for copy.deepcopy() for instances of your class. copy.deepcopy() returns a deep copy of your object -- the object and its data are both copied. memodict is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, call copy.deepcopy() on that attribute with memodict as the first argument.

Part 13: Pickling

Pickling isn’t just for built-in types. It’s for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it’s a bit different for C extensions, but that’s not in our scope)

Part 14: Module

__file__ get the module’s installation directory