· Part 1: Introduction
· Part 2: Construction and Initialization
· Part 3: Comparison
· Part 4: Numeric Magic Method
· Part 5: Type Conversion
· Part 6: Class Representation
· Part 7: Attribute Controlling
· Part 8: Container
· Part 9: Callable
· Part 10: Context Managers
· Part 11: Descriptor Object
· Part 12: Copying
· Part 13: Pickling
· Reference
Part 1: Introduction
- The biggest advantages of using Python’s magic method is that they provide a simple way to make objects behave like built-in types.
- Python’s magic methods are incredibly powerful, and with great power comes great responsibility. It’s important to know the proper way to use magic methods so you don’t break any code.
- Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they’re more like guidelines.
Part 2: Construction and Initialization
_new_is the first method to get called in an object’s instantiation. However, it is rarely used. Like__del__, which is also rarely used, is mainly used for garbage collection.
Part 3: Comparison
__cmp__is the most basic comparison magic method. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.). It’s usually best to define each comparison you need rather than define them all at once, but__cmp__can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria.- The other comparison methods include
__eq__,__ne__,__lt__,__gt__,__le__,__ge__.
Part 4: Numeric Magic Method
- Unary operators include the following:
- arithmetic operators
- reflected arithmetic operators : So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around.
- augmented assignment: Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for
a += b,__iadd__might returna + b, which would be assigned toa
Part 5: Type Conversion
Python also has an array of magic methods designed to implement behavior for built in type conversion functions like float()
Part 6: Class Representation
- The major difference between
__str__and__repr__is intended audience.repr()is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereasstr()is intended to be human-readable.
class Point(object):
def __init__(self, u, v, d):
self.u = u
self.v = v
self.d = d
def __repr__(self):
return f"Point(u={self.u},v={self.v}, d={self.d})"
def __eq__(self, other):
return self.u==other.u and self.v == other.v and self.d == other.d
a = Point(3,4, 5)
print(a)
b=eval('Point(u=3,v=4, d=5)')
print(b)
print(a==b)The output is:
Point(u=3,v=4, d=5)
Point(u=3,v=4, d=5)
TrueTwo special methods, __repr__ and __str__, are essential for creating proper string representations of your custom class, which will give the code readers more intuitive information about your classes. Between them, the major difference is that the __repr__ method defines the string, using which you can re-create the object by calling eval(repr(“the repr”)), while the __str__ method defines the string that is more descriptive and allows more customization. In other words, you can think that the string defined in the __repr__ method is to be viewed by developers while that used in the __str__ method is to be viewed by regular users. The following shows you an example.
__unicode__(self)Defines behavior for whenunicode()is called on an instance of your class.unicode()is likestr(), but it returns a unicode string. Be wary: if a client callsstr()on an instance of your class and you've only defined__unicode__(), it won't work. You should always try to define__str__()as well in case someone doesn't have the luxury of using unicode.__format__(self, formatstr)Defines behavior for when an instance of your class is used in new-style string formatting. For instance,"Hello, {0:abc}!".format(a)would lead to the calla.__format__("abc"). This can be useful for defining your own numerical or string types that you might like to give special formatting options.__hash__(self)Defines behavior for whenhash()is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing__eq__as well. Live by the following rule:a == bimplieshash(a) == hash(b).__nonzero__(self)Defines behavior for whenbool()is called on an instance of your class. Should returnTrueorFalse, depending on whether you would want to consider the instance to beTrueorFalse.
Part 7: Attribute Controlling
__getattr__(self, name)You can define behavior for when a user attempts to access an attribute that doesn't exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing anAttributeError. It only gets called when a nonexistent attribute is accessed, however, so it isn't a true encapsulation solution.__setattr__(self, name, value)Unlike__getattr__,__setattr__is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes.__delattr__(self, name)This is the exact same as__setattr__, but for deleting attributes instead of setting them. The same precautions need to be taken as with__setattr__as well in order to prevent infinite recursion.__getattribute__(self, name)After all this,__getattribute__fits in pretty well with its companions__setattr__and__delattr__. However, I don't recommend you use it.__getattribute__can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassingobject. It allows you to define rules for whenever an attribute's value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class's__getattribute__method to prevent this). It also mainly obviates the need for__getattr__, which, when__getattribute__is implemented, only gets called if it is called explicitly or anAttributeErroris raised. This method can be used (after all, it's your choice), but I don't recommend it because it has a small use case (it's far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free.
Part 8: Container
__len__(self)Returns the length of the container. Part of the protocol for both immutable and mutable containers.__getitem__(self, key)Defines behavior for when an item is accessed, using the notationself[key]. This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions:TypeErrorif the type of the key is wrong andKeyErrorif there is no corresponding value for the key.__setitem__(self, key, value)Defines behavior for when an item is assigned to, using the notationself[nkey] = value. This is part of the mutable container protocol. Again, you should raiseKeyErrorandTypeErrorwhere appropriate.__delitem__(self, key)Defines behavior for when an item is deleted (e.g.del self[key]). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used.__iter__(self)Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by theiter()built in function and when a container is looped over using the formfor x in container:. Iterators are their own objects, and they also must define an__iter__method that returnsself.__reversed__(self)Called to implement behavior for thereversed()built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple.__contains__(self, item)__contains__defines behavior for membership tests usinginandnot in. Why isn't this part of a sequence protocol, you ask? Because when__contains__isn't defined, Python just iterates over the sequence and returnsTrueif it comes across the item it's looking for.__missing__(self, key)__missing__is used in subclasses ofdict. It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionarydand saidd["george"]when"george"is not a key in the dict,d.__missing__("george")would be called).
Example 1 for __setitem__ and __getitem__:
class Building(object):
def __init__(self, floors):
self._floors = [None]*floors
def __setitem__(self, floor_number, data):
self._floors[floor_number] = data
def __getitem__(self, floor_number):
return self._floors[floor_number]building1 = Building(4) # Construct a building with 4 floors
building1[0] = 'Reception'
building1[1] = 'ABC Corp'
building1[2] = 'DEF Inc'
print( building1[2] )
Example 2 for __iter__
class MyClass:
def __init__(self):
self.data_ = [1, 2, 10, 200]
self.index_ = -1
def __len__(self):
return len(self.data_)
def __getitem__(self, index):
print('I am in __getitem__')
return self.data_[index]
def __iter__(self):
for i in range(len(self.data_)):
print('I am in __iter__')
yield self.data_[i]obj = MyClass()
print(list(obj))
for a in obj:
print(a)
__iter__ is tried first before falling back to the __getitem__ approach.
Part 9: Callable
__call__(self, [args...])Allows an instance of a class to be called as a function. Essentially, this means that x() is the same as x.__call__(). Note that __call__ takes a variable number of arguments; this means that you define __call__ as you would any other function, taking however many arguments you'd like it to.
__call__ can be particularly useful in classes with instances that need to often change state. "Calling" the instance can be an intuitive and elegant way to change the object's state. An example might be a class representing an entity's position on a plane:
class Entity:
'''Class to represent an entity. Callable to update the entity's position.'''
def __init__(self, size, x, y):
self.x, self.y = x, y
self.size = size
def __call__(self, x, y):
'''Change the position of the entity.'''
self.x, self.y = x, yAnother example is to use __call__ to invoke a function:
class Operator_A(object):
def __call__(self, img, *args, **kwargs):
print("a")
img = img+1
return img
class Operator_B(object):
def __call__(self, img, *args, **kwargs):
print("b")
img = img+1
return img
class Operator_C(object):
def __call__(self, img, *args, **kwargs):
print("c")
img = img+1
return img
operator_list=[Operator_A(), Operator_B(), Operator_C()]
img = 3
for op in operator_list:
img = op(img)
print(img)Part 10: Context Managers
Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a with statement. The behavior of the context manager is determined by two magic methods:
__enter__(self)Defines what the context manager should do at the beginning of the block created by the with statement. Note that the return value of __enter__ is bound to the target of the with statement, or the name after the as.__exit__(self, exception_type, exception_value, traceback)Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, exception_type, exception_value, and traceback will be None. Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure __exit__ returns True after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen.
__enter__ and __exit__ can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects.
class ContextManager():
def __init__(self):
print('init method called') def __enter__(self):
print('enter method called')
return self def __exit__(self, exc_type, exc_value, exc_traceback):
print ('exit method called') def print(self, my_str):
print(my_str+" from Context Manager")with ContextManager() as manager:
manager.print("hello")
Part 11: Descriptor Object
To be a descriptor, a class must have at least one of __get__, __set__, and __delete__ implemented. Let's take a look at those magic methods:
__get__(self, instance, owner)Define behavior for when the descriptor's value is retrieved. instance is the instance of the owner object. owner is the owner class itself.
__set__(self, instance, value)Define behavior for when the descriptor's value is changed. instance is the instance of the owner class and value is the value to set the descriptor to.
__delete__(self, instance)Define behavior for when the descriptor's value is deleted. instance is the instance of the owner object.
Part 12: Copying
Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python’s copy comes into play. However (fortunately), Python modules are not sentient, so we don't have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things.
__copy__(self)Defines behavior for copy.copy() for instances of your class. copy.copy() returns a shallow copy of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original).
__deepcopy__(self, memodict={})Defines behavior for copy.deepcopy() for instances of your class. copy.deepcopy() returns a deep copy of your object -- the object and its data are both copied. memodict is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, call copy.deepcopy() on that attribute with memodict as the first argument.
Part 13: Pickling
Pickling isn’t just for built-in types. It’s for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it’s a bit different for C extensions, but that’s not in our scope)
Part 14: Module
__file__ get the module’s installation directory