Python Special/Magic Methods

· Part 1: Introduction
· Part 2: Construction and Initialization
· Part 3: Comparison
· Part 4: Numeric Magic Method
· Part 5: Type Conversion
· Part 6: Class Representation
· Part 7: Attribute Controlling
· Part 8: Container
· Part 9: Callable
· Part 10: Context Managers
· Part 11: Descriptor Object
· Part 12: Copying
· Part 13: Pickling
· Reference

Part 1: Introduction

  • The biggest advantages of using Python’s magic method is that they provide a simple way to make objects behave like built-in types.
  • Python’s magic methods are incredibly powerful, and with great power comes great responsibility. It’s important to know the proper way to use magic methods so you don’t break any code.
  • Protocols are somewhat similar to interfaces in other languages in that they give you a set of methods you must define. However, in Python protocols are totally informal and require no explicit declarations to implement. Rather, they’re more like guidelines.

Part 2: Construction and Initialization

  • is the first method to get called in an object’s instantiation. However, it is rarely used. Like , which is also rarely used, is mainly used for garbage collection.

Part 3: Comparison

  • is the most basic comparison magic method. It actually implements behavior for all of the comparison operators (<, ==, !=, etc.). It’s usually best to define each comparison you need rather than define them all at once, but can be a good way to save repetition and improve clarity when you need all comparisons implemented with similar criteria.
  • The other comparison methods include , , , , , .

Part 4: Numeric Magic Method

  • Unary operators include the following:
  • arithmetic operators
  • reflected arithmetic operators : So, all of these magic methods do the same thing as their normal equivalents, except the perform the operation with other as the first operand and self as the second, rather than the other way around.
  • augmented assignment: Each of these methods should return the value that the variable on the left hand side should be assigned to (for instance, for , might return , which would be assigned to

Part 5: Type Conversion

Python also has an array of magic methods designed to implement behavior for built in type conversion functions like

Part 6: Class Representation

  • The major difference between and is intended audience. is intended to produce output that is mostly machine-readable (in many cases, it could be valid Python code even), whereas is intended to be human-readable.
class Point(object):
def __init__(self, u, v, d):
self.u = u
self.v = v
self.d = d
def __repr__(self):
return f"Point(u={self.u},v={self.v}, d={self.d})"
def __eq__(self, other):
return self.u==other.u and self.v == other.v and self.d == other.d

a = Point(3,4, 5)
print(a)
b=eval('Point(u=3,v=4, d=5)')
print(b)
print(a==b)

The output is:

Point(u=3,v=4, d=5)
Point(u=3,v=4, d=5)
True

Two special methods, __repr__ and __str__, are essential for creating proper string representations of your custom class, which will give the code readers more intuitive information about your classes. Between them, the major difference is that the __repr__ method defines the string, using which you can re-create the object by calling eval(repr(“the repr”)), while the __str__ method defines the string that is more descriptive and allows more customization. In other words, you can think that the string defined in the __repr__ method is to be viewed by developers while that used in the __str__ method is to be viewed by regular users. The following shows you an example.

  • Defines behavior for when is called on an instance of your class. is like , but it returns a unicode string. Be wary: if a client calls on an instance of your class and you've only defined , it won't work. You should always try to define as well in case someone doesn't have the luxury of using unicode.
  • Defines behavior for when an instance of your class is used in new-style string formatting. For instance, would lead to the call . This can be useful for defining your own numerical or string types that you might like to give special formatting options.
  • Defines behavior for when is called on an instance of your class. It has to return an integer, and its result is used for quick key comparison in dictionaries. Note that this usually entails implementing as well. Live by the following rule: implies .
  • Defines behavior for when is called on an instance of your class. Should return or , depending on whether you would want to consider the instance to be or .

Part 7: Attribute Controlling

  • You can define behavior for when a user attempts to access an attribute that doesn't exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes (you can still choose to compute and return that attribute, if you wish), or deftly handing an . It only gets called when a nonexistent attribute is accessed, however, so it isn't a true encapsulation solution.
  • Unlike , is an encapsulation solution. It allows you to define behavior for assignment to an attribute regardless of whether or not that attribute exists, meaning you can define custom rules for any changes in the values of attributes.
  • This is the exact same as , but for deleting attributes instead of setting them. The same precautions need to be taken as with as well in order to prevent infinite recursion.
  • After all this, fits in pretty well with its companions and . However, I don't recommend you use it. can only be used with new-style classes (all classes are new-style in the newest versions of Python, and in older versions you can make a class new-style by subclassing . It allows you to define rules for whenever an attribute's value is accessed. It suffers from some similar infinite recursion problems as its partners-in-crime (this time you call the base class's method to prevent this). It also mainly obviates the need for , which, when is implemented, only gets called if it is called explicitly or an is raised. This method can be used (after all, it's your choice), but I don't recommend it because it has a small use case (it's far more rare that we need special behavior to retrieve a value than to assign to it) and because it can be really difficult to implement bug-free.

Part 8: Container

  • Returns the length of the container. Part of the protocol for both immutable and mutable containers.
  • Defines behavior for when an item is accessed, using the notation . This is also part of both the mutable and immutable container protocols. It should also raise appropriate exceptions: if the type of the key is wrong and if there is no corresponding value for the key.
  • Defines behavior for when an item is assigned to, using the notation . This is part of the mutable container protocol. Again, you should raise and where appropriate.
  • Defines behavior for when an item is deleted (e.g. ). This is only part of the mutable container protocol. You must raise the appropriate exceptions when an invalid key is used.
  • Should return an iterator for the container. Iterators are returned in a number of contexts, most notably by the built in function and when a container is looped over using the form . Iterators are their own objects, and they also must define an method that returns .
  • Called to implement behavior for the built in function. Should return a reversed version of the sequence. Implement this only if the sequence class is ordered, like list or tuple.
  • defines behavior for membership tests using and . Why isn't this part of a sequence protocol, you ask? Because when isn't defined, Python just iterates over the sequence and returns if it comes across the item it's looking for.
  • is used in subclasses of . It defines behavior for whenever a key is accessed that does not exist in a dictionary (so, for instance, if I had a dictionary and said when is not a key in the dict, would be called).

Example 1 for and :

class Building(object):
def __init__(self, floors):
self._floors = [None]*floors
def __setitem__(self, floor_number, data):
self._floors[floor_number] = data
def __getitem__(self, floor_number):
return self._floors[floor_number]
building1 = Building(4) # Construct a building with 4 floors
building1[0] = 'Reception'
building1[1] = 'ABC Corp'
building1[2] = 'DEF Inc'
print( building1[2] )

Example 2 for

class MyClass:
def __init__(self):
self.data_ = [1, 2, 10, 200]
self.index_ = -1
def __len__(self):
return len(self.data_)
def __getitem__(self, index):
print('I am in __getitem__')
return self.data_[index]

def __iter__(self):
for i in range(len(self.data_)):
print('I am in __iter__')
yield self.data_[i]
obj = MyClass()
print(list(obj))
for a in obj:
print(a)

is tried first before falling back to the approach.

Part 9: Callable

Allows an instance of a class to be called as a function. Essentially, this means that is the same as . Note that takes a variable number of arguments; this means that you define as you would any other function, taking however many arguments you'd like it to.

can be particularly useful in classes with instances that need to often change state. "Calling" the instance can be an intuitive and elegant way to change the object's state. An example might be a class representing an entity's position on a plane:

class Entity:
'''Class to represent an entity. Callable to update the entity's position.'''

def __init__(self, size, x, y):
self.x, self.y = x, y
self.size = size

def __call__(self, x, y):
'''Change the position of the entity.'''
self.x, self.y = x, y

Another example is to use to invoke a function:

class Operator_A(object):
def __call__(self, img, *args, **kwargs):
print("a")
img = img+1
return img

class Operator_B(object):
def __call__(self, img, *args, **kwargs):
print("b")
img = img+1
return img

class Operator_C(object):
def __call__(self, img, *args, **kwargs):
print("c")
img = img+1
return img

operator_list=[Operator_A(), Operator_B(), Operator_C()]
img = 3
for op in operator_list:
img = op(img)
print(img)

Part 10: Context Managers

Context managers allow setup and cleanup actions to be taken for objects when their creation is wrapped with a statement. The behavior of the context manager is determined by two magic methods:

Defines what the context manager should do at the beginning of the block created by the statement. Note that the return value of is bound to the target of the statement, or the name after the .Defines what the context manager should do after its block has been executed (or terminates). It can be used to handle exceptions, perform cleanup, or do something always done immediately after the action in the block. If the block executes successfully, , , and will be . Otherwise, you can choose to handle the exception or let the user handle it; if you want to handle it, make sure returns after all is said and done. If you don't want the exception to be handled by the context manager, just let it happen.

and can be useful for specific classes that have well-defined and common behavior for setup and cleanup. You can also use these methods to create generic context managers that wrap other objects.

class  ContextManager():
def __init__(self):
print('init method called')
def __enter__(self):
print('enter method called')
return self
def __exit__(self, exc_type, exc_value, exc_traceback):
print ('exit method called')
def print(self, my_str):
print(my_str+" from Context Manager")
with ContextManager() as manager:
manager.print("hello")

Part 11: Descriptor Object

To be a descriptor, a class must have at least one of , , and implemented. Let's take a look at those magic methods:

Define behavior for when the descriptor's value is retrieved. is the instance of the owner object. is the owner class itself.

Define behavior for when the descriptor's value is changed. is the instance of the owner class and is the value to set the descriptor to.

Define behavior for when the descriptor's value is deleted. is the instance of the owner object.

Part 12: Copying

Sometimes, particularly when dealing with mutable objects, you want to be able to copy an object and make changes without affecting what you copied from. This is where Python’s comes into play. However (fortunately), Python modules are not sentient, so we don't have to worry about a Linux-based robot uprising, but we do have to tell Python how to efficiently copy things.

Defines behavior for for instances of your class. returns a shallow copy of your object -- this means that, while the instance itself is a new instance, all of its data is referenced -- i.e., the object itself is copied, but its data is still referenced (and hence changes to data in a shallow copy may cause changes in the original).

Defines behavior for for instances of your class. returns a deep copy of your object -- the object and its data are both copied. is a cache of previously copied objects -- this optimizes copying and prevents infinite recursion when copying recursive data structures. When you want to deep copy an individual attribute, call on that attribute with as the first argument.

Part 13: Pickling

Pickling isn’t just for built-in types. It’s for any class that follows the pickle protocol. The pickle protocol has four optional methods for Python objects to customize how they act (it’s a bit different for C extensions, but that’s not in our scope)

Reference

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store