Part I: pythons-descriptors-part-i-let-the-hunt-begin/

In part I, we started the travel to find the way to craft Python’s descriptors. We known why we need descriptors, and some trouble to know why descriptors are not easy to implement. However, it’s just a way to implement our code, right? So keep going, in this part we will find another way to do a right descriptor.

Descriptor with dictionary

In part I, we are going to find a way to store kind instance separated for each Dog. I was thinking about dictionary to store kind for each Dog, so let’s try to use some dictionary. But first, do you notice the instance param in descriptor’s protocol (__set__(), __get__(), __delete__()). Take a look:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class StringField(object):
    def __set__(self, instance, value):
        print("SET:", instance)
        if not isinstance(value, str):
            raise ValueError("Value must be string")
        self.value = value

    def __get__(self, instance, owner):
        return self.value

class Dog(object):
    kind = StringField()

a = Dog()
a.kind = "Husky"
print("OBJ:", a)

1
2
3

SET: <__main__.Dog object at 0x10b94ba20>
OBJ: <__main__.Dog object at 0x10b94ba20>

Actually, instance is the object have descriptors as its attributes. So I go to this implement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from weakref import WeakKeyDictionary
class StringField(object):
    def __init__(self):
        self.data = WeakKeyDictionary()
    def __set__(self, instance, value):
        if not isinstance(value, str):
            raise ValueError("Value must be string")
        self.data[instance] = value

    def __get__(self, instance, owner):
        return self.data.get(instance)

class Dog(object):
    kind = StringField()

a = Dog()
b = Dog()
a.kind = "Husky"
b.kind = "Pug"

print("DOG a:", a.kind)
print("DOG b:", b.kind)

1
2
DOG a: Husky
DOG b: Pug

Yes, it worked. But it’s not done yet. Let’s think about dictionary’s key. Dictionary’s keys must be hashable, so if instance is not hashable objects, such as list, our code will be ruined.

1
2
3
4
5
class List(list):
    kind = StringField()

a = List()
a.kind = "ERROR"

1
2
3
4
5
6
Traceback (most recent call last):
  File "*.py", line 18, in 
    a.kind = "ERROR"
  File "*.py", line 8, in __set__
    self.data[instance] = value
TypeError: unhashable type: 'Dog'

It’s not a really big problem if we can use a label for each instance to avoid it. Unfortunately, we can’t set weakref key is str or int. At this time, I find out I forgot a python build-in __dict__

Finally

No more dictionary

So, let’s change one or two lines of code, we will have descriptors, and believe me, it will work this time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class StringField(object):
    def __init__(self, label):
        self.label = label

    def __set__(self, instance, value):
        if not isinstance(value, str):
            raise ValueError("Value must be string")
        instance.__dict__[self.label] = value

    def __get__(self, instance, owner):
        return instance.__dict__.get(self.label)

class List(list):
    kind = StringField("kind")  # Repeated attribute's name ?
    name = StringField("name")  # Repeated attribute's name ?

a = List()
b = List()
a.kind = "Husky"

print(a.kind, a.name)
print(b.kind, b.name)

1
2
Husky None
None None

Yes. It’s worked now. However, we will repeat label as attributes name every time. Not good huh? Actually, this is an acceptable way to do descriptors and it’s fairly common. There are some books using this implementation. However, I don’t like it because as I mentioned in Part I, I’m going to find a way to do data objects as Mongoengine did, and I will continue my travel. We have nearly done. We need to find a way to set label automatically by attributes name.

Auto set label with metaclasses

Yeah, the true magic behind the most beautiful way to craft descriptors is metaclass. A metaclass is the class of a class. Like a class defines how an instance of the class behaves, a metaclass defines how a class behaves. A class is an instance of a metaclass. This blog will explain it more detailed.

So, let’s apply metaclass in our code and finish it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
class DescriptorOwner(type):
    """Descriptor metaclass, the class of class will auto-set label"""
    def __new__(mcs, name, bases, attrs):
        # find all descriptors, auto-set their labels
        for n, v in attrs.items():
            if isinstance(v, DescriptorField):
                v.label = n
        return super(DescriptorOwner, mcs).__new__(mcs, name, bases, attrs)

class DescriptorField(object):
    """General descriptor field"""
    def __init__(self):
        # notice we aren't setting the label here
        self.error_message = None
        self.validate = None
        self.label = None

    def __get__(self, instance, owner):
        return instance.__dict__.get(self.label)

    def __set__(self, instance, value):
        if not self.validate and value is not None:
            raise ValueError(self.error_message)
        else:
            instance.__dict__[self.label] = value

class StringField(DescriptorField):
    """
    This is your playground, any "binding behavior" will be set here. And don't
    forget __get__() and __delelte__(). It can work like a trigger for your
    object.
    """

    def __set__(self, instance, value):
        self.validate = isinstance(value, str)
        self.error_message = self.label + " must be a "
        super(StringField, self).__set__(instance, value)

class Dog(object, metaclass=DescriptorOwner):
    kind = StringField()
    name = StringField()

a = Dog()
b = Dog()
a.kind = "Husky"

print(a.kind, a.name)
print(b.kind, b.name)

1
2
Husky None
None None

OK, now our descriptors are completed and work like a charm, like some libraries such as Mongoengine do. We can do many things with descriptors not just check attribute’s type, such as setting value is nullable or not, setting default value or calling another function to do some triggers …

Conclusion

So, this is a full road since I started the hunt to find descriptor’s recipe, every single step I walked is detailed. I hope you now have an understanding of what descriptors are, and how to use it. Feel free to leave your comments and discuss about descriptors here.