Python double-under, double-wonder

Python has a number of protocols that classes can opt into by implementing one or more “dunder methods”, aka double-underscore methods. Examples include __call__ (make an object behave like a function) or __iter__ (make an object iterable).

The choice of wrapping these functions with double-underscores on either side was really just a way of keeping the language simple. The Python creators didn’t want to steal perfectly good method names from you (such as “call” or “iter”), but they also did not want to introduce some new syntax just to declare certain methods “special”. The dunders achieve the dual goal of calling attention to these methods while also making them just the same as other plain methods in every aspect except naming convention.

Some people call these “magic methods”. Indeed one of the best guides online, A Guide to Python’s Magic Methods, uses this term. The reason I don’t like this term is that it makes it seem like dunders are only reserved for “real experts”, when quite the opposite is true. Indeed, nearly any new Python programmer uses __init__ to implement object initializers (aka constructors). The double-underscores don’t mean “reserved for wizards”; they simply mean, “reserved by the core Python team”.

One somewhat perplexing trend is that a few library authors have chosen to use the dunder conventions for their own code. For example, in SQLAlchemy, you use __tablename__ to map a SQLAlchemy ORM class to a SQL table. It then exposes new properties, __table__ and __mapper__. Ugh. For an otherwise beautifully designed library, this is so very wrong and completely misses the point. The dunder convention is a namespace reserved for the core Python team to implement their own protocols. Never use the namespace for your own, library-specific things! This defeats the whole purpose. If you need to hide a property, use _attribute or __attribute.

A recent video called “__dunder__ functions” walks through and demystifies many of the dunder protocols. It’s worth a watch, especially for Python beginners.

Moving forward, here are the rules you should follow for dunders:

  1. Call them “dunders” — Terminology like “magic” makes them seem much more complicated than they actually are. For example, it’s the “dunder call method”, not the “double-underscore call double-underscore method” and not the “magic call method”.
  2. Implement dunders on your classes at will — There is nothing magic about them, you shouldn’t feel like you’re using an esoteric language feature when you implement the __call__ method. Because, you’re not! It’s a standard language feature just like __init__!
  3. Never, ever, invent your own dunders — Python leaves you with a number of clean namespaces (classes, modules, etc.) for your own code. Use them! The core Python team reserved a somewhat ugly namespace for themselves — don’t trample all over their compromise by stealing their names.

2 Responses to “Python double-under, double-wonder”

  1. tm Says:

    indeed in the sqlalchemy case a single underscore would have perfectly done the job!

    just realized that internally i hadn’t really had a name for the dunder functions and “full control” or “behind the courtains” would be the closest that i have mapped in my head.

    although __init__ is by far the most used “dunder”, my personal favourites are __setattr__ and __getattr__ that get especially useful when bunch of class attributes affect each other and it makes sense to centralize the effort. like, say, you would have a class that you would like to mark as “changed” whenever any property on it is set

    that then gets as simple as def __setattr__(name, value): super(ParentClass, self).__setattr__(name, value); super(ParentClass, self).__setattr__(“_changed”, True)

    (the latter could also be set straight via __dict__, but it’s a better idea to allow the setattrs bubble up so that parents can decide if there is more to be done there.

  2. pixelmonkey Says:

    @tm Indeed, __setattr__ and __getattr__ are wonderful and under-utilized! I also personally love the descriptor protocol, which is extremely under-utilized. At PyCon, Luciano Ramalho gave a wonderful presentation that demystifies descriptors and shows that they are a powerful tool for encapsulation. See:

    http://pyvideo.org/video/1760/encapsulation-with-descriptors

Leave a Reply