Friday, October 16, 2009

Missing Method in Python

Preamble - optional

Ruby has a whole boat load of methods which allow a programmer to overload and override all kinds of stuff. One of them is 'method_missing', which is a method which is called if you invoke '' and 'bar' is not the name of a method in 'foo'.

The Rails folks use it to transform function names like: find_foo_by_date_and_color(date, color) into a parameterized sql call, thus trading speed for apparent code clarity, added confusion for humans trying to understand the code, and adding inches to their stature as cool programmers.

You probably get from this that I think the Rails people overuse things like method_missing.

You're right.

But that doesn't mean it isn't useful.

So to add inches to my cool programmer stature, here's a case I think is justified - and how to do it in Python, which doesn't directly support method_missing (at least they don't seem to admit it - as far as I was able to find)

The Problem

I'm writing a little local HTTP server for my YASiteKit CMS/web-site kit thing ( and decided to make it HTTPS capable. This turned out to be just a few lines of code in Python 2.6, but it broke the server. It turns out that while SSLSocket objects support some 'file-like' methods, they don't support all of them - such as readline() - which are used in the bowels of the HTTP server library.

The 'obvious solution' is to wrap the SSLSocket in something which adds the necessary methods.

But I'm lazy - in a sense - and decided to try to do this using a Pythonic equivalent to Ruby's missing method machinery.

Python has a rich set of special object methods for defining operations and the like, but it doesn't directly support a missing method call. It does support customizing attribute access so that you can define dynamic attributes - that is attributes which don't actually exist, but have computed values or that you want to create and delete dynamically after compile time.

I was stuck on how to pass arguments to an arbitrary method of a wrapped object instance until I realized that method invocation in Python is a two step process:
  1. look up the attribute
  2. call it with arguments
So the solution is easy:
  1. wrap the object by passing it to the object constructor
  2. create a __getattr__() method which returns getattr(wrapped-instance, method-name)
  3. the Python interpreter then completes the call, with arguments, on the bound method.
If the method exists in wrapped-instance, then the returned value is the method, bound to the wrapped-instance. If not, then wrapped-instance throws an AttributeError exception.

It doesn't interfere with the wrapper's methods because __getattr__() is only called if the attribute is not found.

Besides avoiding writing a lot of method wrapping code, this has the advantage of telling me exactly which methods I have to implement - because when they are called on my wrapper, the wrapped object throws up.

Here's the sample code I wrote to verify the method:
class Floof(object):
var = 'This is Floof'
def __init__(self, var):
self.var = var
def func(self, var):
print('\nThis is Floof.func called on self')
print('Floof.var: ', Floof.var)
print('Floof.self.var: ', self.var)
print('Floof.func(var): ', var)

class Foo(object):
var = 'This is Foo'
def __init__(self, var, floof_instance):
self.var = var
self.floof_instance = floof_instance
def __getattr__(self, name):
print('\nFoo.__getattr__(%s) called' % name)
return getattr(self.floof_instance, name)
raise AttributeError('Foo instance: Attribute %s not found' % name)
def foo_func(self, var):
print('\nThis is Foo.func called on self')
print('Foo.var: ', Foo.var)
print('Foo.self.var: ', self.var)
print('Foo.foo_func(var): ', var)
def wrapped_func(self, var):
print('\nI\'m wrapping Floof.func()')

floof = Floof('I\'m a Floof')
floof.func('calling floof.func() directly')
foo = Foo('I\'m a Foo', floof)
foo.foo_func('calling foo.foo_func() directly')
foo.func('calling floof.func() through Foo wrapper')
foo.wrapped_func('calling floof.func() via a wrapper method')
foo.flob('calling foo.flob() which does not exist')
Here's the output:

This is Floof.func called on self
('Floof.var: ', 'This is Floof')
('Floof.self.var: ', "I'm a Floof")
('Floof.func(var): ', 'calling floof.func() directly')

This is Foo.func called on self
('Foo.var: ', 'This is Foo')
('Foo.self.var: ', "I'm a Foo")
('Foo.foo_func(var): ', 'calling foo.foo_func() directly')

Foo.__getattr__(func) called

This is Floof.func called on self
('Floof.var: ', 'This is Floof')
('Floof.self.var: ', "I'm a Floof")
('Floof.func(var): ', 'calling floof.func() through Foo wrapper')

I'm wrapping Floof.func()

This is Floof.func called on self
('Floof.var: ', 'This is Floof')
('Floof.self.var: ', "I'm a Floof")
('Floof.func(var): ', 'calling floof.func() via a wrapper method')

Foo.__getattr__(flob) called

AttributeError: 'Floof' object has no attribute 'flob'

function __getattr__ in at line 52
return getattr(self.floof_instance, name)

See - It works!