Architecture for providing different linear algebra back-ends

Question

I am prototyping a new system in Python; the functionality is mostly numerical.

An important requirement is the ability to use different linear algebra back-ends: from individual user implementations to generic libraries, such as Numpy. The linear algebra implementation (that is, the back-end) must be independent from the interface.

My initial architectural attempt is as follows:

(1) Define the system interface

>>> v1 = Vector([1,2,3])
>>> v2 = Vector([4,5,6])
>>> print v1 * v2
>>> # prints "Vector([4, 10, 18])"

(2) Implement the code allowing to use that interface independently of the back-end

# this example uses numpy as the back-end, but I mean
# to do this for a general back-end
import numpy 
def numpy_array(*args): # creates a numpy array from the arguments
    return numpy.array(*args)

class VectorBase(type):
    def __init__(cls, name, bases, attrs):
        engine = attrs.pop("engine", None)
        if not engine:
            raise RuntimeError("you need to specify an engine")
        # this implementation would change depending on `engine`
        def new(cls, *args):
            return numpy_array(*args)   
        setattr(cls, "new", classmethod(new))

class Vector(object):   
    __metaclass__ = VectorBase        
    # I could change this at run time
    # and offer alternative back-ends
    engine = "numpy"  
    @classmethod
    def create(cls, v):
        nv = cls()
        nv._v = v
        return nv    
    def __init__(self, *args):  
        self._v = None
        if args:
            self._v = self.new(*args)
    def __repr__(self):
        l = [item for item in self._v]
        return "Vector(%s)" % repr(l)
    def __mul__(self, other):
        try:
            return Vector.create(self._v * other._v)
        except AttributeError:
            return Vector.create(self._v * other)
    def __rmul__(self, other):
        return self.__mul__(other)

This simple example works as follows: the Vector class keeps a reference to a vector instance made by the back-end (numpy.ndarray in the example); all arithmetic calls are implemented by the interface, but their evaluation is deferred to the back-end.

In practice, the interface overloads all the appropriate operators and defers to the back-end (the example only shows __mul__ and __rmul__, but you can follow that the same would be done for every operation).

I am willing to loose some performance in exchange of customizability. Even while my example works, it does not feel right -- I would be crippling the back-end with so many constructor calls! This begs for a different metaclass implementation, allowing for a better call deferment.

So, how would you recommend that I implement this functionality? I need to stress the importance of keeping all of the system's Vector instances homogeneous and independent of the linear algebra back-end.

Answer 1

You should check out PEP-3141 and the standard lib module ABCMeta.

For a detailed explanation of how to use ABCMeta, the always helpful PyMOTW has a nice write-up.