How to avoid massive amounts of boilerplate in Julia with new types?

Question

I'm considering writing a type similar to those defined in NamedArrays and Images. Let's say I just want essentially an Array with a piece of metadata, say a user-friendly name that I'll write at the top of a file when I write the array to disk. (This detail is not relevant; I'm just contriving an example.)

So I might do

type MyNamedArray
    data::Array
    name::ASCIIString
end

function mywrite(f,x::MyNamedArray)
     write(f,x.name)
     write(f,x.data)
end

or something, and no other behavior needs to be different from the base Array behavior.

In my head, it's "obvious" that I just want every existing function that operates on Arrays to operate on the data field of this type. In another language e.g. Java I might have just subclassed Array and added name as an instance field to the subclass, which would automatically preserve compatibility with all the existing Array operations. But in Julia, if I try a solution such as that above, I now need to define a ton more functions, e.g. as @TimHoly and 'davidavdav' have done in the linked packages.

Of course I am aware that being forced to write out some of these functions by hand is useful for realizing things that you haven't thought through. E.g. in the example MyNamedArray I give above, one could object by pointing out that I haven't defined the name of x::MyNamedArray * y::MyNamedArray. But what if I just don't care about that, and want code that "just works," without so much boilerplate? (See e.g. looping over symbols to push new method definitions in NamedArrays and manually writing out a hundred lines of definitions in Images. The vast majority of these definitions are boilerplate / the "obvious" definition.)

Specifically to continue the example I cited, for MyNamedArray, the default could be x*y is no longer a MyNamedArray, i.e. since every function just defaults to the "inherited" behavior of applying the same function on the underlying data, we can just forget the metadata on all pre-existing functions.

Note, I find Tomas Lycken's answer here insightful, and so are the question and answers here.

The best synthesis I can come up with is "you just have to suck it up and write out the functions, or write a macro that does that for you." If this is the case, so be it; I'm just wondering if I'm missing a better option, particularly a better way to design the solution to make it more Julian and avoid the boilerplate.

Answer 1

You can get most of the way there by simply subclassing AbstractArray: http://docs.julialang.org/en/latest/manual/interfaces/#abstract-arrays. In fact, you can do one better and subclass DenseArray, which additionally requires defining a stride (and probably pointer) function… allowing your custom array to work with BLAS. That's just a handful of methods you need to define. It's not 100% since many authors still have a tendency to overly restrict methods to only accept Array when they could easily accept all AbstractArrays. This is something that's gotten significantly better in the past two years, and it's still improving.

In general, a pattern I've found very useful here is to define interfaces in terms of abstract supertypes and loosen method signatures as much as possible. If dispatch restrictions aren't necessary, you can allow any type and just lean upon duck-typing. If you only restrict dispatch to specific leaf-types when telling Julia how it should quack or when leaning on its internal implementation, then your work becomes much more extensible and re-usable.