Is Python's "with" monadic?

Question

Like many a foolhardy pioneer before me, I'm endeavoring to cross the trackless wasteland that is Understanding Monads.

I'm still staggering through, but I can't help noticing a certain monad-like quality about Python's with statement. Consider this fragment:

with open(input_filename, 'r') as f:    for line in f:        process(line) 

Consider the open() call as the "unit" and the block itself as the "bind". The actual monad isn't exposed (uh, unless f is the monad), but the pattern is there. Isn't it? Or am I just mistaking all of FP for monadry? Or is it just 3 in the morning and anything seems plausible?

A related question: if we have monads, do we need exceptions?

In the above fragment, any failure in the I/O can be hidden from the code. Disk corruption, the absence of the named file, and an empty file can all be treated the same. So no need for a visible IO Exception.

Certainly, Scala's Option typeclass has eliminated the dreaded Null Pointer Exception. If you rethought numbers as Monads (with NaN and DivideByZero as the special cases)...

Like I said, 3 in the morning.

Answer 1

It's almost too trivial to mention, but the first problem is that with isn't a function and doesn't take a function as an argument. You can easily get around this by writing a function wrapper for with:

def withf(context, f):     with context as x:         f(x) 

Since this is so trivial, you could not bother to distinguish withf and with.

The second problem with with being a monad is that, as a statement rather than an expression, it doesn't have a value. If you could give it a type, it would be M a -> (a -> None) -> None (this is actually the type of withf above). Speaking practically, you can use Python's _ to get a value for the with statement. In Python 3.1:

class DoNothing (object):     def __init__(self, other):         self.other = other     def __enter__(self):         print("enter")         return self.other     def __exit__(self, type, value, traceback):         print("exit %s %s" % (type, value))  with DoNothing([1,2,3]) as l:     len(l)  print(_ + 1) 

Since withf uses a function rather than a code block, an alternative to _ is to return the value of the function:

def withf(context, f):     with context as x:         return f(x) 

There is another thing preventing with (and withf) from being a monadic bind. The value of the block would have to be a monadic type with the same type constructor as the with item. As it is, with is more generic. Considering agf's note that every interface is a type constructor, I peg the type of with as M a -> (a -> b) -> b, where M is the context manager interface (the __enter__ and __exit__ methods). In between the types of bind and with is the type M a -> (a -> N b) -> N b. To be a monad, with would have to fail at runtime when b wasn't M a. Moreover, while you could use with monadically as a bind operation, it would rarely make sense to do so.

The reason you need to make these subtle distinctions is that if you mistakenly consider with to be monadic, you'll wind up misusing it and writing programs that will fail due to type errors. In other words, you'll write garbage. What you need to do is distinguish a construct that is a particular thing (e.g. a monad) from one that can be used in the manner of that thing (e.g. again, a monad). The latter requires discipline on the part of a programmer, or the definition of additional constructs to enforce the discipline. Here's a nearly monadic version of with (the type is M a -> (a -> b) -> M b):

def withm(context, f):     with context as x:         return type(context)(f(x)) 

In the final analysis, you could consider with to be like a combinator, but a more general one than the combinator required by monads (which is bind). There can be more functions using monads than the two required (the list monad also has cons, append and length, for example), so if you defined the appropriate bind operator for context managers (such as withm) then with could be monadic in the sense of involving monads.

Answer 2

Yes.

Right below the definition, Wikipedia says:

In object-oriented programming terms, the type construction would correspond to the declaration of the monadic type, the unit function takes the role of a constructor method, and the binding operation contains the logic necessary to execute its registered callbacks (the monadic functions).

This sounds to me exactly like the context manager protocol, the implementation of the context manager protocol by the object, and the with statement.

From @Owen in a comment on this post:

Monads, at their most basic level, are more or less a cool way to use continuation-passing style: >>= takes a "producer" and a "callback"; this is also basically what with is: a producer like open(...) and a block of code to be called once it's created.

The full Wikipedia definition:

A type construction that defines, for every underlying type, how to obtain a corresponding monadic type. In Haskell's notation, the name of the monad represents the type constructor. If M is the name of the monad and t is a data type, then "M t" is the corresponding type in the monad.

This sounds like the context manager protocol to me.

A unit function that maps a value in an underlying type to a value in the corresponding monadic type. The result is the "simplest" value in the corresponding type that completely preserves the original value (simplicity being understood appropriately to the monad). In Haskell, this function is called return due to the way it is used in the do-notation described later. The unit function has the polymorphic type t→M t.

The actual implementation of the context manager protocol by the object.

A binding operation of polymorphic type (M t)→(t→M u)→(M u), which Haskell represents by the infix operator >>=. Its first argument is a value in a monadic type, its second argument is a function that maps from the underlying type of the first argument to another monadic type, and its result is in that other monadic type.

This corresponds to the with statement and its suite.

So yes, I'd say with is a monad. I searched PEP 343 and all the related rejected and withdrawn PEPs, and none of them mentioned the word "monad". It certainly applies, but it seems the goal of the with statement was resource management, and a monad is just a useful way to get it.