What forces drove WAI Application to be redesigned five times?

Question

I took a curious look at WAI interface and while it looks simple, I was surprised to see how many iterations it took to stabilize at the current form!

I had assumed that CPS style for resource safety would be the most interesting thing but it looks like there is much more to learn from!

$ git log -p --reverse -- wai/Network/Wai.hs | grep '\+type Application'
+type Application = Request -> Iteratee B.ByteString IO Response
+type Application = Request -> ResourceT IO Response
+type Application = Request -> C.ResourceT IO Response
+type Application = Request -> IO Response
+type Application = Request -> (forall b. (Response -> IO b) -> IO b)
+type Application = Request -> (Response -> IO ResponseReceived)
                            -> IO ResponseReceived

Some archeology yields somewhat unsatisfactory results:

$ git log --reverse -G 'type Application' --pretty=oneline -- wai/Network/Wai.hs | cat
879d4a23047c3585e1cba4cdd7c3e8fc13e17592 Moved everything to wai subfolder
360442ac74f7e79bb0e320110056b3f44e15107c Began moving wai/warp to conduit
af7d1a79cbcada0b18883bcc5e5e19a1cd06ae7b conduit 0.3
fe2032ad4c7435709ed79683acac3b91110bba04 Pass around an InternalState instead of living in ResourceT
63ad533299a0a5bad01a36171d98511fdf8d5821 Application uses bracket pattern
1e1b8c222cce96c3d58cd27318922c318642050d ResponseReceived, to avoid existential issues

Answer 1

All the designs seem to be driven by three main concerns:

• Requests can have streamed bodies (so we don't have to load them all in memory before starting to process them). How to best represent it?
• Responses can be streamed as well. How to best represent it?
• How to ensure that resources allocated in the production of a response are properly freed? (For example, how to ensure that file handles are freed after serving a file?)

---

type Application = Request -> Iteratee B.ByteString IO Response

This version uses iteratees, which were an early solution for streaming data in Haskell. Iteratee consumers had to be written in a "push-based" way, which was arguably less natural than the "pull-based" consumers used in modern streaming libraries.

The streamed body of the request is fed to the iteratee and we get a Response value at the end. The Response contains an enumerator (a function that feeds streamed response bytes to a response iteratee supplied by the server). Presumably, the enumerator would control resource allocation using functions like bracket.

---

type Application = Request -> ResourceT IO Response

This version uses the resourcet monad transformer for resource management, instead of doing it in the enumerator. There is a special Source type inside both Request and Response which handles streamed data (and which is a bit hard to understant IMHO).

---

type Application = Request -> IO Response

This version uses the streaming abstractions from conduit, but eschews resourcet and instead provides a bracket-like responseSourceBracket function for handling resources in streamed responses.

---

type Application = Request -> (forall b. (Response -> IO b) -> IO b)
type Application = Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived

This version moves to a continuation-based approach which enables the handler function to use regular bracket-like functions to control resource allocation. Back to square one, in that respect!

Conduits are no longer used for streaming. Now there is a plain Request -> IO ByteString function for reading chunks of the request body, and a (Builder -> IO ()) -> IO () -> IO () function in the Response for generating the response stream. (The Builder -> IO () write function along with a flush action are supplied by the server.)

Like the resourcet-based versions, and unlike the iteratee-based version, this implementation lets you overlap reading the request body with streaming the response.

The polymorphic handler is a neat trick to ensure that the response-taking callback Response -> IO b is always called: the handler needs to return a b, and the only way to get one is to actually invoke the callback!

This polymorphic solution seems to have caused some problems (perhaps with storing handlers in containers?) Instead of using polymorphism, we can use a ResponseReceived token without a public constructor. The effect is the same: the only way for handler code to get hold of the token it needs to return is to invoke the callback.