How to handle state from external systems functionally?

Question

I recently got into functional programming and I've learned several ways of dealing with some kinds of side effects in a referentially transparent way:

• the State monad for mutable state like updating a variable
• the IO monad for I/O like reading/writing from/to the console
• FRP for interactivity like graphics and input device events

But most of the "real-world" applications nowadays are interfacing with external systems (like web services, databases and such) which can be modified concurrently by multiple users, they have state, long running operations etc. So the case is not as simple as in the above categories: the result of either asking the system for the state of an entity or trying to control it depends on its state. Moreover, interactivity is also a requirement: there is some GUI which the user can click arbitrarily, and maybe we also have to automatically react to changes coming from the system.

What are the patterns of designing and implementing this sort of application by maximizing the benefit of pure functions? Or can some of the above approaches be applied to this problem in a way I haven't thought of? The language (say Java or Scala) doesn't enforce 100% purity, so I'm interested in pragmatic solutions supported by practical experience.

Answer 1

I haven't done a whole lot of this sort of thing that's actually come to completion in a practical way, so others will hopefully be able to do more. I have however written an Android app in Scala, which had several of your requirements; the UI is interactive, and the "state" is all stored in an SQLite database. Both the database and the UI require interfacing with the Android framework, which are very much Java-oriented and so not an easy fit for either Scala or functional programming.

What I did was to adopt something like an MVC design, where the model part was implemented as a set of ADTs supporting only pure operations. This had the additional advantage that the model code was completely independent of the Android framework, so it could be tested outside the emulator any way I liked.

This left me with the controller (the view was a very thin layer that was mostly just configuration, with the way Android works), plus the additional operations of "load the model from the database" and "save the model to the database". Being Scala, I simply implemented these parts with impure code that calls the pure model code to do actual data manipulation. In Haskell these parts would probably have been entirely in the IO monad[1].

All up, this allowed me to think about my problem domain in pure functional terms, without having to "go against the grain" when interfacing with the external systems. The database layer becomes the problem of mapping between a database schema and the ADTs I used for my data model; dealing with the fact that these operations could fail is the responsibility of the controller (which initiated the DB operation), and doesn't impact on the model. The controller operations become conceptually very simple like "when this button is pressed, set the current state to the result of calling a function on the current state, then update the display table". In the end there was much more of this impure "glue" code than the actual model code, but I still think it was a win to do the program this way, because the core of my app was manipulation of complex structured data, and so getting that right was the trickiest bit. The rest was mostly tedious to write rather than difficult to design.

This works when you have a substantial amount of computation in your program (not necessarily large amounts of data, just that you actually compute things on it). If the program is almost entirely gluing different external systems together, then you don't necessarily have as much to gain.

---

[1] Remember the IO monad isn't just for reading/writing from the console. Modelling operations whose result is affected by the state of something external to your program is exactly what the IO monad is for; generally Haskellers would try to avoid using the IO monad for almost the entirety of their program if it isn't interacting with an external system all the time (or preferably even if it is). You can do pure computation on data in response to events coming from "outside", or even do pure computation of the IO actions that need to be performed in response to an external event, if that's complex.

Answer 2

But most of the "real-world" applications nowadays are interfacing with external systems (like web services, databases and such) which can be modified concurrently by multiple users, they have state, long running operations etc. So the case is not as simple as in the above categories: the result of either asking the system for the state of an entity or trying to control it depends on its state. Moreover, interactivity is also a requirement: there is some GUI which the user can click arbitrarily, and maybe we also have to automatically react to changes coming from the system.

Interactively, concurrently editing a shared state is just another example of a state monad. You might uses lenses or some other abstraction for composing edits on a data structure, but under the hood all you have is a global state being shared.

If you want machine-level concurrency support, you might use a concurrent structure, such as an STM var or MVar , to resolve conflicts from concurrent edits. This means you'll be in an STM or IO monad.

There are many, many examples of monadic environments designed for these kinds of jobs on Hackage, for Haskell packages.