how does one _model_ data from relational databases in clojure?

Question

I have asked this question on twitter as well the #clojure IRC channel, yet got no responses.

There have been several articles about Clojure-for-Ruby-programmers, Clojure-for-lisp-programmers.. but what is the missing part is Clojure for ActiveRecord programmers .

There have been articles about interacting with MongoDB, Redis, etc. - but these are key value stores at the end of the day. However, coming from a Rails background, we are used to thinking about databases in terms of inheritance - has_many, polymorphic, belongs_to, etc.

The few articles about Clojure/Compojure + MySQL (ffclassic) - delve right into sql. Of course, it might be that an ORM induces impedence mismatch, but the fact remains that after thinking like ActiveRecord, it is very difficult to think any other way.

I believe that relational DBs, lend themselves very well to the object-oriented paradigm because of them being , essentially, Sets. Stuff like activerecord is very well suited for modelling this data. For e.g. a blog - simply put

class Post < ActiveRecord::Base
  has_many :comments
 end


 class Comment < ActiveRecord::Base
   belongs_to :post
 end

How does one model this in Clojure - which is so strictly anti-OO ? Perhaps the question would have been better if it referred to all functional programming languages, but I am more interested from a Clojure standpoint (and Clojure examples)

Answer 1

There are a couple ORM-like libraries in the works nowadays.

• clj-record
• Carte
• Oyako (Disclaimer, I wrote this one.)
• ClojureQL is more of an SQL-generating lib from what I can see, but it deserves mention.

On the mailing list, some (smart) people recently described some other models for how this might work. Each of these libraries takes a fairly different approach to the problem, so be sure to take a look at them all.

Here's an extended example using Oyako, for example. This library isn't production ready and still under heavy development, so the example may be invalid in a week, but it's getting there. It's a proof-of-concept in any case. Give it some time and someone will come up with a good library.

Note that clojure.contrib.sql already lets you fetch records from a DB (via JDBC) and end up with immutable hash-maps representing records. Because the data ends up in normal maps, all of the myriad Clojure core functions that work on maps already work on this data.

What else does ActiveRecord give you? I can think of a couple of things.

A concise SQL-query DSL

The way I model this mentally: First you define the relationship between the tables. This doesn't require mutation or objects. It's a static description. AR spreads this information out in a bunch of classes, but I see it as a separate (static) entity.

Using the defined relationships, you can then write queries in a very concise manner. With Oyako for example:

(def my-data (make-datamap db [:foo [has-one :bar]]
                              [:bar [belongs-to :foo]]))

(with-datamap my-data (fetch-all :foo includes :bar))

Then you'll have some foo objects, each with a :bar key that lists your bars.

In Oyako, the "data map" is just a map. The query itself is a map. The returned data is a vector of maps (of vectors of maps). So you end up with a standard, easy way to construct and manipulate and iterate over all of these things, which is nice. Add some sugar (macros and normal functions), to let you concisely create and manipulate these maps more easily, and it ends up being quite powerful. This just just one way, there are a lot of approaches.

If you look at a library like Sequel for another example, you have things like:

Artist.order(:name).last

But why do these functions have to be methods that live inside of objects? An equivalent in Oyako might be:

(last (-> (query :artist) 
          (order :name)))

Save/update/delete records

Again, why do you need OO-style objects or mutation or implementation inheritance for this? First fetch the record (as an immutable map), then thread it through a bunch of functions, associng new values onto it as needed, then stuff it back into the database or delete it by calling a function on it.

A clever library could make use of metadata to keep track of which fields have been altered, to reduce the amount of querying needed to do updates. Or to flag the record so the DB functions know which table to stick it back into. Carte even does cascading updates (updating sub-records when a parent record is altered), I think.

Validations, hooks

Much of this I see as belonging in the database rather than in the ORM libary. For example, cascading deletes (deleting child records when parent records are deleted): AR has a way to do this, but you can just throw a clause onto the table in the DB and then let your DB handle it, and never worry again. Same with many kinds of constraints and validations.

But if you want hooks, they can be implemented in a very lightweight way using plain old functions or multimethods. At some point in the past I had a database library that called hooks at different times in the CRUD cycle, for example after-save or before-delete. They were simple multimethods dispatching on table names. This lets you extend them to your own tables as you like.

(defmulti before-delete (fn [x] (table-for x)))
(defmethod before-delete :default [& _]) ;; do nothing
(defn delete [x] (when (before-delete x) (db-delete! x) (after-delete x)))

Then later as an end user I could write:

(defmethod before-delete ::my_table [x] 
  (if (= (:id x) 1)
    (throw (Exception. "OH NO! ABORT!"))
    x))

Easy and extensible, and took a couple seconds to write. No OO in sight. Not as sophisticated as AR maybe, but sometimes simple is good enough.

Look at this library for another example of defining hooks.

Migrations

Carte has these. I haven't thought much about them, but versioning a database and slurping data into it doesn't seem beyond the realm of possibility for Clojure.

Polish

A lot of the good of AR comes from all the conventions for naming tables and naming columns, and all the convenience functions for capitalizing words and formatting dates and such. This has nothing to do with OO vs. non-OO; AR just has a lot of polish because a lot of time has gone into it. Maybe Clojure doesn't have an AR-class library for working with DB data yet, but give it some time.

So...

Instead of having an object that knows how to destroy itself, mutate itself, save itself, relate itself to other data, fetch itself, etc., instead you have data that's just data, and then you define functions that work on that data: saves it, destroys it, updates it in the DB, fetches it, relates it to other data. This is how Clojure operates on data in general, and data from a database is no different.

Foo.find(1).update_attributes(:bar => "quux").save!

=> (with-db (-> (fetch-one :foo :where {:id 1})
                (assoc :bar "quux")
                (save!)))

Foo.create!(:id => 1)

=> (with-db (save (in-table :foo {:id 1})))

Something like that. It's inside-out from the way objects work, but it provides the same functionality. But in Clojure you also get all the benefits of writing code in an FP kind of way.

Answer 2

This question hasn't been answered in a while, but Korma http://sqlkorma.com is another project that also helps reduce the dissonance between SQL and Clojure. It's been worked on more recently and should work with more recent Clojure versions.