How to prevent transactions from violating application invariants in Datomic

Question

To elaborate, most relational databases have the idea of a database constraint. Here is the Postgres documentation on constraints. What tools does Datomic offer to constrain data or maintain some set of invariants on the data stored?

Answer 1

EDIT 2019-06-28: Since 0.9.5927 (Datomic On-Prem) / 480-8770 (Datomic Cloud), Datomic supports finer write-time validation via Attribute Predicates, Entity Specs and Entity Predicates. This makes most of the initial answer invalid or irrelevant.

In particular, observe that Entity Predicates accept a database value as a parameter, so they can actually enforce invariants that span several Entities.

---

By default, Datomic enforces only a very limited set of constraints regarding what data may be written, including mostly:

• uniqueness constraints: see Identity and Uniqueness
• type constraints, e.g you may not write a number to an attribute that is :db.type/string
• entity resolution: operations like [:db/add [:my.app/id "fjdsklfjsl"] :my.app/content 42] will fail if the [:my.app/id "fjdsklfjsl"] lookup-ref does not resolve to an existing entity
• conflicts, e.g Datomic won't let you :db/add 2 different values for the same entity-attribute pair if the attribute cardinality is one.

(I may be forgetting some, if so please comment.)

In particular at the time of writing, there is no built-in way to add custom validation or 'foreign-key' constraint to a given attribute.

However, combining Transaction Functions and speculative writes (a.k.a db.with()) gives you a powerful way of enforcing arbitrary invariants. For instance, you can wrap a transaction in a transaction function that applies the transaction speculatively using db.with(), then searches the speculative result for invariant violations, throwing an Exception if it finds some. You can even make this transaction function very generic by implementing the 'search invariant violations' part in Datalog.

Here's an example of what the API may look like:

[:myapp.fns/checking-invariants
 ;; a description of the invariant
 {:query
  [:find ?message ?user-id
   :in $db-before $db-after ?tx-data ?tempids ?user-id
   :where
   [$db-before ?user :myapp.user/id ?user-id]
   [$db-before ?user :myapp.user/email ?email-before]
   [$db-after ?user :myapp.user/email ?email-after]
   [(not= ?email-before ?email-after)]
   [(ground "A user may not change her email") ?message]]
  :inputs ["user-id-12342141"]}
 ;; the wrapped transaction
 [[:db/add 125315815291 :myapp.user/email "[email protected]"]
  [:db/add 125315815291 :myapp.user/name "Foo Bar"]]]

Here's an (untested) implementation of :myapp.fns/checking-invariants:

{:db/ident :myapp.fns/checking-invariants,
 :db/fn #db/fn{:lang :clojure,
               :imports [],
               :requires [[datomic.api :as d]],
               :params [db invariant-q tx],
               :code
               (let [{:keys [query inputs]} invariants-q
                     {:keys [db-before db-after tx-data tempids]}
                     (d/with db tx)]
                 (when-some [violations (apply d/q query
                                          db-before db-after tx-data tempids
                                          inputs)]
                   (throw (ex-info
                            "Transaction would violate invariants."
                            {:tx tx
                             :violations violations
                             :t (d/basis-t db-before)})))
                 tx)}}

Limitations:

• you can only protect externally: the client has to opt in to using this invariant-checking transaction function.
• be careful about performance - abusing this approach may put too much load on the Transactor. In cases where it is safe to do so, you may prefer to perform validation on the Peer using db.invoke()
• make sure your transaction is deterministic, as it will be run twice (more precisely, make sure that whether your transaction violates the invariant is deterministic)