Equivalent of SQL "limit" clause in Datomic

Question

Title sort of says it all, but say I have a simple query as follows:

(q '[:find ?c ?n :where [?c :my-thing/its-attribute ?n]]
   (d/db conn))

against a schema like

[{:db/id (d/tempid :db.part/db)
  :db/ident :my-thing/its-attribute
  :db/valueType :db.type/string
  :db/doc "My thing's attribute"
  :db/cardinality :db.cardinality/one
  :db.install/_attribute :db.part/db}]

If the query matches everything (say, 100M entries), the returned results will be large. If I just want a few of them, what's the best way to do that?

Answer 1

Two random names using rand (duplicates tolerated) and sample (only distinct)

(d/q '[:find [(rand 2 ?name) (sample 2 ?name)]
       :where [_ :artist/name ?name]]
     db)

This example came from the day-of-datomic github repo.

Answer 2

This answer is sort of a compilation of that of @adamneilson and the comments on the original question. I was trying to accomplish the same thing as the OP, but wasn't quite able to find my answer here, so hopefully this'll help someone.

My use case was to pull 100k records with pagination. It was absolutely not feasible to simply use take/drop, as it took a really long time (tens of seconds).

My workaround was to first fetch the entity ids required, do take/drop on that collection, then map over them with entity. Here's my final code:

(defn eid->entity
  [eid]
  (into {} (d/touch (d/entity (d/db (get-conn)) eid))))

(defn find-eids
  [attr value limit offset]
  (let [query '[:find ?eid
                :in $ ?attr ?value
                :where [?eid ?attr ?value]]
        db (d/db (get-conn))
        result (drop offset (sort (d/q query db attr value)))]
    (map first (take limit result))))

(map eid->entity (find-eids :attr-name "value" 10 10)

This feels super wrong to my SQL-trained brain, but I think it's the datomic way. And it's not terribly slow - about 500ms for 100k records, which is good enough for me.

Answer 3

The equivalent of a SQL "limit" clause can be achieved with the key :limit when using a query-map.

 (d/q {:query '[:find ?c ?n :where [?c :my-thing/its-attribute ?n]] 
       :offset 1
       :limit 10
       :args [(d/db conn)]})

You can read more about this in the datomic client api documentation:

https://docs.datomic.com/client-api/datomic.client.api.html

or about the query-map syntax:

https://docs.datomic.com/on-prem/query.html#timeout