Negation in z3 datalog for expressing optimality

Question

I use Z3 with :fixedpoint.engine set to datalog. I have an enumerated filter relation (f pos min max). Let us say that we have (f #x10 #x100000 #x200000), (f #x20 #x150000 #x200000) and (f #x20 #x300000 #x500000).

For a given x, I search the greatest pos such that (f pos min max) and min <= x <= max (Here I use intervals but filters can be arbitrarily complex). Priority (sort t) and values (sort s) are BitVectors but x, min and max are on a rather large space (eg. 24 bits).

; configuration 
(set-option :fixedpoint.engine datalog)
; sorts
(define-sort s () (_ BitVec 24))
(define-sort t () (_ BitVec 8))
; Relations
(declare-rel f (t s s))
(declare-rel match (t s))
(declare-rel better (t s))
(declare-rel best (t s))
(declare-rel a (t))
(declare-rel b ())
(declare-rel c ())
(declare-var x s)
(declare-var xmin s)
(declare-var xmax s)
(declare-var p t)
(declare-var q t)
; Facts (EDB)
(rule (f #x10 #x100000 #x200000))
(rule (f #x20 #x150000 #x200000))
(rule (f #x20 #x300000 #x500000))
; Rules
(rule (=> (and (f p xmin xmax) (bvule xmin x) (bvule x xmax))
          (match p x)))
(rule (=> (and (match q x) (bvugt q p))
          (better p x)))
(rule (=> (and (match p x) (not (better p x)))
          (best p x)))
; Queries
(rule (=> (match p #x170000) (a p)))
(rule (=> (better #x10 #x170000) b))
(rule (=> (best #x10 #x170000) c))
; output: sat
; (or (= (:var 0) #x20) (= (:var 0) #x10))
(query (a p) :print-answer true)
; output: sat
(query b)
; Output 'WARNING: creating large table of size 16777216 for relation better' and fails
(query c)

• (match p x) codes the fact that a filter at priority p filters x.
• (better p x) if a rule with a better priority than p filters x.
• (best p x) codes that the best filter matching x has priority p.

If I query (match p #x170000), I quickly get #x10 and #x20. If I ask (better #x10 #x170000) I quickly get an answer same for priority #20. But the query on (best p #x170000) fails to execute in reasonable time and reasonable space.

It seems that (not (better p x)) is computed independently of (match p x) and so is represented by a very large table (the possible values of x are not forwarded). In some cases I can restrict x by some tricks in better (sometimes I know that I am only interested by x that explicitly appear in other relations) so that the space is reduced but this is not a real generic solutions and sometimes I am stuck.

How should I rephrase the problem or which options should I use to avoid this problem ?

Answer 1

Z3's default Datalog tables are over concrete values, so if you use large bit-vectors, Z3 may end-up creating huge tables. You can try a simpler table data-structure, that supports fewer operations but it's sparse (uses "don't care" bits). You can try it out with: z3 fixedpoint.engine=datalog fixedpoint.datalog.default_relation=doc file.smt2