Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Improving performance of Racket Code and error when trying to byte compile

I hacked together several code snippets from various sources and created a crude implementation of a Wolfram Blog article at http://bit.ly/HWdUqK - for those that are mathematically inclined, it is very interesting!

Not surprisingly, given that I'm still a novice at Racket, the code takes too much time to calculate the results (>90 min versus 49 seconds for the author) and eats up a lot of memory. I suspect it is all about the definition (expListY) which needs to be reworked.

Although I have it working in DrRacket, I am also having problems byte-compiling the source, and still working on it (Error message: +: expects type <number> as 1st argument, given: #f; other arguments were: 1 -1)

Anybody want to take a stab at improving the performance and efficiency? I apologize for the unintelligible code and lack of better code comments.

PS: Should I be cutting and pasting the code directly here?

like image 244
lifebalance Avatar asked Apr 15 '12 12:04

lifebalance


3 Answers

Probably similar to soegaard's solution, except this one rolls its own "parser", so it's self contained. It produces the complete 100-year listing in a bit under 6 seconds on my machine. There's a bunch of tricks that this code uses, but it's not really something that would be called "optimized" in any serious way: I'm sure that it can be made much faster with some memoization, care for maximizing tree sharing etc etc. But for such a small domain it's not worth the effort... (Same goes for the quality of this code...)

BTW#1, more than parsing, the original solution(s) use eval which does not make things faster... For things like this it's usually better to write the "evaluator" manually. BTW#2, this doesn't mean that Racket is faster than Mathematica -- I'm sure that the solution in that post makes it grind redundant cpu cycles too, and a similar solution would be faster.

#lang racket

(define (tuples list n)
  (let loop ([n n])
    (if (zero? n)
      '(())
      (for*/list ([y (in-list (loop (sub1 n)))] [x (in-list list)])
        (cons x y)))))

(define precedence
  (let ([t (make-hasheq)])
    (for ([ops '((#f) (+ -) (* /) (||))] [n (in-naturals)])
      (for ([op ops]) (hash-set! t op n)))
    t))

(define (do op x y)
  (case op
    [(+) (+ x y)] [(-) (- x y)] [(*) (* x y)] [(/) (/ x y)]
    [(||) (+ (* 10 x) y)]))

(define (run ops nums)
  (unless (= (add1 (length ops)) (length nums)) (error "poof"))
  (let loop ([nums     (cddr nums)]
             [ops      (cdr ops)]
             [numstack (list (cadr nums) (car nums))]
             [opstack  (list (car ops))])
    (if (and (null? ops) (null? opstack))
      (car numstack)
      (let ([op    (and (pair? ops) (car ops))]
            [topop (and (pair? opstack) (car opstack))])
        (if (> (hash-ref precedence op)
               (hash-ref precedence topop))
          (loop (cdr nums)
                (cdr ops)
                (cons (car nums) numstack)
                (cons op opstack))
          (loop nums
                ops
                (cons (do topop (cadr numstack) (car numstack))
                      (cddr numstack))
                (cdr opstack)))))))

(define (expr ops* nums*)
  (define ops  (map symbol->string ops*))
  (define nums (map number->string nums*))
  (string-append* (cons (car nums) (append-map list ops (cdr nums)))))

(define nums  (for/list ([i (in-range 10 0 -1)]) i))
(define year1 2012)
(define nyears 100)
(define year2 (+ year1 nyears))
(define years (make-vector nyears '()))
(for ([ops (in-list (tuples '(+ - * / ||) 9))])
  (define r (run ops nums))
  (when (and (integer? r) (<= year1 r) (< r year2))
    (vector-set! years (- r year1)
                 (cons ops (vector-ref years (- r year1))))))

(for ([solutions (in-vector years)] [year (in-range year1 year2)])
  (if (pair? solutions)
    (printf "~a = ~a~a\n"
            year (expr (car solutions) nums)
            (if (null? (cdr solutions))
              ""
              (format " (~a more)" (length (cdr solutions)))))
    (printf "~a: no combination!\n" year)))
like image 95
Eli Barzilay Avatar answered Sep 21 '22 06:09

Eli Barzilay


Below is my implementation. I tweaked and optimized a thing or two in your code, in my laptop it takes around 35 minutes to finish (certainly an improvement!) I found that the evaluation of expressions is the real performance killer - if it weren't for the calls to the procedure to-expression, the program would finish in under a minute.

I guess that in programming languages that natively use infix notation the evaluation would be much faster, but in Scheme the cost for parsing and then evaluating a string with an infix expression is just too much.

Maybe someone can point out a suitable replacement for the soegaard/infix package? or alternatively, a way to directly evaluate an infix expression list that takes into account operator precedence, say '(1 + 3 - 4 & 7) - where & stands for number concatenation and has the highest precedence (for example: 4 & 7 = 47), and the other arithmetic operators (+, -, *, /) follow the usual precedence rules.

#lang at-exp racket

(require (planet soegaard/infix)
         (planet soegaard/infix/parser))

(define (product lst1 lst2) 
  (for*/list ([x (in-list lst1)] 
              [y (in-list lst2)]) 
    (cons x y))) 

(define (tuples lst n)
  (if (zero? n)
      '(())
      (product lst (tuples lst (sub1 n)))))

(define (riffle numbers ops)
  (if (null? ops)
      (list (car numbers))
      (cons (car numbers)
            (cons (car ops)
                  (riffle (cdr numbers)
                          (cdr ops))))))

(define (expression-string numbers optuple)
  (apply string-append
         (riffle numbers optuple)))

(define (to-expression exp-str)
  (eval
   (parse-expression
    #'here (open-input-string exp-str))))

(define (make-all-combinations numbers ops)
  (let loop ((opts (tuples ops (sub1 (length numbers))))
             (acc '()))
    (if (null? opts)
        acc
        (let ((exp-str (expression-string numbers (car opts))))
          (loop (cdr opts)
                (cons (cons exp-str (to-expression exp-str)) acc))))))

(define (show-n-expressions all-combinations years)
  (for-each (lambda (year)
              (for-each (lambda (comb)
                          (when (= (cdr comb) year)
                            (printf "~s ~a~n" year (car comb))))
                        all-combinations)
              (printf "~n"))
            years))

Use it like this for replicating the results in the original blog post:

(define numbers '("10" "9" "8" "7" "6" "5" "4" "3" "2" "1"))
(define ops '("" "+" "-" "*" "/"))
; beware: this takes around 35 minutes to finish in my laptop
(define all-combinations (make-all-combinations numbers ops))
(show-n-expressions all-combinations
                    (build-list 5 (lambda (n) (+ n 2012))))

UPDATE :

I snarfed Eli Barzilay's expression evaluator and plugged it into my solution, now the pre-calculation of all combinations is done in around 5 seconds! The show-n-expressions procedure still needs some work to avoid iterating over the whole list of combinations each time, but that's left as an exercise for the reader. What matters is that now brute-forcing the values for all the possible expression combinations is blazing fast.

#lang racket

(define (tuples lst n)
  (if (zero? n)
      '(())
      (for*/list ((y (in-list (tuples lst (sub1 n))))
                  (x (in-list lst)))
        (cons x y))))

(define (riffle numbers ops)
  (if (null? ops)
      (list (car numbers))
      (cons (car numbers)
            (cons (car ops)
                  (riffle (cdr numbers)
                          (cdr ops))))))

(define (expression-string numbers optuple)
  (string-append*
   (map (lambda (x)
          (cond ((eq? x '&) "")
                ((symbol? x) (symbol->string x))
                ((number? x) (number->string x))))
        (riffle numbers optuple))))

(define eval-ops
  (let ((precedence (make-hasheq
                     '((& . 3) (/ . 2) (* . 2)
                       (- . 1) (+ . 1) (#f . 0))))
        (apply-op   (lambda (op x y)
                      (case op
                        ((+) (+ x y)) ((-) (- x y))
                        ((*) (* x y)) ((/) (/ x y))
                        ((&) (+ (* 10 x) y))))))
    (lambda (nums ops)
      (let loop ((nums     (cddr nums))
                 (ops      (cdr ops))
                 (numstack (list (cadr nums) (car nums)))
                 (opstack  (list (car ops))))
        (if (and (null? ops) (null? opstack))
            (car numstack)
            (let ((op    (and (pair? ops) (car ops)))
                  (topop (and (pair? opstack) (car opstack))))
              (if (> (hash-ref precedence op)
                     (hash-ref precedence topop))
                  (loop (cdr nums)
                        (cdr ops)
                        (cons (car nums) numstack)
                        (cons op opstack))
                  (loop nums
                        ops
                        (cons (apply-op topop (cadr numstack) (car numstack))
                              (cddr numstack))
                        (cdr opstack)))))))))

(define (make-all-combinations numbers ops)
  (foldl (lambda (optuple tail)
           (cons (cons (eval-ops numbers optuple) optuple) tail))
         empty (tuples ops (sub1 (length numbers)))))

(define (show-n-expressions all-combinations numbers years)
  (for-each (lambda (year)
              (for-each (lambda (comb)
                          (when (= (car comb) year)
                            (printf "~s ~a~n"
                                    year
                                    (expression-string numbers (cdr comb)))))
                        all-combinations)
              (printf "~n"))
            years))

Use it like this:

(define numbers '(10 9 8 7 6 5 4 3 2 1))
(define ops '(& + - * /))
; this is very fast now!
(define all-combinations (make-all-combinations numbers ops))
(show-n-expressions all-combinations numbers
                    (build-list 5 (lambda (n) (+ n 2012))))
like image 34
Óscar López Avatar answered Sep 23 '22 06:09

Óscar López


As Óscar points out, the problem is that soegaard/infix is slow for this type of problem.

I found a standard shunting-yard parser for infix expressions on GitHub and wrote the following program in Racket:

#lang racket
(require "infix-calc.scm")

(define operators '("*" "/" "+" "-" ""))
(time
(for*/list ([o1  (in-list operators)]
            [o2  (in-list operators)]
            [o3  (in-list operators)]
            [o4  (in-list operators)]
            [o5  (in-list operators)]
            [o6  (in-list operators)]
            [o7  (in-list operators)]
            [o8  (in-list operators)]
            [o9  (in-list operators)]
            [expr (in-value
                  (apply string-append
                        (list "1" o1 "2" o2 "3" o3 "4" o4 "5" o5 "6" o6 "7" o7 "8" o8 "9" o9 "10")))]
             #:when (= (first (calc expr)) 2012))
 expr))

After a little less than 3 minutes the results are:

Welcome to DrRacket, version 5.2.900.2--2012-03-29(8c22c6c/a) [3m].
Language: racket; memory limit: 128 MB.
cpu time: 144768 real time: 148818 gc time: 25252
'("1*2*3+4*567*8/9-10"
  "1*2+34*56+7+89+10"
  "1*23+45*6*7+89+10"
  "1+2+3/4*5*67*8+9-10"
  "1+2+3+4*567*8/9-10"
  "1+2+34*56+7+8+9*10"
  "1+23+45*6*7+8+9*10"
  "1-2+345*6-7*8+9-10"
  "12*34*5+6+7*8-9*10"
  "12*34*5+6-7-8-9-10"
  "1234+5-6+789-10")

The infix parser was written by Andrew Levenson. The parser and the above code can be found here:

https://github.com/soegaard/Scheme-Infix-Calculator

like image 41
soegaard Avatar answered Sep 20 '22 06:09

soegaard