Why can't the set difference function compare lists with data from files?

Question

Here are my files and functions:

List1.txt => cat dog tiger bear

List2.txt => cat dog tiger

These files are ANSI encoded on winodws xp.

(defun get-file (filename)
  (with-open-file (stream filename)
  (loop for line = (read-line stream nil)
  while line collect line)))

(defparameter *file1* (get-file "list1.txt"))
(defparameter *file2* (get-file "list2.txt"))

(set-difference *file1* *file2*)

I would think the output would be just "bear". However, it returns ("cat", "dog", "tiger", "bear") as the difference. I am assuming it has to due with me reading the information in from files but I was stuck there. Thank you for your time.

Answer 1

Common Lisp uses EQL as default test. You want string-equal or similar.

CL-USER > (set-difference '("cat" "dog" "tiger" "bear")
                          '("cat" "dog" "tiger")
                          :test #'string-equal)
->
("bear")

Answer 2

Apart from the :test keyword parameter of set-difference which Rainer already mentioned, there is another problem in your code. Your function get-file returns a list of lines, where each line is just one string. You probably want to split the lines into words. You can do it easily with split-sequence function (google for the split-sequence Common Lisp package or get it with

(ql:quickload :split-sequence)

if you have quicklisp installed. So, your function could look like, for example:

(defun get-file (path)
  (with-open-file (s path)
    (let* ((len (file-length s))
           (data (make-string len)))
      (read-sequence data s)
      (mapcar #'(lambda (string) (string-trim '(#\Space #\Tab #\Newline) string))
        (split-sequence:split-sequence '#\Space data :remove-empty-subseqs t)))))


(defparameter *file1* (get-file "list1.txt"))
(defparameter *file2* (get-file "list2.txt"))

(format t "~s~%" (set-difference *file1* *file2* :test #'string-equal))

This will, correctly, print:

("bear")