WebSocket client in Common Lisp with usocket library

Question

I'm trying to upgrade a protocol, switching from HTTP 1.1 to WebSockets. I've tried to use the usocket. My code so far follows (and is available as a GitHub gist). After the handshake reading, functions return NIL or unexpected EOF error.

;; Define parameter sock for usocket stream
;; echo.websocket.org is a site for testing websockets
(defparameter sock (usocket:socket-connect "echo.websocket.org" 80))

;; Output confirms WebSocket protocol handshake as this implemented in browsers 
(format (usocket:socket-stream sock) "~A~%~A~%~A~%~A~%~A~A~%~A~%~A~%~%" 
        "GET /?encoding=text HTTP/1.1"
        "Connection: Upgrade"
        "Host: echo.websocket.org"
        "Origin: http://www.websocket.org"
        "Sec-WebSocket-Key: " (generate-websocket-key)
        "Sec-WebSocket-Version: 13"
        "Upgrade: websocket")

;; Write output to stream
(force-output (usocket:socket-stream sock))

;; Returns NIL
(do ((line                                                             
      (read-line (usocket:socket-stream sock) nil)                        
      (read-line (usocket:socket-stream sock) nil)))                      
    ((not line))                                                       
  (format t "~A" line))

;; Returns error: unexpected EOF
(read-line (usocket:socket-stream sock))

;; Returns NIL
(listen (usocket:socket-stream sock))

Answer 1

~% in a FORMAT statement is not portable for the purpose of HTTP protocols. It outputs the newline character #\newline. A newline depends on the platform the code runs on: cr (old Macs, Lisp Machines), crlf (Windows) or lf (Unix). Thus if you write a newline character to a stream, the output depends on the platform you are running on (or what the Lisp system thinks it should do). Windows has the same line end convention as HTTP.

Note:

• on a Unix system usually #\newline is the same as #\linefeed. A single character.
• on a Windows system often #\newline is the same as the sequence #\return #\linefeed. Two characters. Some Lisp systems might ignore this and use Unix conventions.

HTTP uses crlf. To reliably write crlf, you have to write the characters #\return and #\linefeed: (format stream "~a~a" #\return #\linefeed).

(defun write-crlf (stream)
  (write-char #\return stream)
  (write-char #\linefeed stream))

Some servers might be dumb enough to read input which does not follow the standard HTTP crlf conventions, though.

There might be ways to specify the line ending convention when opening a stream. usocket does not provide such a functionality, though.

I would also use finish-output (waits for completion) and not force-output (does NOT wait for completion of IO operations).