How to create a bidirectional binary stream in Common Lisp?

Question

I read How to create a binary stream (not a file) in Common Lisp?, and it describes how to create a binary stream but not a bidirectional one. I've attempted to do it myself using the library referenced, but failed. My attempt looks like:

(defun make-test-binary-stream ()
  (make-two-way-stream (flexi-streams:make-in-memory-input-stream 
                         (vector))
   (flexi-streams:make-in-memory-output-stream)))

I use it like:

(let ((str (make-test-binary-stream))) 
   (lisp-binary:write-float :double 123.45d0 :stream str) 
   (lisp-binary:read-binary-type 'double-float str))

The result I'd expect there is 123.45d0, but instead it returns 0.0d0.

How could I create a binary stream that behaves as I expect, allowing me to write a value in then read the same value back? I want such a stream for testing the correctness of encoding and decoding functions that take streams as input and output to streams.

Answer 1

A two-way stream S is a couple (I,O) where I is an input stream and O an output stream. Both streams are not necessarily related, it just means that when you read from S, you read from I, and when you write to S, you write to O.

Here, you try to read from an in-memory stream that is backed by an empty sequence. The stream just gives the items in the sequence, but as a stream; here, the stream reaches end-of-file immediately.

Anonymous buffer

Not directly answering the question, but I sometimes use lisp-binary and the way I test is as follows:

(with-input-from-sequence 
    (in (with-output-to-sequence (out)
          (write-float :double 123.45d0 :stream out)))
  (read-binary-type 'double-float in))

Let's decompose:

(flex:with-output-to-sequence (out)
  (write-float :double 123.45d0 :stream out))

The above locally bind out to a stream that writes into a hidden, in-memory sequence, and eventually return that sequence. The whole expression returns a buffer of bytes:

#(205 204 204 204 204 220 94 64)

This buffer is given to with-input-from-sequence to bind in to a local stream that reads data from that sequence. read-binary-type uses that input stream to decode the value.

Temporary file

(defpackage :so (:use :cl :lisp-binary :flexi-streams :osicat))
(in-package :so)

The osicat system has a macro to open a temporary file in both input and output mode:

(with-temporary-file (io :element-type '(unsigned-byte 8))
  (write-float :double pi :stream io)
  (file-position io 0)
  (read-binary-type 'double-float io))

In memory circular buffer

I couldn't find an existing implementation of an in-memory i/o stream that reads and writes from/to the same vector; you can hack something with two flexi-streams that share an internal vector (but this is dangerous, there are cases where data consistency breaks), or built one from gray streams; see also cl-stream.