In-memory file for testing

Question

How does one create in-memory files for unit testing in Go?

In Python, I test reading from a file or writing to a file using io.BytesIO or io.StringIO. For example, to test a file parser, I would have

def test_parse_function():
    infile = io.StringIO('''\
line1
line2
line3
''')
    parsed_contents = parse_function(infile)
    expected_contents = ['line1', 'line2', 'line3']  # or whatever is appropriate
    assert parsed_contents == expected_contents

Similarly for file output, I would have something like the following:

def test_write_function():
    outfile = io.StringIO()
    write_function(outfile, ['line1', 'line2', 'line3'])
    outfile.seek(0)
    output = outfile.read()
    expected_output = '''\
line1
line2
line3
'''
    assert output == expected_output

Answer 1

You can use a Buffer.

In general, it's a good idea to use io.Reader and io.Writer interfaces in your code (Buffer implements both) to deal with IO. That way you can deal with various methods of input/output (local file, memory buffer, network connection...) in the same way, without knowing what you are dealing with in the specific function you're using. It makes it more abstract and makes testing trivial.

---

Example of use with a trivial function:

Function definition:

// mypkg project mypkg.go
package mypkg

import (
    "bufio"
    "io"
    "strings"
)

func MyFunction(in io.Reader, out io.Writer) {
    rd := bufio.NewReader(in)
    str, _ := rd.ReadString('\n')
    io.WriteString(out, strings.TrimSuffix(str, "\n")+" was input\n")
}

Function use within a program:

package main

import (
    "mypkg"
    "os"
)

func main() {
    mypkg.MyFunction(os.Stdin, os.Stdout)
}

Test:

// mypkg project mypkg_test.go
package mypkg

import (
    "bytes"
    "testing"
)

func TestMyFunction(t *testing.T) {
    ibuf := bytes.NewBufferString("hello\n")
    obuf := bytes.NewBufferString("")
    MyFunction(ibuf, obuf)
    if obuf.String() != "hello was input\n" {
        t.Fail()
    }
}

Answer 2

As mentioned in "Pragmatic and Effective Testing in Go", you can use spf13/afero for file abstraction which leads to easier tests.

That library has a Using Afero for Testing:

There is a large benefit to using a mock filesystem for testing.

It has a completely blank state every time it is initialized and can be easily reproducible regardless of OS.
You could create files to your heart’s content and the file access would be fast while also saving you from all the annoying issues with deleting temporary files, Windows file locking, etc.
The MemMapFs backend is perfect for testing.

• Much faster than performing I/O operations on disk
• Avoid security issues and permissions
• Far more control. 'rm -rf /' with confidence
• Test setup is far more easier to do
• No test cleanup needed

It uses a fully atomic memory backed filesystem (MemMapFs).
It is fully concurrent and will work within go routines safely.