Golang: append file to an existing tar archive

Question

How would I append a file to an existing tar archive in Go? I don't see anything obvious in the docs on how to do it.

I have a tar file that has already been created and I want to add more to it after it has already been closed.

EDIT

Altering the example in the docs and following the answer given, I'm still not getting the expected result. The first three files are being written to the tar but when I close and open up the file again to write to it, the new file is never being written. The code runs fine. I don't know what I'm missing.

The following code gives me a tar file with three files in it: readme.txt, gopher.txt, todo.txt. foo.bar never gets written.

package main

import (
    "archive/tar"
    "log"
    "os"
)

func main() {
    f, err := os.Create("/home/jeff/Desktop/test.tar")
    if err != nil {
        log.Fatalln(err)
    }

    tw := tar.NewWriter(f)

    var files = []struct {
        Name, Body string
    }{
        {"readme.txt", "This archive contains some text files."},
        {"gopher.txt", "Gopher names:\nGeorge\nGeoffrey\nGonzo"},
        {"todo.txt", "Get animal handling licence."},
    }
    for _, file := range files {
        hdr := &tar.Header{
            Name: file.Name,
            Size: int64(len(file.Body)),
        }
        if err := tw.WriteHeader(hdr); err != nil {
            log.Fatalln(err)
        }
        if _, err := tw.Write([]byte(file.Body)); err != nil {
            log.Fatalln(err)
        }
    }
    if err := tw.Close(); err != nil {
        log.Fatalln(err)
    }
    f.Close()

    // Open up the file and append more things to it

    f, err = os.OpenFile("/home/jeff/Desktop/test.tar", os.O_APPEND|os.O_WRONLY, os.ModePerm)
    if err != nil {
        log.Fatalln(err)
    }
    tw = tar.NewWriter(f)

    test := "this is a test"

    hdr := &tar.Header{
        Name: "foo.bar",
        Size: int64(len(test)),
    }

    if err := tw.WriteHeader(hdr); err != nil {
        log.Fatalln(err)
    }

    if _, err := tw.Write([]byte(test)); err != nil {
        log.Fatalln(err)
    }

    if err := tw.Close(); err != nil {
        log.Fatalln(err)
    }
    f.Close()

}

Answer 1

The tar file specification states:

A tar archive consists of a series of 512-byte records. Each file system object requires a header record which stores basic metadata (pathname, owner, permissions, etc.) and zero or more records containing any file data. The end of the archive is indicated by two records consisting entirely of zero bytes.

The Go implementation of adding these two zero filled records happens here .

To get around the tar file format trailer (basically 1024 bytes of nothing) you could replace the lines:

f, err = os.OpenFile("/home/jeff/Desktop/test.tar", os.O_APPEND|os.O_WRONLY, os.ModePerm)
if err != nil {
    log.Fatalln(err)
}
tw = tar.NewWriter(f)

With:

f, err = os.OpenFile("/home/jeff/Desktop/test.tar", os.O_RDWR, os.ModePerm)
if err != nil {
    log.Fatalln(err)
}
if _, err = f.Seek(-1024, os.SEEK_END); err != nil {
    log.Fatalln(err)
}
tw = tar.NewWriter(f)

It opens the file read / write (instead of append / write-only) and then seeks to 1024 bytes before the end of the file and writes from there.

It works, but it is a horrible hack.

EDIT: After understanding the tar file spec a little better, I no longer believe this is such a hack.

Full code: http://play.golang.org/p/0zRScmY4AC

Answer 2

It's just a writer interface so write bytes to it after writing your files header.

import (
  "archive/tar"
  "os"
)

f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, os.ModePerm)
if err != nil {
// handle error here
}

hdr := tar.Header{}
// populate your header
tw := tar.NewWriter(f)
// append a file
tw.WriteHeader(hdr)
tw.Write(content_of_file_as_bytes)

http://golang.org/pkg/archive/tar/#Writer tells you all you need to know.

EDIT: It turns out that tar files get a trailer written to the end when it's closed. So even though you are writing new data to the tar archive it won't be read past that trailer. So instead it looks like you'll have to read in the tar archive first and then rewrite the whole archive to disk which is suboptimal. The package doesn't support the necessary stuff to append to them though so that's the best I can recommend right now.