Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting `write too long` error when trying to create tar.gz file from file and directories

Tags:

People also ask

How do I change a directory to a tar file?

To change the working directory in the middle of a list of file names, either on the command line or in a file specified using ' --files-from ' (' -T '), use ' --directory ' (' -C '). This will change the working directory to the specified directory after that point in the list.

Can 7zip create tar gz?

The trick is that 7-Zip will only gzip a single file. So creating a tar. gz is a two step process. First create the tar archive, then use 7-Zip to select the tar and you will get an option to gzip it.


So I'm trying to crate a tar.gz file file from multiple directories and files. Something with the same usage as:

tar -cvzf sometarfile.tar.gz somedir/ someotherdir/ somefile.json somefile.xml

Assuming the directories have other directories inside of them. I have this as an input:

    paths := []string{
      "somedir/",
      "someotherdir/",
      "somefile.json",
      "somefile.xml",
    }

and using these:

    func TarFilesDirs(paths []string, tarFilePath string ) error {
       // set up the output file
       file, err := os.Create(tarFilePath)
       if err != nil {
           return err
       }

       defer file.Close()
       // set up the gzip writer
       gz := gzip.NewWriter(file)
       defer gz.Close()

       tw := tar.NewWriter(gz)
       defer tw.Close()

       // add each file/dir as needed into the current tar archive
       for _,i := range paths {
          if err := tarit(i, tw); err != nil {
               return err
          }
       }

       return nil
   }

func tarit(source string, tw *tar.Writer) error {
    info, err := os.Stat(source)
    if err != nil {
        return nil
    }

    var baseDir string
    if info.IsDir() {
        baseDir = filepath.Base(source)
    }

    return filepath.Walk(source,
        func(path string, info os.FileInfo, err error) error {
            if err != nil {
                return err
            }

            header, err := tar.FileInfoHeader(info, info.Name())
            if err != nil {
                return err
            }

            if baseDir != "" {
                header.Name = filepath.Join(baseDir, strings.TrimPrefix(path, source))
            }

            if err := tw.WriteHeader(header); err != nil {
                return err
            }

            if info.IsDir() {
                return nil
            }

            file, err := os.Open(path)
            if err != nil {
                return err
            }

            defer file.Close()

            _, err = io.Copy(tw, file)
            if err != nil {
                log.Println("failing here")
                return err
            }

            return err
        })
}

Problem: if a directory is large I'm getting:

archive/tar: write too long

error, when I remove it everything works.

Ran out of ideas and wasted many hours on this trying to find a solution...

Any ideas?

Thanks