Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Golang: Convert to JSON.GZ and write to file

Tags:

json

gzip

go

Trying to accomplish the following output with my data:

  1. Convert to JSON string and write to file: output.json (this part is working)
  2. Gzip Compress the JSON string and write that to a json.gz file: output.json.gz (NOT WORKING)

The code runs fine and writes to both files. But the gzipped file gives this error when I try to unzip it: Data error in 'output.json'. File is broken

Here's the code:

package main

import (
    "bytes"
    "compress/gzip"
    "encoding/json"
    "fmt"
    "io/ioutil"
)

type Generic struct {
    Name string
    Cool bool
    Rank int
}

func main() {
    generic := Generic{"Golang", true, 100}
    fileJson, _ := json.Marshal(generic)
    err := ioutil.WriteFile("output.json", fileJson, 0644)
    if err != nil {
        fmt.Printf("WriteFileJson ERROR: %+v", err)
    }

    var fileGZ bytes.Buffer
    zipper := gzip.NewWriter(&fileGZ)
    defer zipper.Close()
    _, err = zipper.Write([]byte(string(fileJson)))
    if err != nil {
        fmt.Printf("zipper.Write ERROR: %+v", err)
    }
    err = ioutil.WriteFile("output.json.gz", []byte(fileGZ.String()), 0644)
    if err != nil {
        fmt.Printf("WriteFileGZ ERROR: %+v", err)
    }
}

What did I miss?

like image 583
Joe Bergevin Avatar asked Aug 18 '14 21:08

Joe Bergevin


1 Answers

You need to call zipper.Close() immediately after finishing writing

http://play.golang.org/p/xNeMg3aXxO

_, err = zipper.Write(fileJson)
if err != nil {
    log.Fatalf("zipper.Write ERROR: %+v", err)
}
err := zipper.Close() // call it explicitly and check error 

Calling defer zipper.Close() would trigger the call at the end of the main function. Until you call .Close() the data is being written to an intermediate buffer and not flushed to the actual file.

like image 70
fabrizioM Avatar answered Oct 02 '22 14:10

fabrizioM