Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

limitation on bytes.Buffer?

Tags:

go

I am trying to gzip a slice of bytes using the package "compress/gzip". I am writing to a bytes.Buffer and I am writing 45976 bytes, when I am trying to uncompress the content using a gzip.reader and then reader function - I find that the not all of the content is recovered. Is there some limitations to bytes.buffer? and is it a way to by pass or alter this? here is my code (edit):

func compress_and_uncompress() {
    var buf bytes.Buffer
    w := gzip.NewWriter(&buf)
    i,err := w.Write([]byte(long_string))
    if(err!=nil){
            log.Fatal(err)
    }
    w.Close()

    b2 := make([]byte, 80000)
    r, _ := gzip.NewReader(&buf)
    j, err := r.Read(b2)
    if(err!=nil){
            log.Fatal(err)
    }
    r.Close()

    fmt.Println("Wrote:", i, "Read:", j)
}

output from testing (with a chosen string as long_string) would give Wrote: 45976, Read 32768

like image 322
stian Avatar asked Dec 02 '25 04:12

stian


1 Answers

Continue reading to get the remaining 13208 bytes. The first read returns 32768 bytes, the second read returns 13208 bytes, and the third read returns zero bytes and EOF.

For example,

package main

import (
    "bytes"
    "compress/gzip"
    "fmt"
    "io"
    "log"
)

func compress_and_uncompress() {
    var buf bytes.Buffer
    w := gzip.NewWriter(&buf)
    i, err := w.Write([]byte(long_string))
    if err != nil {
        log.Fatal(err)
    }
    w.Close()

    b2 := make([]byte, 80000)
    r, _ := gzip.NewReader(&buf)
    j := 0
    for {
        n, err := r.Read(b2[:cap(b2)])
        b2 = b2[:n]
        j += n
        if err != nil {
            if err != io.EOF {
                log.Fatal(err)
            }
            if n == 0 {
                break
            }
        }
        fmt.Println(len(b2))
    }
    r.Close()

    fmt.Println("Wrote:", i, "Read:", j)
}

var long_string string

func main() {
    long_string = string(make([]byte, 45976))
    compress_and_uncompress()
}

Output:

32768
13208
Wrote: 45976 Read: 45976
like image 198
peterSO Avatar answered Dec 04 '25 20:12

peterSO