Golang - Docker API - parse result of ImagePull

Question

I'm developing a Go script that uses the Docker API for the purposes of my project. After I login to my repository, I pull the Docker image I want, but the problem is that the ImagePull function returns an instance of io.ReadCloser, which I'm only able to pass to the system output via:

io.Copy(os.Stdout, pullResp)

It's cool that I can see the response, but I can't find a decent way to parse it and implement a logic depending on it, which will do some things if a new version of the image have been downloaded, and other things if the image was up to date.
I'll be glad if you share your experience, if you have ever faced this problem.

Answer 1

You can import github.com/docker/docker/pkg/jsonmessage and use both JSONMessage and JSONProgress to decode the stream but it's easier to call DisplayJSONMessagesToStream: it both parses the stream and displays the messages as text. Here's how you can display the messages using stderr:

    reader, err := cli.ImagePull(ctx, myImageRef, types.ImagePullOptions{})
    if err != nil {
            return err
    }
    defer reader.Close()

    termFd, isTerm := term.GetFdInfo(os.Stderr)
    jsonmessage.DisplayJSONMessagesStream(reader, os.Stderr, termFd, isTerm, nil)

The nice thing is that it adapts to the output: it updates the lines if this a TTY (the way docker pull does) but it doesn't if the output is redirected to a file.

Answer 2

@radoslav-stoyanov before use my example do

# docker rmi busybox

then run code

package main

import (
    "encoding/json"
    "fmt"
    "github.com/docker/distribution/context"
    docker "github.com/docker/engine-api/client"
    "github.com/docker/engine-api/types"
    "io"
    "strings"
)

func main() {
    // DOCKER
    cli, err := docker.NewClient("unix:///var/run/docker.sock", "v1.28", nil, map[string]string{"User-Agent": "engine-api-cli-1.0"})
    if err != nil {
        panic(err)
    }

    imageName := "busybox:latest"

    events, err := cli.ImagePull(context.Background(), imageName, types.ImagePullOptions{})
    if err != nil {
        panic(err)
    }

    d := json.NewDecoder(events)

    type Event struct {
        Status         string `json:"status"`
        Error          string `json:"error"`
        Progress       string `json:"progress"`
        ProgressDetail struct {
            Current int `json:"current"`
            Total   int `json:"total"`
        } `json:"progressDetail"`
    }

    var event *Event
    for {
        if err := d.Decode(&event); err != nil {
            if err == io.EOF {
                break
            }

            panic(err)
        }

        fmt.Printf("EVENT: %+v\n", event)
    }

    // Latest event for new image
    // EVENT: {Status:Status: Downloaded newer image for busybox:latest Error: Progress:[==================================================>]  699.2kB/699.2kB ProgressDetail:{Current:699243 Total:699243}}
    // Latest event for up-to-date image
    // EVENT: {Status:Status: Image is up to date for busybox:latest Error: Progress: ProgressDetail:{Current:0 Total:0}}
    if event != nil {
        if strings.Contains(event.Status, fmt.Sprintf("Downloaded newer image for %s", imageName)) {
            // new
            fmt.Println("new")
        }

        if strings.Contains(event.Status, fmt.Sprintf("Image is up to date for %s", imageName)) {
            // up-to-date
            fmt.Println("up-to-date")
        }
    }
}

You can see API formats to create your structures (like my Event) to read them here https://docs.docker.com/engine/api/v1.27/#operation/ImageCreate

I hope it helps you solve your problem, thanks.

Answer 3

I have used similar approach for my purpose (not a moby client). Typically idea is same for reading stream response. Give it a try and implement yours.

Reading stream response of any response type:

reader := bufio.NewReader(pullResp)
defer pullResp.Close()  // pullResp is io.ReadCloser
var resp bytes.Buffer
for {
    line, err := reader.ReadBytes('\n')
    if err != nil {
        // it could be EOF or read error
        // handle it
        break
    }
    resp.Write(line)
    resp.WriteByte('\n')
}

// print it
fmt.Println(resp.String())

---

However your sample response in the comment seems valid JSON structure. The json.Decoder is best way to read JSON stream. This is just an idea-

type ImagePullResponse struct {
   ID             string `json"id"`
   Status         string `json:"status"`
   ProgressDetail struct {
     Current int64 `json:"current"`
     Total   int64 `json:"total"`
   } `json:"progressDetail"`
   Progress string `json:"progress"`
}

And do

d := json.NewDecoder(pullResp)
for {
   var pullResult ImagePullResponse
  if err := d.Decode(&pullResult); err != nil {
    // handle the error
    break
  }
  fmt.Println(pullResult)
}