Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Find the source code for computing size of a docker image

I have heard the number is not equal to all sizes of layers adding together inside an image. And it is also not the size of disk space it occupies.

Now I want to check the logic by source code (in this repo: https://github.com/docker/docker-ce), because seeing is believing! But after navigating the code for a lot of time, I found that I was not able to find the real imag-size-computing code.

So which function/file is the docker used to perform the size logic?

like image 514
Wallace Avatar asked Apr 23 '20 15:04

Wallace


People also ask

Can I get source code from docker image?

Hi, if you write a compiled application (C, C++, go, …), the source code will not be part of the docker image, and they cannot access source code. But if you are using python, PHP, shell, for sure, if they get your docker image, they can access everything.

How do I find the size of a docker image?

To view the approximate size of a running container, you can use the command docker container ls -s . Running docker image ls shows the sizes of your images. To see the size of the intermediate images that make up your image use docker image history my_image:my_tag .

What does docker image ls command do?

DESCRIPTION. This command lists the images stored in the local Docker repository. By default, intermediate images, used during builds, are not listed.

How do I get the size of a docker image?

If the image resides in a registry, you can retrieve the size of the image without pulling it locally. To do this you can utilize the docker manifest command. At the time this article was written, the docker manifest command is in experimental and must be enabled.

What is a dockerfile and how to create one?

A Dockerfile is a text document that contains the instructions to assemble a Docker image. When we tell Docker to build our image by executing the docker build command, Docker reads these instructions, executes them, and creates a Docker image as a result. Let’s walk through the process of creating a Dockerfile for our application.

What does uncompressed size mean in Docker?

Uncompressed size - This is often referred to as the size on disk. This affects how much local storage is required to support your Docker workloads. The example commands shown below will work on Windows, MacOS, and Linux. Measuring the compressed size of a Docker image depends on where the image resides - in a Docker registry or locally.

What affects the first run experience of Docker images?

This affects how fast/slow images can be pulled from a registry. This impacts the first run experience on machines where images are not cached. Uncompressed size - This is often referred to as the size on disk. This affects how much local storage is required to support your Docker workloads.


1 Answers

Before digging too deep, you may find it useful to understand how Linux implements the overlay filesystem. I include a bit on this the first exercise of my intro presentation's build section. The demo notes include each of the commands I'm running and it gives you an idea of how layers are merged, and what happens when you add/modify/delete from a layer.


This is implementation dependent, based on your host OS and the graph driver being used. I'm taking the example of a Linux OS and Overlay2 since that's the most common use case.

It starts by looking at the image layer storage size:

// GetContainerLayerSize returns the real size & virtual size of the container.
func (i *ImageService) GetContainerLayerSize(containerID string) (int64, int64) {
    var (
        sizeRw, sizeRootfs int64
        err                error
    )

    // Safe to index by runtime.GOOS as Unix hosts don't support multiple
    // container operating systems.
    rwlayer, err := i.layerStores[runtime.GOOS].GetRWLayer(containerID)
    if err != nil {
        logrus.Errorf("Failed to compute size of container rootfs %v: %v", containerID, err)
        return sizeRw, sizeRootfs
    }
    defer i.layerStores[runtime.GOOS].ReleaseRWLayer(rwlayer)

    sizeRw, err = rwlayer.Size()
    if err != nil {
        logrus.Errorf("Driver %s couldn't return diff size of container %s: %s",
            i.layerStores[runtime.GOOS].DriverName(), containerID, err)
        // FIXME: GetSize should return an error. Not changing it now in case
        // there is a side-effect.
        sizeRw = -1
    }

    if parent := rwlayer.Parent(); parent != nil {
        sizeRootfs, err = parent.Size()
        if err != nil {
            sizeRootfs = -1
        } else if sizeRw != -1 {
            sizeRootfs += sizeRw
        }
    }
    return sizeRw, sizeRootfs
}

In there is a call to layerStores which itself is a mapping to layer.Store:

// ImageServiceConfig is the configuration used to create a new ImageService
type ImageServiceConfig struct {
    ContainerStore            containerStore
    DistributionMetadataStore metadata.Store
    EventsService             *daemonevents.Events
    ImageStore                image.Store
    LayerStores               map[string]layer.Store
    MaxConcurrentDownloads    int
    MaxConcurrentUploads      int
    MaxDownloadAttempts       int
    ReferenceStore            dockerreference.Store
    RegistryService           registry.Service
    TrustKey                  libtrust.PrivateKey
}

Digging into the layer.Store implementation for GetRWLayer, there is the following definition:

func (ls *layerStore) GetRWLayer(id string) (RWLayer, error) {
    ls.locker.Lock(id)
    defer ls.locker.Unlock(id)

    ls.mountL.Lock()
    mount := ls.mounts[id]
    ls.mountL.Unlock()
    if mount == nil {
        return nil, ErrMountDoesNotExist
    }

    return mount.getReference(), nil
}

Following that to find the Size implementation for the mount reference, there is this function that gets into the specific graph driver:

func (ml *mountedLayer) Size() (int64, error) {
    return ml.layerStore.driver.DiffSize(ml.mountID, ml.cacheParent())
}

Looking at the overlay2 graph driver to find the DiffSize function:

func (d *Driver) DiffSize(id, parent string) (size int64, err error) {
    if useNaiveDiff(d.home) || !d.isParent(id, parent) {
        return d.naiveDiff.DiffSize(id, parent)
    }
    return directory.Size(context.TODO(), d.getDiffPath(id))
}

That is calling naiveDiff which implements Size in the graphDriver package:

func (gdw *NaiveDiffDriver) DiffSize(id, parent string) (size int64, err error) {
    driver := gdw.ProtoDriver

    changes, err := gdw.Changes(id, parent)
    if err != nil {
        return
    }

    layerFs, err := driver.Get(id, "")
    if err != nil {
        return
    }
    defer driver.Put(id)

    return archive.ChangesSize(layerFs.Path(), changes), nil
}

Following archive.ChangeSize we can see this implementation:

// ChangesSize calculates the size in bytes of the provided changes, based on newDir.
func ChangesSize(newDir string, changes []Change) int64 {
    var (
        size int64
        sf   = make(map[uint64]struct{})
    )
    for _, change := range changes {
        if change.Kind == ChangeModify || change.Kind == ChangeAdd {
            file := filepath.Join(newDir, change.Path)
            fileInfo, err := os.Lstat(file)
            if err != nil {
                logrus.Errorf("Can not stat %q: %s", file, err)
                continue
            }

            if fileInfo != nil && !fileInfo.IsDir() {
                if hasHardlinks(fileInfo) {
                    inode := getIno(fileInfo)
                    if _, ok := sf[inode]; !ok {
                        size += fileInfo.Size()
                        sf[inode] = struct{}{}
                    }
                } else {
                    size += fileInfo.Size()
                }
            }
        }
    }
    return size
}

At which point we are using os.Lstat to return a struct that includes Size on each entry that is an add or modify to each directory. Note that this is one of several possible paths the code takes, but I believe it's one of the more common ones for this scenario.

like image 117
BMitch Avatar answered Oct 21 '22 17:10

BMitch