How to build the resultant docker image file directory?

Question

I have the following Dockerfile:

FROM busybox
COPY ./manifest.json /usr/manifest.json
RUN rm /usr/manifest.json

where manifest.json is just some random file I found..

When I build an image from this Dockerfile, there will be 3 layers as expected.. And when I inspect these 3 layers, I will see the file hierarchies as expected.. So the first one will be pretty lengthy like..

bin/acpid
bin/add-shell
bin/addgroup
bin/adduser
bin/adjtimex
bin/ar
bin/arch
bin/arp
bin/arping
bin/ash
... many more

Second layer will be:

usr/manifest.json

And the third layer will be:

proc/.wh..wh..opq
sys/.wh..wh..opq
usr/.wh.manifest.json

I am guessing .wh.manifest.json means remove this file when applying this layer? (Not sure, just guessing..)

My ultimate goal is to actually create the folder structure that the resulting image will have. So for the above case it should essentially be equal to the first layer since first I am adding a file then deleting it.

I could not find any documentation on what .wh stands for.. Does it make sense to iterate through layers like this and keep adding files (to some target, does not matter for now) and deleting the added files in case they are found with a .wh prefix? Or am I totally off and is there a much better way?

If anyone is interested, I have some Java code that downloads and inspects existing layers for an image:

package com.docker.poc;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.zip.GZIPInputStream;

import com.google.gson.Gson;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
import org.apache.commons.io.IOUtils;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class Main
{
  static class TokenResponse
  {
    String access_token;
  }

  static class Layer
  {
    String digest;

    long size;
  }

  static class ManifestResponse
  {
    List<Layer> layers;
  }

  static String imageName = "your-image-name";

  static String repoName = "your-repo-name";

  static String base64EncodedUsernamePassword = "not-gonna-tell-ya";

  public static void main(String[] args) throws Exception {
    try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
      HttpGet httpget = new HttpGet(
          "https://auth.docker.io/token?service=registry.docker.io&scope=repository:" + repoName + "/" + imageName +
              ":pull");
      httpget.setHeader("Authorization", "Basic " + base64EncodedUsernamePassword);

      CloseableHttpResponse response = httpclient.execute(httpget);
      String text = IOUtils.toString(response.getEntity().getContent(), StandardCharsets.UTF_8.name());
      String token = new Gson().fromJson(text, TokenResponse.class).access_token;

      httpget = new HttpGet("https://registry-1.docker.io/v2/" + repoName + "/" + imageName + "/manifests/latest");
      httpget.setHeader("Authorization", "Bearer " + token);
      httpget.setHeader("Accept", "application/vnd.docker.distribution.manifest.v2+json");

      response = httpclient.execute(httpget);
      text = IOUtils.toString(response.getEntity().getContent(), StandardCharsets.UTF_8.name());
      ManifestResponse manifestResponse = new Gson().fromJson(text, ManifestResponse.class);

      List<File> files = new ArrayList<>();

      int counter = 100;

      for (Layer layer : manifestResponse.layers) {
        System.out.println("Downloading layer: " + layer.digest);
        System.out.println("Layer size:" + layer.size + " bytes.(" + ((double) layer.size) / 1_000_000 + " MB)");
        httpget =
            new HttpGet("https://registry-1.docker.io/v2/" + repoName + "/" + imageName + "/blobs/" + layer.digest);
        httpget.setHeader("Authorization", "Bearer " + token);
        httpget.setHeader("Accept", "application/vnd.docker.distribution.manifest.v2+json");

        response = httpclient.execute(httpget);
        File targetFile = new File(counter + "_" + layer.digest.substring(7, 17) + ".tar");
        counter++;
        files.add(targetFile);

        InputStream inputStream = response.getEntity().getContent();
        OutputStream outputStream = new FileOutputStream(targetFile);
        IOUtils.copy(inputStream, outputStream);
        outputStream.close();
      }

      for (File file : files) {
        TarArchiveInputStream tarInput = new TarArchiveInputStream(new GZIPInputStream(new FileInputStream(file)));
        TarArchiveEntry entry;
        while (null != (entry = tarInput.getNextTarEntry())) {
          System.out.println(entry.getName());
        }
        System.out.println("==========");
      }
    }
  }
}

Answer 1

I understand you want to basically just extract the files and directory structure of a docker image, and only the final result of them applying all layers.

The easiest way to do this is to just use the standard docker commands and let docker do all the heavy lifting, so you don't have to bother with the internal details yourself.

For this docker provides the docker export command which exports a container as a tar archive. To create a container for your image without starting it (to not modify any files) you can use the docker create command:

# create container from image without starting it
docker create --name my_container my_image

# export container file system to tar archive
docker export -o my_image.tar my_container

# or directly extract the archive in the current folder
docker export my_container | tar -x

# clean up temporary container
docker rm my_container