How to compile a c++ application using static opencv libraries within docker

Question

I'm building my first OpenCV based application in C++. My goal is to build an intermediate docker image that can compile the application statically so that it can run standalone in the resulting smaller image. I'm open to using any docker image for this step, but just so that you can see exactly what I have, here's the dockerfile to reproduce the entire environment:

FROM ubuntu:18.04 as compiler

ENV OPENCV_VERSION='3.4.2' DEBIAN_FRONTEND=noninteractive

RUN apt-get -y update && \
    apt-get -y upgrade && \
    apt-get -y dist-upgrade && \
    apt-get -y autoremove && \
    apt-get install -y build-essential cmake 
RUN apt-get install -y qt5-default libvtk6-dev
RUN apt-get install -y zlib1g-dev libjpeg-dev libwebp-dev libpng-dev libtiff5-dev libopenexr-dev libgdal-dev
RUN apt-get install -y libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev
RUN apt-get install -y unzip wget
RUN wget --progress=dot:giga https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
    unzip -q ${OPENCV_VERSION}.zip && \
    rm ${OPENCV_VERSION}.zip && \
    mv opencv-${OPENCV_VERSION} OpenCV && \
    cd OpenCV && \
    mkdir build && \
    cd build && \
    cmake  \
        -D BUILD_SHARED_LIBS=OFF \ 
        -D WITH_QT=ON \
        -D WITH_OPENGL=ON \ 
        -D FORCE_VTK=ON \
        -D WITH_TBB=ON \ 
        -D WITH_GDAL=ON \ 
        -D WITH_XINE=ON \ 
        -D BUILD_EXAMPLES=OFF \
        -D ENABLE_PRECOMPILED_HEADERS=OFF \
        -D BUILD_DOCS=OFF \
        -D BUILD_PERF_TESTS=OFF \
        -D BUILD_TESTS=OFF \
        -D BUILD_opencv_apps=OFF \
        .. && \
    make -j4 && \
    make install && \
    ldconfig

COPY compile-test.cpp compile-test.cpp

RUN g++ -std=c++11 -static compile-test.cpp -o /app $(pkg-config --cflags --libs opencv)

I can currently compile my c++ apps without issue using the dyanmic libs, but this creates a massive docker image, and I really want to be able to build standalone binaries for distribution, with minimal size.

As you can see I'm compiling OpenCV from source including the flag BUILD_SHARED_LIBS=OFF to make sure I get the .a static libs, rather than the .so dynamic libs. I took a hint from a highly recommended build script, and modified it for use with docker omitting a few python things as I'm using c++.

Because I was having so much trouble with my real application, I've gone ahead and created a much simpler app, which also blows up during compilation. I believe this has something to do with the included cflags and libs. The problem is currently beyond my comprehension. I get mountains of errors that seem to change when I adjust a single include on my compile command. Here's the simplest app I'm trying to compile. It really doesn't do anything, but it does include a lib.

#include "opencv2/imgcodecs.hpp"
using namespace cv; 
Mat img;

int main( int argc, char** argv ) {
  img = cv::imread( argv[1], IMREAD_COLOR );
}

Then I try to compile this like so:

g++ -std=c++11 -static compile-test.cpp -o /app $(pkg-config --cflags --libs opencv)

And it ends up in a pile of errors much too long to completely paste here.

//usr/local/lib/libopencv_imgcodecs.a(grfmt_jpeg.cpp.o): In function `cv::JpegEncoder::write(cv::Mat const&, std::vector<int, std::allocator<int> > const&)':
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0xf8): undefined reference to `jpeg_CreateCompress'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x105): undefined reference to `jpeg_std_error'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2b5): undefined reference to `jpeg_set_defaults'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2d0): undefined reference to `jpeg_set_quality'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x2fe): undefined reference to `jpeg_quality_scaling'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x30d): undefined reference to `jpeg_quality_scaling'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x367): undefined reference to `jpeg_default_qtables'
grfmt_jpeg.cpp:(.text._ZN2cv11JpegEncoder5writeERKNS_3MatERKSt6vectorIiSaIiEE+0x379): undefined reference to `jpeg_start_compress'
grfmt_jpeg.cpp:
...
collect2: error: ld returned 1 exit status

Some Things I've Already Tried

• Beginning to google each of the seemingly unique compile errors and adding related flags to the end of my compile code.
• Reordering some of the include flags, but there are just too many to do this effectively
• Using the opencv-dev package instead of compiling it myself, but it seems you can't do this and expect to use static libs.

Answer 1

After a lot of experimentation I finally got something working! There were a few issues that are all fixed in this Dockerfile. In order to reproduce this, create a Dockerfile with the following contents, and create another file called app.cpp with the simple code from my question above, in the same folder.

I will explain what the issues were below:

FROM alpine:3.8 as compiler

RUN echo -e '@edgunity http://nl.alpinelinux.org/alpine/edge/community \
    @edge http://nl.alpinelinux.org/alpine/edge/main \
    @testing http://nl.alpinelinux.org/alpine/edge/testing \
    @community http://dl-cdn.alpinelinux.org/alpine/edge/community' \
    >> /etc/apk/repositories

RUN apk add --update --no-cache \
      build-base \
      openblas-dev \
      unzip \
      wget \
      cmake \
      g++ \
      libjpeg  \
      libjpeg-turbo-dev \
      libpng-dev \
      jasper-dev \
      tiff-dev \
      libwebp-dev \
      clang-dev \
      linux-headers 

ENV CC /usr/bin/clang
ENV CXX /usr/bin/g++
ENV OPENCV_VERSION='3.4.2' DEBIAN_FRONTEND=noninteractive

RUN mkdir /opt && cd /opt && \
  wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
  unzip ${OPENCV_VERSION}.zip && \
  rm -rf ${OPENCV_VERSION}.zip

RUN mkdir -p /opt/opencv-${OPENCV_VERSION}/build && \
  cd /opt/opencv-${OPENCV_VERSION}/build && \
  cmake \
    -D BUILD_DOCS=OFF \
    -D BUILD_EXAMPLES=OFF \
    -D BUILD_opencv_apps=OFF \
    -D BUILD_opencv_python2=OFF \
    -D BUILD_opencv_python3=OFF \
    -D BUILD_PERF_TESTS=OFF \
    -D BUILD_SHARED_LIBS=OFF \ 
    -D BUILD_TESTS=OFF \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D ENABLE_PRECOMPILED_HEADERS=OFF \
    -D FORCE_VTK=OFF \
    -D WITH_FFMPEG=OFF \
    -D WITH_GDAL=OFF \ 
    -D WITH_IPP=OFF \
    -D WITH_OPENEXR=OFF \
    -D WITH_OPENGL=OFF \ 
    -D WITH_QT=OFF \
    -D WITH_TBB=OFF \ 
    -D WITH_XINE=OFF \ 
    -D BUILD_JPEG=ON  \
    -D BUILD_TIFF=ON \
    -D BUILD_PNG=ON \
  .. && \
  make -j$(nproc) && \
  make install && \
  rm -rf /opt/opencv-${OPENCV_VERSION}

RUN wget --progress=dot:giga https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.0.0-linux-x86-64.tar.gz && \
    pwd && \
    tar -xzf libwebp-1.0.0-linux-x86-64.tar.gz && \
    mv /libwebp-1.0.0-linux-x86-64/lib/libwebp.a /usr/lib && \
    rm -rf /libwebp*

RUN wget --progress=dot:giga http://www.ece.uvic.ca/~frodo/jasper/software/jasper-2.0.10.tar.gz && \
    tar -xzf jasper-2.0.10.tar.gz && \
    cd jasper-2.0.10 && \
    mkdir BUILD && \
    cd BUILD && \
    cmake -DCMAKE_INSTALL_PREFIX=/usr    \
      -DCMAKE_BUILD_TYPE=Release     \
      -DCMAKE_SKIP_INSTALL_RPATH=YES \
      -DCMAKE_INSTALL_DOCDIR=/usr/share/doc/jasper-2.0.10 \
      -DJAS_ENABLE_SHARED=FALSE \
      ..  && \
    make install && \
    rm -rf /jasper-2.0.10*

ENV PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/lib/pkgconfig

COPY app.cpp app.cpp

RUN g++ -Wl,-Bstatic -static-libgcc -std=c++11 \
    app.cpp \ 
    -o /app \
    $(pkg-config --cflags --libs -static opencv) \
    -lgfortran -lquadmath

FROM alpine    
COPY --from=compiler /app /bin/app

Problems

Linker

There were indeed files that needed linking that weren't present, there were two reasons for this:

• The pkg-config command is supposed to emit all of the necessary flags for compilation, but in my earlier attempt I hadn't included the -static flag to pkg-config. When you add the -static flag it makes sure to link the extra required packages. I saw a few people run into this problem with the solution of adding the extra flags like -pthread, but I found that the -static flag did this for me and so was preferable.
• ld: cannot find -lgcc_s error. This appeared to be fixed by adding the -static-libgcc flag to g++. Some of this is still a mystery to me.

Missing Static Libraries

There were two libraries that I wanted to be included as static which needed to be acquired from sources other than apk. These were libjasper and libwebp. There are build steps above that acquire and build these as necessary and copy the resources into the required place.

More missing links

For reasons I can't yet explain pkg-config didn't provide the last two necessary flags. Those were -lgfortran and -lquadmath.

Notes about this Solution

I switched to alpine linux, just because I had read that some people had success with that, I'm sure the same could be done with Ubuntu. It did result in a much smaller image, so I do like that. This is about 900mb for the intermediate image, which, while huge, is much smaller than the 1.9GB Ubuntu image.

The actual resulting image is about 44mb including all of the statically linked OpenCV libs. This seems like a good solution for those that need a small docker image to run a single C++ bin.