What does the -j option do for docker-php-ext-install?

Question

I have read a lot of examples for creating a dockerfile for your specific setup of PHP including installing extensions. The docker-php-ext-install command sometimes includes -j$(nproc) as an option. What exactly is happening there? I suspect the nproc has something to do with the number of processes?

Here's an example from https://github.com/docker-library/docs/tree/master/php#php-core-extensions

FROM php:7.0-fpm
RUN apt-get update && apt-get install -y \
        libfreetype6-dev \
        libjpeg62-turbo-dev \
        libmcrypt-dev \
        libpng-dev \
    && docker-php-ext-install -j$(nproc) iconv mcrypt \
    && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ \
    && docker-php-ext-install -j$(nproc) gd

I'm new to docker and want to understand exactly what's happening in each step rather than blindly copying and pasting things from various examples and tutorials.

Answer 1

It is the number of Jobs for the make calls contained inside the script docker-php-ext-install (line 53 stores the option in the variable $j and 105-106 call make -j$j).

The command nproc is giving directly to the script the number of physical thread available for your system. For example, on my system it will be reduced to:

make -j$(nproc) -> make -j8

thus it runs make with 8 parallel recipes.

From make manual:

-j [jobs], --jobs[=jobs]: Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.

with more information in the GNU make documentation about parallel jobs:

GNU make knows how to execute several recipes at once. Normally, make will execute only one recipe at a time, waiting for it to finish before executing the next. However, the -j or --jobs option tells make to execute many recipes simultaneously. [...] On MS-DOS, the -j option has no effect, since that system doesn’t support multi-processing.

If the -j option is followed by an integer, this is the number of recipes to execute at once; this is called the number of job slots. If there is nothing looking like an integer after the -j option, there is no limit on the number of job slots. The default number of job slots is one, which means serial execution (one thing at a time).

Ideally, if that number is equal to the number of the physical threads available (roughly the number of processors, or as in this case the number returned by nproc), you should get the fastest compilation possible.

nproc is a linux command - see man nproc too. It means "number of processing units available" - your CPU Cores. You can imagine it as "Number of processors" [source: @tron5 comment]

You must consider the memory available though. For example, if you allocate 8 slots with only 1GB of RAM and the compilation of 3 simultaneous jobs fill the RAM, then when the fourth will start it will exit with an error due to insufficient memory, arresting the whole compilation process.