Stop pip from failing on single package when installing with requirements.txt

Question

Running each line with pip install may be a workaround.

cat requirements.txt | xargs -n 1 pip install

Note: -a parameter is not available under MacOS, so old cat is more portable.

This solution handles empty lines, whitespace lines, # comment lines, whitespace-then-# comment lines in your requirements.txt.

cat requirements.txt | sed -e '/^\s*#.*$/d' -e '/^\s*$/d' | xargs -n 1 pip install

Hat tip to this answer for the sed magic.

For Windows:

pip version >=18

import sys
from pip._internal import main as pip_main

def install(package):
    pip_main(['install', package])

if __name__ == '__main__':
    with open(sys.argv[1]) as f:
        for line in f:
            install(line)

pip version <18

import sys
import pip

def install(package):
    pip.main(['install', package])

if __name__ == '__main__':
    with open(sys.argv[1]) as f:
        for line in f:
            install(line)

For windows users, you can use this:

FOR /F %k in (requirements.txt) DO ( if NOT # == %k ( pip install %k ) )

Logic: for every dependency in file(requirements.txt), install them and ignore those start with "#".

The xargs solution works but can have portability issues (BSD/GNU) and/or be cumbersome if you have comments or blank lines in your requirements file.

As for the usecase where such a behavior would be required, I use for instance two separate requirement files, one which is only listing core dependencies that need to be always installed and another file with non-core dependencies that are in 90% of the cases not needed for most usecases. That would be an equivalent of the Recommends section of a debian package.

I use the following shell script (requires sed) to install optional dependencies:

#!/bin/sh

while read dependency; do
    dependency_stripped="$(echo "${dependency}" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
    # Skip comments
    if [[ $dependency_stripped == \#* ]]; then
        continue
    # Skip blank lines
    elif [ -z "$dependency_stripped" ]; then
        continue
    else
        if pip install "$dependency_stripped"; then
            echo "$dependency_stripped is installed"
        else
            echo "Could not install $dependency_stripped, skipping"
        fi
    fi
done < recommends.txt

For Windows using PowerShell:

foreach($line in Get-Content requirements.txt) {
    if(!($line.StartsWith('#'))){
        pip install $line
    }
}