How to automatically install required packages from a python script as necessary?

Question

Is there anything in python or linux what basically instructs the system to "install whatever is necessary". Basically I find it annoying to install python packages for each new script/system/server that I work on. Each time I end up doing a sudo pip or an apt-get or dnf anyway. Why not automate that within the script itself. Whereever a 'no package found' error crops up, pass the library name to the install statement. Is this there ?

PS: I know docker exists, but am talking at a python/script level or a direct system level for purely execution purposes.

Thanks

Answer 1

How to automatically install required packages from a python script as necessary?

Let's assume that your Python script is example.py:

import os
import time
import sys
import fnmatch
import requests
import urllib.request
from bs4 import BeautifulSoup
from multiprocessing.dummy import Pool as ThreadPool 
print('test')

You can use pipreqs to automatically generate a requirements.txt file based on the import statements that the Python script(s) contain. To use pipreqs, assuming that you are in the directory where example.py is located:

pip install pipreqs
pipreqs .

It will generate the following requirements.txt file:

requests==2.23.0
beautifulsoup4==4.9.1

which you can install with:

pip install -r requirements.txt

Answer 2

The best way I know is, create a file requirements.txt list out all the packages name in it and install it using pip

pip install -r requirements.txt

Example requirements.txt:

BeautifulSoup==3.2.0
Django==1.3
Fabric==1.2.0
Jinja2==2.5.5
PyYAML==3.09
Pygments==1.4
SQLAlchemy==0.7.1
South==0.7.3
amqplib==0.6.1
anyjson==0.3
...

Answer 3

You can use setuptools to install dependencies automatically when you install your custom project on a new machine. Requirements file works just fine if all you want to do is to install a few PyPI packages.

Here is a nice comparison between the two. From the same link you can see that if your project has two dependent packages A and B, all you have to include in your setp.py file is a line

install_requires=[
   'A',
   'B'
] 

Of course, setuptools can do much more. You can include setups for external libraries (say C files), non PyPI dependencies, etc. The documentation gives a detailed overview on installing dependencies. There is also a really good tutorial on getting started with python packaging.

From their example, a typical setup.py file would look like this.

from setuptools import setup

setup(name='funniest',
      version='0.1',
      description='The funniest joke in the world',
      url='http://github.com/storborg/funniest',
      author='Flying Circus',
      author_email='[email protected]',
      license='MIT',
      packages=['funniest'],
      install_requires=[
          'markdown',
      ],
      zip_safe=False)

In conclusion, it is so simple to get started with setuptools. This package can make it fairly easy to migrate your code to a new machine.