I am writing a project that relies on tensorflow
, but that can be provided by either of two pip
packages: tensorflow
or tensorflow-gpu
. My project works fine with either, but I don't want people running it on a machine without gpu support to have to install the extra overhead, but I still want people running on machines with gpu support to be able to leverage that. Is there a way to mark in my requirements.txt
file that I require either tensorflow
or tensorflow-gpu
but not both?
EDIT:
In this specific case I should note that from the programmer's point of view, both tensorflow
and tensorflow-gpu
are identical, as they both provide a module tensorflow
which has the same functions/classes/methods etc., and only differ in that tensorflow-gpu
benefits from GPU acceleration. The problem that I am having is that if I put tensorflow
in requirements.txt
then in order to run with GPU acceleration, users would have to do pip install -r requirements.txt && pip uninstall tensorflow && pip install tensorflow-gpu
which is not ideal, and if I instead put tensorflow-gpu
in requirements.txt
, then it will require a bunch of unnecessary system libraries (CUDNN etc) and wont work out-of-the-box for non-gpu users.
EDIT AGAIN
As a work-around, I've decided to provide two different requirement files, requirements.txt
and requirements-gpu.txt
, both of which include a shared -r .requirements-core.txt
and add their respective version of tensorflow. That way people who want GPU support can pip install -r requirements-gpu.txt
but the standard pip install -r requirements.txt
will still work out-of-the box for everyone.
You cannot condition download packages with requirements.txt
, but you might do one of the following solutions:
1 - Install both packages tensorflow and tensorflow-gpu as dependencies and do a try / except to choose which package will actually be used, like:
tensorflow = null
try:
tensorflow = __import__("tensorflow-gpu")
tensorflow.operation_that_requires_gpu()
except:
tensorflow = __import__("tensorflow")
enter code here
2 - On your project you ask the client to pass the dependency direct to you:
def my_function_that_uses_tensorflow(tensorflow):
# do stuff
from my_module import my_function_that_uses_tensorflow
import tensorflow # or tensorflow = __import__("tensorflow-gpu")
my_function_that_uses_tensorflow(tensorflow)
3 - If tensorflow-gpu and tensorflow both install their package with the same tensorflow
name on your site-packages, then my suggestion is do a try /except as I said on option number 1, but don't include tensorflow-gpu or tensorflow as a dependency of your package (treat it as a "peer dependency" that the code using your package should include as their dependency in order to use it):
try:
import tensorflow
except:
raise ImportError('You need to include tensorflow or tensorflow-gpu as a dependency in order to use this package')
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With