How can I use TensorFlow without CUDA on Linux?

Question

I have two computers without CUDA: one runs on Microsoft Windows, the other one runs on Linux (Ubuntu 14.04 64bit / Linux 3.13.0-100-generic))

I can use TensorFlow without CUDA on Microsoft Windows without any issue: TensorFlow uses the CPU. However, if on the Linux machine I run in python import tensorflow as tf, then TensorFlow fails to get imported due to CUDA being not installed:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 72, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 61, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#import_error

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

How can I use TensorFlow without CUDA on Linux?

I use tensorflow-gpu==1.0.0.

---

I'm aware of the parameter device_count in tensorflow.ConfigProto, which allows to disable the GPU, e.g.:

import tensorflow as tf

a = tf.constant(1, name = 'a')
b = tf.constant(3, name = 'b')
c = tf.constant(9, name = 'c')
d = tf.add(a, b, name='d')
e = tf.add(d, c, name='e')

config = tf.ConfigProto(device_count={'CPU': 1, 'GPU': 0})
sess = tf.Session(config=config)
print(sess.run([d, e]))

but it doesn't help since import tensorflow as tf is the issue.

I also know how to install CUDA 8:

# Install Nvidia drivers, CUDA and CUDA toolkit, following some instructions from http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-ubuntu1404-8-0-local-ga2_8.0.61-1_amd64-deb
sudo dpkg -i cuda-repo-ubuntu1404-8-0-local-ga2_8.0.61-1_amd64-deb
sudo apt-get update
sudo apt-get install cuda

but would prefer to avoid it on these two machines.

Answer 1

If you build the binary with --config=cuda (as was done for tensorflow-gpu) then your machine must have GPU drivers. You can install GPU drivers on the machine even if the machine doesn't have GPU which is the common practical solution.

What happens is that --config=cuda sets GOOGLE_CUDA macro in the code which changes behavior during runtime. In particular it causes dso_loader to run which you can see by following line printed

2017-02-16 17:15:08: I tensorflow/stream_executor/dso_loader.cc:125] successfully opened CUDA library libcurand.8.0.dylib locally

A popular approach at Google is to deploy a "fat" binary -- ie binary that bundles drivers and client code for all possible hardware accelerators because that simplifies testing and deployment.

In open-source release, drivers and client code are separate, but this pattern remains -- the GPU-capable binary is expecting to have GPU drivers available.