Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use only one GPU for tensorflow session?

I have two GPUs. My program uses TensorRT and Tensorflow.

When I run only TensorRT part, it is fine. When I run together with Tensorflow part, I have error as

[TensorRT] ERROR: engine.cpp (370) - Cuda Error in ~ExecutionContext: 77 (an illegal memory access was encountered)
terminate called after throwing an instance of 'nvinfer1::CudaError'
  what():  std::exception

The issue is when Tensorflow session starts as follow

self.graph = tf.get_default_graph()
self.persistent_sess = tf.Session(graph=self.graph, config=tf_config)

It loads two GPUs as

2019-06-06 14:15:04.420265: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6965 MB memory) -> physical GPU (device: 0, name: Quadro P4000, pci bus id: 0000:04:00.0, compute capability: 6.1)
2019-06-06 14:15:04.420713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 7159 MB memory) -> physical GPU (device: 1, name: Quadro P4000, pci bus id: 0000:05:00.0, compute capability: 6.1)

I tried to load only one GPU as

(1)Putting on top of the python code

import os
os.environ["CUDA_VISIBLE_DEVICES"]="0"

(2)

with tf.device('/device:GPU:0'):
    self.graph = tf.get_default_graph()
    self.persistent_sess = tf.Session(graph=self.graph, config=tf_config)

Both don't work.

How to solve the problem?

like image 527
batuman Avatar asked Jun 06 '19 06:06

batuman


People also ask

How do I run TensorFlow on multiple GPUs?

TensorFlow code, and tf.keras models will transparently run on a single GPU with no code changes required. Note: Use tf.config.list_physical_devices ('GPU') to confirm that TensorFlow is using the GPU. The simplest way to run on multiple GPUs, on one or many machines, is using Distribution Strategies.

Why does TensorFlow give priority to GPU over CPU?

If a TensorFlow operation has both CPU and GPU implementations, by default the GPU devices will be given priority when the operation is assigned to a device. For example, tf.matmul has both CPU and GPU kernels.

Should I use TensorFlow or keras for session config?

Tensorflow/Keras also allows to specify gpu to be used with session config. I can recommend it only if setting environment variable is not an options (i.e. an MPI run). Because it tend to be the least reliable of all methods, especially with keras.

What is TensorFlow strategy?

tf.distribute.Strategy is a TensorFlow API to distribute training across multiple GPUs, multiple machines, or TPUs. Using this API, you can distribute your existing models and training code with minimal code changes. tf.distribute.Strategy has been designed with these key goals in mind:


1 Answers

I could manage to load only one GPU be placing the following lines at the first line of the python code.

import sys, os
os.environ["CUDA_VISIBLE_DEVICES"]="0"
like image 189
batuman Avatar answered Nov 10 '22 01:11

batuman