C++ Tensorflow, how to make session->Run() with multithread, or spend less time

Question

I run all the below on CPU. I run the sample ensorflow/examples/label_image spends 7~8 seconds. As I know, python spends about 0.5 second to process the same sample and this is because "The TensorFlow Session object is multithreaded, so multiple threads can easily use the same session and run ops in parallel. ". BUT, how can I set multithread on Session with C++.

What I try ... I hard code line 81 in "tensorflow/tensorflow/core/common_runtime/direct_session.cc": "const int32 num_threads = 16;" However, it does not work.

How can I set some configure or what should I do something?

Answer 1

How about usiung multiple threads?

std::vector<std::thread> threads;

for (std::size_t i = 0; i < 10; ++i) {
   threads.push_back(std::thread([&]{ session->Run(); })); 
}


for (std::size_t i = 0; i < 10; ++i) {
   threads[i].join();
}