How can I execute a TensorFlow graph from a protobuf in C++?

Question

I got a simple code form tutorial and output it to .pb file as below:

mnist_softmax_train.py

x = tf.placeholder("float", shape=[None, 784], name='input_x')
y_ = tf.placeholder("float", shape=[None, 10], name='input_y')

W = tf.Variable(tf.zeros([784, 10]), name='W')
b = tf.Variable(tf.zeros([10]), name='b')
tf.initialize_all_variables().run()
y = tf.nn.softmax(tf.matmul(x,W)+b, name='softmax')

cross_entropy = -tf.reduce_sum(y_*tf.log(y))

train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy, name='train_step')
train_step.run(feed_dict={x:input_x, y_:input_y})

---

In C++, I load the same graph, and feed in fake data for testing:

Tensor input_x(DT_FLOAT, TensorShape({10,784}));
Tensor input_y(DT_FLOAT, TensorShape({10,10}));   
Tensor W(DT_FLOAT, TensorShape({784,10}));   
Tensor b(DT_FLOAT, TensorShape({10,10}));
Tensor input_test_x(DT_FLOAT, TensorShape({1,784}));

for(int i=0;i<10;i++){
    for(int j=0;j<10;j++)
        input_x.matrix<float>()(i,i+j) = 1.0;    

    input_y.matrix<float>()(i,i) = 1.0;
    input_test_x.matrix<float>()(0,i) = 1.0;
}

std::vector<std::pair<string, tensorflow::Tensor>> inputs = {
  { "input_x", input_x },
  { "input_y", input_y },
  { "W", W },
  { "b", b },
  { "input_test_x", input_test_x },
};

std::vector<tensorflow::Tensor> outputs;
status = session->Run(inputs, {}, {"train_step"}, &outputs);

std::cout << outputs[0].DebugString() << "\n";

However, this fails with the error:

Invalid argument: Input 0 of node train_step/update_W/ApplyGradientDescent was passed float from _recv_W_0:0 incompatible with expected float_ref.

The graph runs correctly in Python. How can I run it correctly in C++?

Answer 1

The issue here is that you are running the "train_step" target, which performs much more work than just inference. In particular, it attempts to update the variables W and b with the result of the gradient descent step. The error message

Invalid argument: Input 0 of node train_step/update_W/ApplyGradientDescent was passed float from _recv_W_0:0 incompatible with expected float_ref.

...means that one of the nodes you attempted to run ("train_step/update_W/ApplyGradientDescent") expected a mutable input (with type float_ref) but it got an immutable input (with type float) because the value was fed in.

There are (at least) two possible solutions:

1. If you only want to see predictions for a given input and given weights, fetch "softmax:0" instead of "train_step" in the call to Session::Run().

2. If you want to perform training in C++, do not feed W and b, but instead assign values to those variables, then continue to execute "train_step". You may find it easier to create a tf.train.Saver when you build the graph in Python, and then invoke the operations that it produces to save and restore values from a checkpoint.