Set weight and bias tensors of tensorflow conv2d operation

Question

I have been given a trained neural network in torch and I need to rebuild it exactly in tensorflow. I believe I have correctly defined the network's architecture in tensorflow but I am having trouble transferring the weight and bias tensors. Using a third party package, I converted all the weight and bias tensors from the torch network to numpy arrays then wrote them to disk. I can load them back into my python program but I cannot figure out a way to assign them to the corresponding layers in my tensorflow network.

For instance, I have a convolution layer defined in tensorflow as

kernel_1 = tf.Variable(tf.truncated_normal([11,11,3,64], stddev=0.1))
conv_kernel_1 = tf.nn.conv2d(input, kernel_1, [1,4,4,1], padding='SAME')
biases_1 = tf.Variable(tf.zeros[64])
bias_layer_1 = tf.nn_add(conv_kernel_1, biases_1)

According to the tensorflow documentation, the tf.nn.conv2d operation uses the shape defined in the kernel_1 variable to construct the weight tensor. However, I cannot figure out how to access that weight tensor to set it to the weight array I have loaded from file.

Is it possible to explicitly set the weight tensor? And if so, how?

(The same question applies to bias tensor.)

Answer 1

If you have the weights and biases in a NumPy array, it should be easy to connect them into your TensorFlow network:

weights_1_array = ...  # ndarray of weights for layer 1
biases_1_array = ...  # ndarray of biases for layer 1

conv_kernel_1 = tf.nn.conv2d(input, weights_1_array, [1, 4, 4, 1], padding='SAME')
bias_layer_1 = tf.nn.bias_add(conv_kernel_1, biases_1_array)

Note that you must ensure that weights_1_array and biases_1_array are in the correct data format. See the documentation for tf.nn.conv2d() for an explanation of the required filter shape.