How do I fix a dimension error in TensorFlow?

Question

I'm trying to apply the expert portion of the tutorial to my own data but I keep running into dimension errors. Here's the code leading up to the error.

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)

def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)

def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1], padding='SAME')

W_conv1 = weight_variable([1, 8, 1, 4])
b_conv1 = bias_variable([4])

x_image = tf.reshape(tf_in, [-1,2,8,1])

h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

And then when I try to run this command:

W_conv2 = weight_variable([1, 4, 4, 8])
b_conv2 = bias_variable([8])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

I get the following errors:

ValueError                                Traceback (most recent call last)
<ipython-input-41-7ab0d7765f8c> in <module>()
      3 
      4 h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
----> 5 h_pool2 = max_pool_2x2(h_conv2)

ValueError: ('filter must not be larger than the input: ', 'Filter: [', Dimension(2), 'x', Dimension(2), '] ', 'Input: [', Dimension(1), 'x', Dimension(4), '] ')

Just for some background information, the data that I'm dealing with is a CSV file where each row contains 10 features and 1 empty column that can be a 1 or a 0. What I'm trying to get is a probability in the empty column that the column will equal a 1.

Answer 1

You have to shape the input so it is compatible with both the training tensor and the output. If you input is length 1, your output should be length 1 (length is substituted for dimension).

When you're dealing with-

def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 1, 1, 1],
                    strides=[1, 1, 1, 1], padding='SAME')

Notice how I changed the strides and the ksize to [1, 1, 1, 1]. This will match an output to a 1 dimensional input and prevent errors down the road.

When you're defining your weight variable (see code below)-

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)

def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)

you're going to have to make the first 2 numbers conform to the feature tensor that you are using to train your model, the last two numbers will be the dimension of the predicted output (same as the dimension of the input).

W_conv1 = weight_variable([1, 10, 1, 1])
b_conv1 = bias_variable([1])

Notice the [1, 10, in the beginning which signifies that the feature tensor is going to be a 1x10 feature tensor; the last two numbers 1, 1] correspond to the dimensions of the input and output tensors/predictors.

When you reshape your x_foo tensor (I call it x_ [x prime]), you, for whatever reason, have to define it like so-

x_ = tf.reshape(x, [-1,1,10,1])

Notice the 1 and 10 in the middle- ...1,10,.... Once again, these numbers correspond to the dimension of your feature tensor.

For every bias variable, you choose the final number of the previously defined variable. For example, if W_conv1 = weight_variable([1, 10, 1, 1]) appears like so, you take the final number and put that into your bias variable so it can match the dimensions of the input. This is done like so- b_conv1 = bias_variable([1]).

If you need any more explanation please comment below.

Answer 2

The dimensions you are using for the filter are not matching the output of the hidden layer.

Let me see if I understood you: your input is composed of 8 features, and you want to reshape it into a 2x4 matrix, right?

The weights you created with weight_variable([1, 8, 1, 4]) expect a 1x8 input, in one channel, and produce a 1x8 output in 4 channels (or hidden units). The filter you are using sweeps the input in 2x2 squares. However, since the result of the weights is 1x8, they won't match.

You should reshape the input as

x_image = tf.reshape(tf_in, [-1,2,4,1])

Now, your input is actually 2x4 instead of 1x8. Then you need to change the weight shape to (2, 4, 1, hidden_units) to deal with a 2x4 output. It will also produce a 2x4 output, and the 2x2 filter now can be applied.

After that, the filter will match the output of the weights. Also note that you will have to change the shape of your second weight matrix to weight_variable([2, 4, hidden_units, hidden2_units])