I read What is the difference between 'SAME' and 'VALID' padding in tf.nn.max_pool of tensorflow? but this is not true to my experiment.
import tensorflow as tf
inputs = tf.random_normal([1, 64, 64, 3])
print(inputs.shape)
conv = tf.keras.layers.Conv2D(6, 4, strides=2, padding='same')
outputs = conv(inputs)
print(outputs.shape)
produces
(1, 64, 64, 3)
(1, 32, 32, 6)
. However following the above link produces (1, 31, 31, 6)
because there is no extra values outside filter ranges without any padding.
How does tf.keras.layers.Conv2D with padding='same' and strides > 1 behave?
I want to know the exact answer and its evidence.
With "SAME" padding, if you use a stride of 1, the layer's outputs will have the same spatial dimensions as its inputs.
The padding type is called SAME because the output size is the same as the input size(when stride=1). Using 'SAME' ensures that the filter is applied to all the elements of the input. Normally, padding is set to "SAME" while training the model. Output size is mathematically convenient for further computation.
padding. The padding parameter of the Keras Conv2D class can take one of two values: 'valid' or 'same'. Setting the value to “valid” parameter means that the input volume is not zero-padded and the spatial dimensions are allowed to reduce via the natural application of convolution.
If you set padding='same' then Keras will compute and add the required padding to the input image so the spatial size of the output is the same as the input. If you set padding='valid' (the default), then no padding is added and the output is smaller than the input.
Keras uses TensorFlow implementation of padding. All the details are available in the documentation here
First, consider the 'SAME' padding scheme. A detailed explanation of the reasoning behind it is given in these notes. Here, we summarize the mechanics of this padding scheme. When using 'SAME', the output height and width are computed as:
out_height = ceil(float(in_height) / float(strides[1])) out_width = ceil(float(in_width) / float(strides[2]))
The total padding applied along the height and width is computed as:
if (in_height % strides[1] == 0): pad_along_height = max(filter_height - strides[1], 0) else: pad_along_height = max(filter_height - (in_height % strides[1]), 0) if (in_width % strides[2] == 0): pad_along_width = max(filter_width - strides[2], 0) else: pad_along_width = max(filter_width - (in_width % strides[2]), 0)
Finally, the padding on the top, bottom, left and right are:
pad_top = pad_along_height // 2 pad_bottom = pad_along_height - pad_top pad_left = pad_along_width // 2 pad_right = pad_along_width - pad_left
Note that the division by 2 means that there might be cases when the padding on both sides (top vs bottom, right vs left) are off by one. In this case, the bottom and right sides always get the one additional padded pixel. For example, when pad_along_height is 5, we pad 2 pixels at the top and 3 pixels at the bottom. Note that this is different from existing libraries such as cuDNN and Caffe, which explicitly specify the number of padded pixels and always pad the same number of pixels on both sides.
For the 'VALID' scheme, the output height and width are computed as:
out_height = ceil(float(in_height - filter_height + 1) / float(strides[1])) out_width = ceil(float(in_width - filter_width + 1) / float(strides[2]))
and no padding is used.
In tensorflow, for stride s
and input size n
, padding with same gives:
⌈n/s⌉
or the ceiling of input size divided by stride.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With