Bilinear upsample in tensorflow?

Question

I want to do a simple bilinear resize (not necessarily by an integer factor) in TensorFlow. For example, starting from a (32,3,64,64) tensor, I would like a (32,3,96,96) tensor, where each 64x64 has been rescaled by a factor of 1.5 using bilinear interpolation. What is the best way to do that?

I would like this to support arbitrary factors > 1, not just 1.5 specifically.

Note: the operation on each 64x64 would be the same as what skimage.transform.rescale (scale=1.5, order=1) does.

Answer 1

I would suggest not to use any of the tf.image.resize_* functions as they suffer from a nasty bug that will not be fixed.

A new, different set of image resampling functions is apparently in the pipeline. In the meantime, you can find some examples on the web on how to do that yourself using e.g. transposed convolutions. It is unfortunately much less efficient that per-channel upsampling, but correct is better than fast.

EDIT

They finally fixed this bug in TF 2.0:

• image.resize now considers proper pixel centers (...).

This fix currently does not pass gradient through, which is... a bug that will hopefully also get fixed.

Answer 2

tf.image.resize_images should do what you need. It accepts both 3d (single image) and 4d (batch of images) tensors, with arbitrary depth (number of channels). So this should hopefully work:

# it's height, width in TF - not width, height
new_height = int(round(old_height * scale))
new_width = int(round(old_width * scale))
resized = tf.image.resize_images(input_tensor, [new_height, new_width])

Bilinear interpolation is the default so you don't need to specify it. You could also use resize_bilinear directly.