Image deformations in TensorFlow

Question

Tensorflow 1.8 added utilities to apply deformations to images (see e.g. sparse_image_warp and dense_image_warp). This is great, but I've been unable to find any utilities to create the deformations.

As an example, I can randomly rotate an image according to:

radians = tf.random_uniform([], -1.0, 1.0)
img = tf.contrib.image.rotate(img, radians)

What would the equivalent code be to deform an image?

flow = ...  # your code here
img = tf.contrib.image.dense_image_warp(img, flow)

I'm aware that the flow could be generated in several ways, and I'm not terribly picky about which one is chosen as long as it does not rip the image appart (e.g. I'm looking for a diffeomorphism or at least a homeomorphism).

Answer 1

It seems you might want to look at this kaggle page on elastic transformations, which implements this article.

This is quite similar to my meshgrid+noise baseline although it seems to be done with more care, the examples presented show that you can perturb the image greatly without getting complete garbage, hope it helps.

EDIT: You can take a look at this new library that implements different image transformations including Grid distortion, which might provide an alternative to elastic transform for image warping