expand MNIST - elastic deformations MATLAB

Question

I want to augment the MNIST handwritten digits dataset.
In order to do that I want to create for each image an Elastic Deformations image respectively.

I read on this paper, section 2 'Expanding Data Sets through Elastic Distortions' that they accomplished elastic distortions

Before:

After:

I tried:

http://www.mathworks.com/help/images/ref/imwarp.html http://www.mathworks.com/help/images/examples/creating-a-gallery-of-transformed-images.html

without any success.

How can i do that in MATLAB?

How can I create Elastic Distortion transformation on an image in MATLAB?

Answer 1

I'm not sure I completely followed the "normalization" method of the displacement field, but I think this can get you quite close

img = imread('http://deeplearning.net/tutorial/_images/mnist_2.png');  %// get a digit

Compute a random displacement field dx~U(-1,1), dy~U(-1,1):

dx = -1+2*rand(size(img)); 
dy = -1+2*rand(size(img)); 

Smoothing and normalizing the field:

sig=4; 
alpha=60;
H=fspecial('gauss',[7 7], sig);
fdx=imfilter(dx,H);
fdy=imfilter(dy,H);
n=sum((fdx(:).^2+fdy(:).^2)); %// norm (?) not quite sure about this "norm"
fdx=alpha*fdx./n;
fdy=alpha*fdy./n;

The resulting displacement

[y x]=ndgrid(1:size(img,1),1:size(img,2));
figure;
imagesc(img); colormap gray; axis image; axis tight;
hold on;
quiver(x,y,fdx,fdy,0,'r');

The final stage - applying the displacement to the actual pixels using griddata interpolation:

new = griddata(x-fdx,y-fdy,double(img),x,y);
new(isnan(new))=0;

The resulting digit:

figure;
subplot(121); imagesc(img); axis image;
subplot(122); imagesc(new); axis image;
colormap gray

---

BTW, I'm not sure the proposed method (rand+imfilter) is the most straight-forward way of producing a random smooth deformation, you might consider sampling coefficients for 2-nd or 3-rd degree polynomial deformation,

dx = a*x.^2 + b*x.*y + c*y.^2 + d*x + e*y + f;
dy = g*x.^2 + h*x.*y + k*y.^2 + l*x + m*y + n;