How to crop and rotate an image to bounding box?

Question

• I have a dataset of thousands of images containing hands
• I also have .mat files which contain the coordinates of 4 corners of the bounding box

• However, the edges of these bounding boxes are at an angle with the x & y axis. For example,

• I want to crop out the hands using the bounding box coordinates & then rotate the hands such that they are aligned with the x or y axis.

EDIT:

The hand is represented as follows:

However, please keep in mind that the rectangle is NOT straight. So, I'll have to rotate it to straighten it out.

Answer 1

Good one!

First step:

Compute the size of the rectangle

 width = sqrt( sum( (b-a).^2 ) );
 height = sqrt( sum( (c-b).^2 ) );

Second step:

Compute an affine transformation from a...d to an upright image

 Xin = [a(2) b(2) c(2) d(2)];
 Yin = [a(1) b(1) c(1) d(1)];
 Xout = [width 1 1 width];
 Yout = [1 1 height height];
 A = [Xin;Yin;ones(1,4)]';
 B = [Xout; Yout]';
 H = B \ A; % affine transformation

Note that despite the fact that we allow fo H to be affine, the choise of corners (depending on width and height) will acertain that H will not distort the cropped rectangle.

optionally use cp2tform:

 H2 = cp2tform( [Xin;Yin]', [Xout;Yout]', 'nonreflectivesimilarity' );

Third step

Use the transformation to get the relevant image part

 thumb = tformarray( img, maketform( 'affine', H' ), ... %//'
                     makeresampler( 'cubic', 'fill' ), ...
                     1:2, 1:2, ceil( [height width] ), [], 0 );

optionally use imtransform:

 thumb = imtransform( img, H2, 'bicubic' );

---

A note regarding vectorization:

depends on how the coordinates of the corners are stored (a...d) the first two steps can be easily vectorize.