How to use OpenGL to emulate OpenCV's warpPerspective functionality (perspective transform)

Question

I've done image warping using OpenCV in Python and C++, see the Coca Cola logo warped in place in the corners I had selected:

Using the following images:

and this:

Full album with transition pics and description here

I need to do exactly this, but in OpenGL. I'll have:

• Corners inside which I've to map the warped image

• A homography matrix that maps the transformation of the logo image into the logo image you see inside the final image (using OpenCV's warpPerspective), something like this:

  [[  2.59952324e+00,   3.33170976e-01,  -2.17014066e+02],
  [  8.64133587e-01,   1.82580111e+00,  -3.20053715e+02],
  [  2.78910149e-03,   4.47911310e-05,   1.00000000e+00]]
  

• Main image (the running track image here)

• Overlay image (the Coca Cola image here)

Is it possible ? I've read a lot and started OpenGL basics tutorials, but can it be done from just what I have? Would the OpenGL implementation be faster, say, around ~10ms?

I'm currently playing with this tutorial here: http://ogldev.atspace.co.uk/www/tutorial12/tutorial12.html Am I going in the right direction? Total OpenGL newbie here, please bear. Thanks.

Answer 1

After trying a number of solutions proposed here and elsewhere, I ended solving this by writing a fragment shader that replicates what 'warpPerspective' does.

The fragment shader code looks something like:

varying highp vec2 textureCoordinate;

uniform sampler2D inputImageTexture;

// NOTE: you will need to pass the INVERSE of the homography matrix, as well as 
// the width and height of your image as uniforms!
uniform highp mat3 inverseHomographyMatrix;
uniform highp float width;
uniform highp float height;

void main()
{
   // Texture coordinates will run [0,1],[0,1];
   // Convert to "real world" coordinates
   highp vec3 frameCoordinate = vec3(textureCoordinate.x * width, textureCoordinate.y * height, 1.0);

   // Determine what 'z' is
   highp vec3 m = inverseHomographyMatrix[2] * frameCoordinate;
   highp float zed = 1.0 / (m.x + m.y + m.z);
   frameCoordinate = frameCoordinate * zed;

   // Determine translated x and y coordinates
   highp float xTrans = inverseHomographyMatrix[0][0] * frameCoordinate.x + inverseHomographyMatrix[0][1] * frameCoordinate.y + inverseHomographyMatrix[0][2] * frameCoordinate.z;
   highp float yTrans = inverseHomographyMatrix[1][0] * frameCoordinate.x + inverseHomographyMatrix[1][1] * frameCoordinate.y + inverseHomographyMatrix[1][2] * frameCoordinate.z;

   // Normalize back to [0,1],[0,1] space
   highp vec2 coords = vec2(xTrans / width, yTrans / height);

   // Sample the texture if we're mapping within the image, otherwise set color to black
   if (coords.x >= 0.0 && coords.x <= 1.0 && coords.y >= 0.0 && coords.y <= 1.0) {
       gl_FragColor = texture2D(inputImageTexture, coords);
   } else {
       gl_FragColor = vec4(0.0,0.0,0.0,0.0);
   }
}

Note that the homography matrix we are passing in here is the INVERSE HOMOGRAPHY MATRIX! You have to invert the homography matrix that you would pass into 'warpPerspective'- otherwise this code will not work.

The vertex shader does nothing but pass through the coordinates:

// Vertex shader
attribute vec4 position;
attribute vec4 inputTextureCoordinate;

varying vec2 textureCoordinate;

void main() {
   // Nothing happens in the vertex shader
   textureCoordinate = inputTextureCoordinate.xy;
   gl_Position = position;
}

Pass in unaltered texture coordinates and position coordinates (i.e. textureCoordinates = [(0,0),(0,1),(1,0),(1,1)] and positionCoordinates = [(-1,-1),(-1,1),(1,-1),(1,1)], for a triangle strip), and this should work!