Image effects with rotation and pinch to zoom using GLSurfaceView Android

Question

I was developing an application to apply effects/rotation/pinch to zoom in-out functionality on image. I have downloaded demo application from https://github.com/Grishu/ImageEffects.

It works well, now my problems/questions are as below:

1. Apply multiple effects on images with progress change value (e.g. First apply brightness effect and result of this apply another effect say "contrast".)

   - problem: in code effect always apply on original image. So change in code to apply effect on final image like,

   if(isApplyEffectOnOriginalImage){//call first time only         
       mEffect.apply(mTextures[0], mImageWidth, mImageHeight, mTextures[1]);
   }else{
       mEffect.apply(mTextures[1], mImageWidth, mImageHeight, mTextures[1]);
   }
   

   Now, if apply brightness effect on progress change first time it works fine. But if I apply the same or other(contrast) effect, now I think on progress change effect apply on mTextures[1] and result set in mTextures[1], so is it possible to create temporary mTextures[2] to store mTextures[1] initially and when user change the progress value apply effect on mTextures[2] and set result on mTextures[1] like happen in if condition.

2. How to apply pinch to zoom on images

3. Rotation issue: I was rotating image at 90 angle clockwise, just by setting values(90,180,270, etc.), but the problem is when rotating image from 90 to 180, image rendering is not properly. see

A) 0 angle image

B) 90 angle image

C) 180 angle image

Answer 1

I've modified Grishu's demo project: here's a Screen Recording, then some explanation:

(Image rotated, with three effect applied - flip horizontal, crossprocess, fisheye)

1) For applying the effect on the result, your suggestion works fine. I don't really understand what you mean by ,,progress change effect''. Do you want to tune parameters of the effect?

2) To have gestures you should extend GLSurfaceView and implement GestureDetector.OnGestureListener and/or ScaleGestureDetector.OnScaleGestureListener depending on your needs. See TouchGLView here: Source, or the snippet below:

private class TouchGLView extends GLSurfaceView
        implements GestureDetector.OnGestureListener,
        ScaleGestureDetector.OnScaleGestureListener {
    private TextureRenderer mRenderer;
    private GestureDetector mTapDetector;
    private ScaleGestureDetector mScaleDetector;
    private float mLastSpan = 0;

    TouchGLView(Context c) {
        super(c);
        // Use Android's built-in gesture detectors to detect
        // which touch event the user is doing.
        mTapDetector = new GestureDetector(c, this);
        mTapDetector.setIsLongpressEnabled(false);
        mScaleDetector = new ScaleGestureDetector(c, this);

        // Create an OpenGL ES 2.0 context.
        setEGLContextClientVersion(2);
        mRenderer = new TextureRenderer(c);
        setRenderer(mRenderer);
    }
    @Override
    public boolean onTouchEvent(final MotionEvent e) {
        // Forward touch events to the gesture detectors.
        mScaleDetector.onTouchEvent(e);
        mTapDetector.onTouchEvent(e);
        return true;
    }
    @Override
    public boolean onScroll(MotionEvent e1, MotionEvent e2,
                            final float dx, final float dy) {
        // Forward the drag event to the renderer.
        queueEvent(new Runnable() {
            public void run() {
                mRenderer.drag(dx, dy);
            }});
        return true;
    }
    @Override
    public boolean onScale(ScaleGestureDetector detector) {
        // Forward the scale event to the renderer.
        final float amount = detector.getCurrentSpan() - mLastSpan;
        queueEvent(new Runnable() {
            public void run() {
                mRenderer.zoom(amount);
            }});
        mLastSpan = detector.getCurrentSpan();
        return true;
    }
    ...
}

3) You can rotate the image by modifying the vertex coordinates and taking viewports into account. Also, you can apply different rotations. You can rotate (and zoom) the view (vertex coordinates) itself (which does not affect the original texture buffer) or you can rotate pixels in the texture buffer (which is easy for 90°, 180°, etc) and then update your vertex coordinates to match the new image width/height.

Here's an example for manipulating with the vertex coords:

private void computeOutputVertices() {
    if (mPosVertices != null) {
        float imgAspectRatio = mTexWidth / (float)mTexHeight;
        float viewAspectRatio = mViewWidth / (float)mViewHeight;
        float x0, y0, x1, y1;
        // Set initial vertex coords based in texture aspect
        if (imgAspectRatio > 1.0f) {
            x0 = -1.0f ;
            y0 = -1.0f / imgAspectRatio;
            x1 = 1.0f ;
            y1 = 1.0f / imgAspectRatio;
        } else {
            x0 = -1.0f *imgAspectRatio;
            y0 = -1.0f;
            x1 = 1.0f *imgAspectRatio;
            y1 = 1.0f;
        }
        float[] coords = new float[] { x0, y0, x1, y0, x0, y1, x1, y1 };
        // Scale coordinates with mZoom
        for (int i = 0; i < 8; i++) {
            coords[i] *= mZoom;
        }
        // Rotate coordinates with mRot
        float cosa = (float)Math.cos(mRot);
        float sina = (float)Math.sin(mRot);
        float x,y;
        for (int i = 0; i < 8; i+=2) {
            x = coords[i]; y = coords[i+1];
            coords[i]   = cosa*x-sina*y;
            coords[i+1] = sina*x+cosa*y;
        }
        // Finally scale again to match screen aspect
        if (viewAspectRatio > 1.0f) {
            for (int i = 0; i < 8; i+=2) {
                coords[i] = coords[i]/viewAspectRatio;
            }
        } else {
            for (int i = 1; i < 8; i+=2) {
                coords[i] = coords[i]*viewAspectRatio;
            }
        }
        mPosVertices.put(coords).position(0);
    }
}

I suggest you to dive into OpenGL matrices and do all these transformations using them.

I've modified TextureRenderer class to implement GLSurfaceView.Renderer, and changed renderMode to RENDERMODE_CONTINUOUSLY.

Finally, the source for the modified demo is here.