How to set OpenCV's camera to display preview in both portrait orientation and full screen

Question

I'm trying to create an Android app. which displays a camera preview in portrait mode at all times and performs some heavy image processing operations on (some of) it's frames. Hence, I'm using OpenCV (both of it's OpenCV4Android and native C/C++ components). The problem is that when using CameraBridgeViewBase or JavaCameraView classes, the frame which is returned by OnCameraFrame is in Landscape mode.

Now, if the Activity is defined to use Landscape mode (just like OpenCV's sample apps), the preview looks fine but any additional UI views are tilted by 90 degrees (and as mentioned before, the device should run my App in Portrait mode).

If the Activity is set to portrait mode, UI views will obviously look right, but camera preview will be tilted by 90 degrees.

If I try to rotate each frame by manipulating the image Matrix on OnCameraFrame like this:

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
  mRgba = inputFrame.rgba();
  Core.transpose(mRgba,mRgbaT);
  Core.flip(mRgbaT, mRgbaT, -1);
  Imgproc.resize(mRgbaT, mRgbaT, mRgba.size());

  return mRgbaT;
}

then each camera's frame fills device's width but not it's height (and as a result, it looks streched), plus - it considerably slows down the frame rate. Trying to resize the image to fullscreen (or any size different than the original frame size) results in displaying no image at all (black screen) and the following exception is thrown:

E/CameraBridge(11183): Utils.matToBitmap() throws an exception: /home/reports/ci/slave_desktop/50-SDK/opencv/modules/java/generator/src/cpp/utils.cpp:97: error: (-215) src.dims == 2 && info.height == (uint32_t)src.rows && info.width == (uint32_t)src.cols in function void Java_org_opencv_android_Utils_nMatToBitmap2(JNIEnv*, jclass, jlong, jobject, jboolean)

So, my question is: How can I display OpenCV's camera preview in both portrait mode and full screen?

Unfortunately, due to my beginner SO reputation I can't attach screen-shots. Also, I'm aware of the fact that somewhat similar questions were asked on SO before, but none of the answers seem to solve the problem completely.

Answer 1

I've found a solution: Create a custom camera class which extends from CameraBridgeViewBase in a similar way that JavaCameraView extends from it (most parts are even identical), but when implementing the inner JavaCameraFrame class, replace the method which returns a Mat object to something like this:

    public Mat rgba() {
        Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2BGR_NV12, 4);
        if (mRotated != null)
            mRotated.release();
        mRotated = mRgba.t();
        Core.flip(mRotated, mRotated, -1);
        return mRotated;
    }

There is a great example here (The first answer, courtesy of Zarokka ). There is some downgrade in performance, but it's remotely not as serious as when rotating the output Mat in OnCameraFrame (which doesn't fully solve the problem anyway). I use it on a 640*480 resolution preview frame size (full screen) and it looks good and works smoothly even not not-so new devices.