How Do I Determine the Default Orientation of Camera Preview Frames?

Question

I am trying (again) to create camera preview logic that actually works properly, for all scenarios:

• any device: phone, tablet, toaster, whatever
• any camera: front-facing, rear-facing, side-facing, dog-facing, whatever
• android.hardware.Camera and android.hardware.camera2
• portrait and landscape device orientations

Since my minSdkVersion is 15, and since I am not especially concerned about performance, I am trying to use a TextureView. And, following the advice of fadden in places like here and here, I am trying to use setTransform() on that TextureView with an appropriate Matrix that:

• orients the preview properly, taking device orientation into account
• fills the TextureView completely, at the cost of cropping where the TextureView aspect ratio does not match the preview frame aspect ratio
• does not stretch the image, so that a preview of a square item (e.g., a 3" square Post-It Note®) shows up square in the preview

In my case, the TextureView fills the screen, minus the status bar and navigation bar.

Starting with the adjustAspectRatio() from Grafika's PlayMovieActivity.java, I now have this:

  private void adjustAspectRatio(int videoWidth, int videoHeight,
                                 int rotation) {
    if (iCanHazPhone) {
      int temp=videoWidth;
      videoWidth=videoHeight;
      videoHeight=temp;
    }

    int viewWidth=getWidth();
    int viewHeight=getHeight();
    double aspectRatio=(double)videoHeight/(double)videoWidth;
    int newWidth, newHeight;

    if (getHeight()>(int)(viewWidth*aspectRatio)) {
      newWidth=(int)(viewHeight/aspectRatio);
      newHeight=viewHeight;
    }
    else {
      newWidth=viewWidth;
      newHeight=(int)(viewWidth*aspectRatio);
    }

    int xoff=(viewWidth-newWidth)/2;
    int yoff=(viewHeight-newHeight)/2;

    Matrix txform=new Matrix();

    getTransform(txform);

    float xscale=(float)newWidth/(float)viewWidth;
    float yscale=(float)newHeight/(float)viewHeight;

    txform.setScale(xscale, yscale);

    switch(rotation) {
      case Surface.ROTATION_90:
        txform.postRotate(270, newWidth/2, newHeight/2);
        break;

      case Surface.ROTATION_270:
        txform.postRotate(90, newWidth/2, newHeight/2);
        break;
    }

    txform.postTranslate(xoff, yoff);

    setTransform(txform);
  }

Here, videoWidth and videoHeight are the size of the camera preview, and the method itself is implemented on a subclass of TextureView. I am calling this method when I have established what the camera preview size is and after the TextureView itself is resized.

This appears to be close but not completely correct. In particular, the iCanHazPhone hack — flipping the video width and height — is a stab in the dark, as without this, while a SONY Tablet Z2 works well, a Nexus 5 turns out horrible (stretched preview that does not fill the screen).

With iCanHazPhone set to true, I get good results on a Nexus 5:

With iCanHazPhone set to false, I get stuff like:

Similarly, with iCanHazPhone set to false, I get good results on a SONY Tablet Z2:

But if I flip it to true, I get:

My current theory is that different devices have different default camera orientations, and depending on that default orientation I need to flip the preview width and height in my calculations.

So, the questions:

1. Is the camera guaranteed (as much as anything involving Android hardware) to have a default orientation that matches the default device orientation? For example, a Nexus 9 works correctly with iCanHazPhone set to true, indicating that it's not phone vs. tablet but default-portrait vs. default-landscape.

2. Is there a better way of dealing with this?

Answer 1

Answer to both of your questions is: use the sensor orientation provided by the Camera/Camera2 APIs to adjust your preview image.

To calculate relative camera rotation to screen (which can be used to transform your preview) I use:

static int getRelativeImageOrientation(int displayRotation, int sensorOrientation,
                                       boolean isFrontFacing, boolean compensateForMirroring) {
    int result;
    if (isFrontFacing) {
        result = (sensorOrientation + displayRotation) % 360;
        if (compensateForMirroring) {
            result = (360 - result) % 360;
        }
    } else {
        result = (sensorOrientation - displayRotation + 360) % 360;
    }
    return result;
}

where displayRotation is the current display rotation:

static int getDisplayRotation(Context context) {
    WindowManager windowManager = (WindowManager) context
            .getSystemService(Context.WINDOW_SERVICE);
    int rotation = windowManager.getDefaultDisplay().getRotation();
    switch (rotation) {
        case Surface.ROTATION_0:
            return 0;
        case Surface.ROTATION_90:
            return 90;
        case Surface.ROTATION_180:
            return 180;
        case Surface.ROTATION_270:
            return 270;
    }
    return 0;
}

sensorOrientation for the legacy Camera:

Camera.CameraInfo.orientation

and for the Camera2:

CameraCharacteristics#get(CameraCharacteristics.SENSOR_ORIENTATION)

You should pass false for compansateForMirror when calculating camera preview orientation and pass true when calculating legacy Camera JPG orientation.

I've tested this across a number of devices - it seems to work, although I cannot guarantee that this is bulletproof ;]