Android camera preview stretched using Grafika CameraCapture code

Question

I'm looking for help with an issue I'm facing using Grafika's CameraCaptureActivity code. I want to build an app that can record camera and display a preview, so this sample and code looked like exactly what I wanted and so far it's been great, this issue appart.

The issue I have is that when the camera preview size doesn't match the exact size of the GLSurfaceView I use to display the preview it shows a stretched preview.

Here's how it looks:

As you can see it's stretched horizontally since the note sheet is a perfect square.

For this precise exemple on a Samsung Galaxy S4, the camera preview size is 1280x720 (extracted from available preview sizes using Camera2 API), and the GLSurfaceView is 1920x1080 to cover the entire screen.

When does it appear?

The issue appears when the SurfaceView is upscaled compared to the Camera output, even if I always make sure the ratio remains correct (1920x1080 = 1,5 * 1280x720).

What code are you using?

I use Grafika's code, including:

• CameraCaptureActivity
• AspectFrameLayout
• FullFrameRect
• Drawable2d
• EglCore
• EglBaseSurface

Some log?

Here's what SurfaceFlinger dump looks like:

Hardware Composer state (version 01030000):
  mDebugForceFakeVSync=0
  Display[0] configurations (* current):
    * 0: 1080x1920, xdpi=442.450989, ydpi=439.351013, secure=1 refresh=16666667
  numHwLayers=3, flags=00000000
    type   |  handle  | hint | flag | tr | blnd |  format     |     source crop(l,t,r,b)       |           frame        |      dirtyRect         |  name 
------------+----------+----------+----------+----+-------+----------+-----------------------------------+---------------------------+-------------------
       HWC | b3b12ba0 | 0002 | 0000 | 00 | 0100 | RGB_888     |    0.0,    0.0, 1080.0, 1920.0 |    0,    0, 1080, 1920 | [    0,    0, 1080, 1920] | SurfaceView
       HWC | b3d12740 | 0002 | 0000 | 00 | 0105 | RGBA_8888   |    0.0,    0.0, 1065.0, 1905.0 |    0,    0, 1065, 1905 | [    0,    0, 1065, 1905] | com.mybundle.myapp/com.mybundle.myapp.activity.MyActivity
 FB TARGET | b6a55c40 | 0000 | 0000 | 00 | 0105 | RGBA_8888   |    0.0,    0.0, 1080.0, 1920.0 |    0,    0, 1080, 1920 | [    0,    0,    0,    0] | HWC_FRAMEBUFFER_TARGET

Layer 0xb4094000 (SurfaceView)
  Region transparentRegion (this=0xb4094160, count=1)
    [  0,   0,   0,   0]
  Region visibleRegion (this=0xb4094008, count=1)
    [  0,   0, 1920, 1080]
      layerStack=   0, z=    21015, pos=(0,0), size=(1920,1080), crop=(   0,   0,1920,1080), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000002, tr=[1.00, 0.00][0.00, 1.00]
      client=0xb66190c0
      format= 4, activeBuffer=[1080x1920:1536,  3], queued-frames=0, mRefreshPending=0
            mTexName=116 mCurrentTexture=0
            mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
            mAbandoned=0
            -BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[1920x1080], default-format=4, transform-hint=04, FIFO(0)={}
            >[00:0xb1c18500] state=ACQUIRED, 0xb3b12ba0 [1080x1920:1536,  3]

Camera preview size code?

StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

// Retrieve display size
WindowManager wm = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
Display display = wm.getDefaultDisplay();
Point displaySize = new Point();
display.getSize(displaySize);

// Choose the sizes for camera preview
mPreviewSize = chooseOptimalSize(map.getOutputSizes(SurfaceTexture.class),
                    displaySize.x, displaySize.y, captureWidth, captureHeight);

private static Size chooseOptimalSize(Size[] choices, int screenWidth, int screenHeight, int captureWidth, int captureHeight)
    {
        // Collect the supported resolutions that are at least as big as the screen && wanted capture size
        List<Size> bigEnough = new ArrayList<>();
        // Collect the supported resolutions that are at least as big as the wanted capture size but < to screensize
        List<Size> smallerButEnough = new ArrayList<>();

        for (Size option : choices) {
            if (option.getHeight() == option.getWidth() * captureHeight / captureWidth) {
                if( option.getWidth() >= screenWidth && option.getHeight() >= screenHeight )
                {
                    bigEnough.add(option);
                }
                else
                {
                    smallerButEnough.add(option);
                }
            }
        }

        // Pick the smallest of those, assuming we found any
        if (bigEnough.size() > 0)
        {
            return Collections.min(bigEnough, new CompareSizesByArea());
        }
        // Pick the biggest of those, assuming we found any
        else if( smallerButEnough.size() > 0 )
        {
            return Collections.max(smallerButEnough, new CompareSizesByArea());
        }
        else
        {
            Log.e(TAG, "Couldn't find any suitable preview size");
            return choices[0];
        }
    }

What have you tried?

I first thought it was a Camera2 issue, so I tried to use only the old Camera API, but result is the same.

Then I thought it was a ratio issue, but as you can see the ratio here is ok horizontally and vertically.

My bet is that there's an OpenGL transformation to apply when the Surface is resized but I'm a pure noob in OpenGL so I haven't been able to find anything that works. I try added those lines but it doesn't help:

@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
    Log.d(TAG, "onSurfaceChanged " + width + "x" + height);

    gl.glViewport(0, 0, width, height);
    gl.glClear(GLES10.GL_COLOR_BUFFER_BIT | GLES10.GL_DEPTH_BUFFER_BIT);
}

Another example, right from Grafika:

Here's an other example of the issue, right from the Grafika app.

Phone: Galaxy S3 Neo (GT-I9301L) - Android 4.4.2

The phone has a 1280x720 screen, camera opened with the 640x480 resolution, displayed in a 573x430. Here's how it looks:

SurfaceFlinger logs:

Hardware Composer state (version  1030000):
  mDebugForceFakeVSync=0
  Display[0] : 720x1280, xdpi=304.799011, ydpi=306.716003, refresh=16666667
  numHwLayers=4, flags=00000000
    type    |  handle  |   hints  |   flags  | tr | blend |  format  |          source crop            |           frame           name 
------------+----------+----------+----------+----+-------+----------+---------------------------------+--------------------------------
        HWC | b84a3d18 | 00000002 | 00000000 | 00 | 00100 | 00000002 | [    0.0,    0.0,  430.0,  573.0] | [   32,  353,  462,  926] SurfaceView
        HWC | b841b080 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,  670.0, 1280.0] | [    0,    0,  670, 1280] com.android.grafika/com.android.grafika.CameraCaptureActivity
        HWC | b8423250 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,   50.0, 1280.0] | [  670,    0,  720, 1280] StatusBar
  FB TARGET | b8412d50 | 00000000 | 00000000 | 00 | 00105 | 00000001 | [    0.0,    0.0,  720.0, 1280.0] | [    0,    0,  720, 1280] HWC_FRAMEBUFFER_TARGET


+ Layer 0xb8416080 (SurfaceView) id=112
  Region transparentRegion (this=0xb84169d0, count=1)
    [  0,   0,   0,   0]
  Region visibleRegion (this=0xb8416088, count=1)
    [353, 258, 926, 688]
      layerStack=   0, z=    21015, pos=(353,258), size=( 573, 430), crop=(   0,   0, 573, 430), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000000, tr=[1.00, 0.00][0.00, 1.00]
      client=0xb841d7c8
      format= 4, activeBuffer=[ 430x 573: 448,  2], queued-frames=0, mRefreshPending=0
            mTexName=95 mCurrentTexture=1
            mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
            mAbandoned=0
            -BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[573x430], default-format=4, transform-hint=04, FIFO(0)={}
             [00:0xb84439a8] state=FREE    , 0xb849a528 [ 430x 573: 448,  2]
            >[01:0xb8421128] state=ACQUIRED, 0xb84a3d18 [ 430x 573: 448,  2]
             [02:0xb84238e8] state=FREE    , 0xb84432f8 [ 430x 573: 448,  2]

Grafika's app log:

04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onCreate complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onResume -- acquiring camera
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: Camera preferred preview size for video is 1920x1080
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika-AFL: Setting aspect ratio to 1.3333333333333333 (was -1.0)
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: onResume complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:50.064 14383-32312/com.android.grafika D/Grafika: setCameraPreviewSize
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.124 14383-32312/com.android.grafika D/Grafika: onSurfaceCreated
04-18 18:06:50.134 14383-32312/com.android.grafika D/Grafika: Created program 3 (TEXTURE_EXT)
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: onSurfaceChanged 573x430
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: Updating filter to 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: onItemSelected: 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: CameraHandler [Handler (com.android.grafika.CameraCaptureActivity$CameraHandler) {42aeec70}]: what=0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: onVisibilityChanged() is called, visibility : 4
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: unregisterIRListener() is called 

So...

If one of you guys have an idea of how to make it work, I would love to hear from him.

Thanks!

Answer 1

i`m sorry for my last answer, it was stupid and didn`t resolve problem. But i founded another solution, and tested it on 5 device using different camera aspect ratio. As i understood, you`re using Grafika code. Try to remove parms.setRecordingHint(true); So now i`m using only param.setPictureSize() param.setPreviewSize() with same aspect ratio in camera params. Also i set gl view`s size with that aspect, i made it by myself.In grafika it used fixed size, afaik. Hope it`ll help you.