Getting intermittent errors when acquiring the Google Glass camera

Question

I'm getting intermittent errors when acquiring the camera in a Glass GDK app. The app is a simple app that:

1. responds to a Voice Trigger ("test the camera")
2. starts an activity to take a snapshot
3. returns to the calling Activity to display the snapshot.

The problem is that the app works, but every now and again it does not acquire the camera. I've wrapped up the calls to the camera in multiple try/catch blocks to handle it (the app just exits if it can't acquire the camera), but I'm wondering why it happens in the first place.

I wouldn't be concerned about this except for the following:

1. I've noticed that these errors often occur when launching the app just after I've taken a photo with "take a picture" voice prompt (kind of as if the "take a picture" app didn't release the camera.)

2. I put in many, many try/catches to insure against every bad camera call, but... before I did this, (i.e. when I had code that wasn't so attentive about releasing the camera), the device would get very warm, so much so that I had to turn it off and on again to make sure I didn't damage it.

The only strange thing that I'm seeing in the logs are the following messages. I have no idea what "Unknown message type 8192" could be

11-29 19:38:16.344: E/Camera(4551): Received CAMERA_MSG_RELEASE
11-29 19:38:16.493: D/Camera-JNI(4551): android_hardware_Camera_release - context->decStrong(thiz)
11-29 19:38:16.524: E/Camera(4551): Unknown message type 8192

Because I have no idea what could be causing this, I'm going to post the whole project to see if there is something in an xml file or other obscure place that might be causing this.

Here is the Manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.testcamera"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-sdk
        android:minSdkVersion="15"
        android:targetSdkVersion="15" />

    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
    <uses-feature android:name="android.hardware.camera" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />

    <application
        android:allowBackup="true"
        android:icon="@drawable/ic_launcher"
        android:label="@string/app_name" >
        <activity
            android:name="com.example.testcamera.MainActivity"
            android:label="@string/app_name" >
            <intent-filter>
            <action
                android:name="com.google.android.glass.action.VOICE_TRIGGER" />
            </intent-filter>
            <meta-data android:name="com.google.android.glass.VoiceTrigger"
                       android:resource="@xml/voice_trigger" />
        </activity>
        <activity
            android:name="com.example.testcamera.CameraActivity"
            android:label="@string/app_name" />
    </application>

</manifest>

Here is the voice trigger:

<?xml version="1.0" encoding="utf-8"?>
<trigger keyword="@string/glass_voice_trigger">
        <constraints
        camera="true"
        network="true" 
        microphone="true" />
</trigger>

Here are the strings:

<?xml version="1.0" encoding="utf-8"?>
<resources>

    <string name="app_name">TestCamera</string>
    <string name="action_settings">Settings</string>
    <string name="hello_world">Hello world!</string>
    <string name="glass_voice_trigger">test the camera</string>

    <!-- Menu item strings. -->
    <string name="stop">Done</string>
    <string name="tapforoptions">Tap for options</string>

</resources>

Here is the layout for the main activity:

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@android:color/black">

        <ImageView
        android:id="@+id/bgPhoto"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:src="@drawable/black_bg"
        android:alpha="0.5" />

        <LinearLayout     android:id="@+id/queryLinearLayout"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_marginLeft="10dp"
    android:layout_marginRight="10dp"
    android:orientation="vertical">

       <TextView
        android:id="@+id/text1"
        android:layout_width="wrap_content"
        android:layout_marginLeft="2dp"
        android:layout_height="wrap_content"
        android:gravity="left"
        android:textColor="#FFFFFF"
        android:text="@string/hello_world"  />

       <TextView
        android:id="@+id/text2"
        android:layout_width="wrap_content"
        android:layout_marginLeft="2dp"
        android:layout_height="wrap_content"
        android:gravity="left"
        android:textColor="#FFFFFF"
        android:text="@string/hello_world"  />

</LinearLayout>

                <LinearLayout 
    android:id="@+id/resultLinearLayout"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_marginLeft="10dp"
    android:layout_marginRight="10dp"
    android:orientation="vertical">

       <TextView
        android:id="@+id/titleOfWork"
        android:layout_width="wrap_content"
        android:layout_marginLeft="2dp"
        android:layout_height="wrap_content"
        android:gravity="left"
        android:textColor="#FFFFFF"
        android:textSize="30sp"
        android:text="@string/hello_world"  />

       <TextView
        android:id="@+id/Singer"
        android:layout_width="wrap_content"
        android:layout_marginLeft="2dp"
        android:layout_height="wrap_content"
        android:gravity="left"
        android:textColor="#FFFFFF"
        android:textSize="36sp"
                android:textStyle="bold"
        android:text="@string/hello_world"  />

</LinearLayout>

       <ProgressBar
        android:id="@+id/my_progressBar"
        android:layout_height="wrap_content"
        android:layout_width="match_parent" 
        style="?android:attr/progressBarStyleHorizontal" 
        android:layout_gravity="bottom"
        android:layout_margin="10dp"
    />

                     <TextView
        android:id="@+id/tap_instruction"
        android:layout_width="fill_parent"
        android:layout_marginLeft="2dp"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:layout_gravity="bottom"
        android:textColor="#FFFFFF"
        android:textSize="16sp"
        android:textStyle="bold"
        android:layout_margin="20dp"
        android:text="@string/tapforoptions"  />

</FrameLayout>

Here is the layout for the camera activity:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent" >

    <SurfaceView 
        android:id="@+id/surfaceView"
        android:layout_height="match_parent"
        android:layout_width="match_parent" 
         />    
    <ImageView
        android:id="@+id/imageView"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        />

</LinearLayout>

Here is the code for the main activity:

package com.example.testcamera;

import java.io.File;
import com.google.android.glass.touchpad.Gesture;
import com.google.android.glass.touchpad.GestureDetector;
import android.os.Bundle;
import android.os.Handler;
import android.app.Activity;
import android.content.Context;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Point;
import android.speech.tts.TextToSpeech;
import android.util.Log;
import android.view.Display;
import android.view.Menu;
import android.view.MenuItem;
import android.view.MotionEvent;
import android.view.View;
import android.view.WindowManager;
import android.widget.ImageView;
import android.widget.LinearLayout;
import android.widget.ProgressBar;
import android.widget.TextView;

public class MainActivity extends Activity {

    // App responds to voice trigger "test the camera", takes a picture with CameraActivity and then returns.

    private static final String TAG = MainActivity.class.getSimpleName();
    private static final int TAKE_PHOTO_CODE = 1;
    private static final int PROGRESS_TIMEOUT = 30000; // in ms --> 10s
    private static final String IMAGE_FILE_NAME = "/sdcard/ImageTest.jpg";

    private boolean picTaken = false; // flag to indicate if we just returned from the picture taking intent
    private String theImageFile = ""; // this holds the name of the image that was returned by the camera

    private TextView text1;
    private TextView text2;

    private ProgressBar myProgressBar;
    protected boolean mbActive;

    private String inputQueryString;
    private String queryCategory;

    final Handler myHandler = new Handler(); // handles looking for the returned image file
    private int numberOfImageFileAttempts = 0;

    private String responseBody = "";

    private TextToSpeech mSpeech;

    private boolean readyForMenu = false;
    private boolean gotImageMatch = false;

    private GestureDetector mGestureDetector;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        Log.v(TAG,"creating activity");

        getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

        setContentView(R.layout.activity_main);
        text1 = (TextView) findViewById(R.id.text1);
        text2 = (TextView) findViewById(R.id.text2);
        text1.setText("");
        text2.setText("");
        myProgressBar = (ProgressBar) findViewById(R.id.my_progressBar);
        LinearLayout llResult = (LinearLayout) findViewById(R.id.resultLinearLayout);
        TextView tvResult = (TextView) findViewById(R.id.tap_instruction);
        llResult.setVisibility(View.INVISIBLE);
        tvResult.setVisibility(View.INVISIBLE);
        myProgressBar.setVisibility(View.INVISIBLE);

        // Even though the text-to-speech engine is only used in response to a menu action, we
        // initialize it when the application starts so that we avoid delays that could occur
        // if we waited until it was needed to start it up
        mSpeech = new TextToSpeech(this, new TextToSpeech.OnInitListener() {
            @Override
            public void onInit(int status) {
                // Do nothing.
            }
        });

        mGestureDetector = createGestureDetector(this);

    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.main, menu);
        return true;
    }

    @Override
    protected void onResume() {
        super.onResume();

        if (!picTaken) {
            Intent intent = new Intent(this, CameraActivity.class);
            intent.putExtra("imageFileName",IMAGE_FILE_NAME);
            startActivityForResult(intent,1);
        }
        else {
            // do nothing
        }
    }

    /*
     * Send generic motion events to the gesture detector
     */
    @Override
    public boolean onGenericMotionEvent(MotionEvent event) {
        if (mGestureDetector != null) {
            return mGestureDetector.onMotionEvent(event);
        }
        return false;
    }

    private GestureDetector createGestureDetector(Context context) {
        GestureDetector gestureDetector = new GestureDetector(context);
            //Create a base listener for generic gestures
            gestureDetector.setBaseListener( new GestureDetector.BaseListener() {
                @Override
                public boolean onGesture(Gesture gesture) {
                    if (gesture == Gesture.TAP) {
                        // do something on tap
                        Log.v(TAG,"tap");
                        //if (readyForMenu) {
                            openOptionsMenu();
                        //}
                        return true;
                    } else if (gesture == Gesture.TWO_TAP) {
                        // do something on two finger tap
                        return true;
                    } else if (gesture == Gesture.SWIPE_RIGHT) {
                        // do something on right (forward) swipe
                        return true;
                    } else if (gesture == Gesture.SWIPE_LEFT) {
                        // do something on left (backwards) swipe
                        return true;
                    }
                    return false;
                }
            });
            gestureDetector.setFingerListener(new GestureDetector.FingerListener() {
                @Override
                public void onFingerCountChanged(int previousCount, int currentCount) {
                  // do something on finger count changes
                }
            });
            gestureDetector.setScrollListener(new GestureDetector.ScrollListener() {
                @Override
                public boolean onScroll(float displacement, float delta, float velocity) {
                    // do something on scrolling
                    return false;
                }
            });
            return gestureDetector;
        }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        switch (item.getItemId()) {
            case R.id.stop:
                finish();
                return true;
            default:
                return super.onOptionsItemSelected(item);
        }
    }

    @Override
    public void onActivityResult(int requestCode, int resultCode, Intent data) {
      super.onActivityResult(requestCode, resultCode, data);
      picTaken = true;
      switch(requestCode) {
        case (1) : {
          if (resultCode == Activity.RESULT_OK) {
            // TODO Extract the data returned from the child Activity.
              Log.v(TAG,"onActivityResult"); 

              File f = new File(IMAGE_FILE_NAME);
               if (f.exists()) {
                   Log.v(TAG,"image file from camera was found");

                   Bitmap b = BitmapFactory.decodeFile(IMAGE_FILE_NAME);
                   Log.v(TAG,"bmp width=" + b.getWidth() + " height=" + b.getHeight());
                   ImageView image = (ImageView) findViewById(R.id.bgPhoto);
                   image.setImageBitmap(b);

                   text1 = (TextView) findViewById(R.id.text1);
                   text2 = (TextView) findViewById(R.id.text2);
                   text1.setText("Got a picture.");
                   text2.setText("\nSaved successfully to " + IMAGE_FILE_NAME);

                   LinearLayout llResult = (LinearLayout) findViewById(R.id.resultLinearLayout);
                   llResult.setVisibility(View.VISIBLE);
                   TextView line1 = (TextView) findViewById(R.id.titleOfWork);
                   TextView line2 = (TextView) findViewById(R.id.Singer);
                   TextView tap = (TextView) findViewById(R.id.tap_instruction);
                   line1.setText("");
                   line2.setText("");
                   tap.setVisibility(View.VISIBLE);
               }
          }
          else {
              Log.v(TAG,"onActivityResult returned bad result code");
              finish();
          }
          break;
        } 
      }
    }

    @Override
    protected void onDestroy() {

        //Close the Text to Speech Library
        if(mSpeech != null) {
            mSpeech.stop();
            mSpeech.shutdown();
            mSpeech = null;
            Log.d(TAG, "TTS Destroyed");
        }
        super.onDestroy();
    }

}

Here is the code for the camera activity:

package com.example.testcamera;

import java.io.BufferedOutputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.List;

import android.app.Activity;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Bitmap.CompressFormat;
import android.hardware.Camera;
import android.hardware.Camera.Parameters;
import android.os.Bundle;
import android.util.Log;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.widget.ImageView;

public class CameraActivity extends Activity implements SurfaceHolder.Callback
{
    private static final String TAG = CameraActivity.class.getSimpleName();
    public static final int BUFFER_SIZE = 1024 * 8;
    String imageFileName = "";
    //a variable to store a reference to the Image View at the main.xml file.
    private ImageView iv_image;
    //a variable to store a reference to the Surface View at the main.xml file
    private SurfaceView sv;
    //a bitmap to display the captured image
    private Bitmap bmp;
    //Camera variables
    //a surface holder
    private SurfaceHolder sHolder; 
    //a variable to control the camera
    private Camera mCamera;
    //the camera parameters
    private Parameters parameters;

  @Override
  public void onCreate(Bundle savedInstanceState)
  {
      Log.v(TAG,"onCreate");
      super.onCreate(savedInstanceState);
      setContentView(R.layout.activity_camera);
      //get the Image View at the main.xml file
      iv_image = (ImageView) findViewById(R.id.imageView);
      sv = (SurfaceView) findViewById(R.id.surfaceView);
      //Get a surface
      sHolder = sv.getHolder();
      sHolder.addCallback(this);
      Bundle extras = getIntent().getExtras();
      // get the image file name from the caller to save the 640x360 image
      imageFileName = extras.getString("imageFileName");
  }

    public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3)
    {
      Log.v(TAG,"surfaceChanged");
           //get camera parameters
      try {
          parameters = mCamera.getParameters();
          Log.v(TAG,"got parms");
          //set camera parameters
          parameters.setPreviewSize(640,360);
          parameters.setPictureSize(1280,720);
          //Camera.Parameters params = mCamera.getParameters();
          parameters.setPreviewFpsRange(30000, 30000);
          Log.v(TAG,"parms were set");
          mCamera.setParameters(parameters);

          mCamera.startPreview();
          Log.v(TAG,"preview started");

          //sets what code should be executed after the picture is taken
          Camera.PictureCallback mCall = new Camera.PictureCallback()
          {
            public void onPictureTaken(byte[] data, Camera camera)
            {
                Log.v(TAG,"pictureTaken");
                Log.v(TAG,"data bytes=" + data.length);
                  //decode the data obtained by the camera into a Bitmap
                Bitmap bmp = decodeSampledBitmapFromData(data,640,360);
                Log.v(TAG,"bmp width=" + bmp.getWidth() + " height=" + bmp.getHeight());
                FileOutputStream outStream = null;
                try{
                    FileOutputStream fos = new FileOutputStream(imageFileName);
                    final BufferedOutputStream bos = new BufferedOutputStream(fos, BUFFER_SIZE);
                    bmp.compress(CompressFormat.JPEG, 100, bos);
                    bos.flush();
                    bos.close();
                    fos.close();
                } catch (FileNotFoundException e){
                    Log.v(TAG, e.getMessage());
                } catch (IOException e){
                    Log.v(TAG, e.getMessage());
                }
                Intent resultIntent = new Intent();
                // TODO Add extras or a data URI to this intent as appropriate.
                resultIntent.putExtra("testString","here is my test");
                setResult(Activity.RESULT_OK, resultIntent);
                finish();
            }
          };
          Log.v(TAG,"set callback");
          mCamera.takePicture(null, null, mCall);
      }       
      catch (Exception e) {
        try {
              mCamera.release();
              Log.e(TAG,"released the camera");
          }
          catch (Exception ee) {
              // do nothing
              Log.e(TAG,"error releasing camera");
              Log.e(TAG,"Exception encountered relerasing camera, exiting:" + ee.getLocalizedMessage());
          }
        Log.e(TAG,"Exception encountered, exiting:" + e.getLocalizedMessage());
         mCamera = null;  
        Intent resultIntent = new Intent();
       setResult(Activity.RESULT_CANCELED, resultIntent);
       finish();
      }
    }

    public static Bitmap decodeSampledBitmapFromData(byte[] data,
            int reqWidth, int reqHeight) {

        // First decode with inJustDecodeBounds=true to check dimensions
        final BitmapFactory.Options options = new BitmapFactory.Options();
        options.inJustDecodeBounds = true;
        BitmapFactory.decodeByteArray(data, 0, data.length,options);
        options.inSampleSize = 2; // saved image will be one half the width and height of the original (image captured is double the resolution of the screen size)
        // Decode bitmap with inSampleSize set
        options.inJustDecodeBounds = false;
        return BitmapFactory.decodeByteArray(data, 0, data.length,options);
    }


    public void surfaceCreated(SurfaceHolder holder)
    {
          Log.v(TAG,"surfaceCreated");
          // The Surface has been created, acquire the camera and tell it where
          // to draw the preview.
          try {
              mCamera = Camera.open();
              Log.v(TAG,"acquired the camera");
              mCamera.setPreviewDisplay(holder);
              Log.v(TAG,"set surface holder for preview");
            }
          catch (Exception e) {
              try {
                  mCamera.release();
                  Log.v(TAG,"released the camera");
              }
              catch (Exception ee) {
                  // do nothing
                  Log.e(TAG,"Exception encountered releasing camera, exiting:" + ee.getLocalizedMessage());
              }
              Log.e(TAG,"Exception encountered, exiting:" + e.getLocalizedMessage());
              mCamera = null;  
             Intent resultIntent = new Intent();
             setResult(Activity.RESULT_CANCELED, resultIntent);
             finish();
          }

     }

    public void surfaceDestroyed(SurfaceHolder holder)
    {
        Log.v(TAG,"surfaceDestroyed");
        if (mCamera != null) {
            mCamera.stopPreview();
            //release the camera
            mCamera.release();
            //unbind the camera from this object
            mCamera = null;
        }  
    }

    @Override
    public void onPause()
    {
        Log.v(TAG,"onPause");
        super.onPause();
        if (mCamera != null) {
            mCamera.stopPreview();
            //release the camera
            mCamera.release();
            //unbind the camera from this object
            mCamera = null;
        }
    }

    @Override
    public void onDestroy()
    {
        Log.v(TAG,"onDestroy");
        super.onDestroy();

        if (mCamera != null) {
            mCamera.stopPreview();
            //release the camera
            mCamera.release();
            //unbind the camera from this object
            mCamera = null;
        }
    }
}

I bet it's something simple I just can't see...

Answer 1

Actually, I just checked on the Google Glass API bug reports and see exactly the same issue experienced by someone else: https://code.google.com/p/google-glass-api/issues/detail?id=259

I did notice the same thing, that this was related to the Voice Trigger. When I selected the item from the swipe menu I didn't see the same errors.

This can probably be closed

* later * Just updating this. I wrote a small sample app that makes repeat attempts to acquire the camera. The project is here: https://github.com/dazza222/GlassCameraSnapshot