record/save audio from voice recognition intent

Question

I want to save/record the audio that Google recognition service used for speech to text operation (using RecognizerIntent or SpeechRecognizer).

I experienced many ideas:

1. onBufferReceived from RecognitionListener: I know, this is not working, just test it to see what happens and onBufferReceived is never called (tested on Galaxy nexus with JB 4.3)

2. Used a media recorder: not working. It's breaking speech recognition. Only one operation is allowed for mic

3. Tried to find where recognition service is saving the temporary audio file before the execution of the speech to text API to copy it, but without success

I was almost desperate but I just noticed that Google Keep application is doing what I need to do! I debuged a little the keep application using logcat and the app is also calling the "RecognizerIntent.ACTION_RECOGNIZE_SPEECH" (like we, developers, do) to trigger speech to text. But, how keep is saving the audio? Can it be a hide API? Is Google "cheating"?

Answer 1

@Kaarel's answer is almost complete - the resulting audio is in intent.getData() and can be read using ContentResolver

Unfortunately, the AMR file that is returned is low quality - I wasn't able to find a way to get high quality recording. Any value I tried other than "audio/AMR" returned null in intent.getData().

If you find a way to get high quality recording - please comment or add an answer!

public void startSpeechRecognition() {    // Fire an intent to start the speech recognition activity.    Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);    // secret parameters that when added provide audio url in the result    intent.putExtra("android.speech.extra.GET_AUDIO_FORMAT", "audio/AMR");    intent.putExtra("android.speech.extra.GET_AUDIO", true);     startActivityForResult(intent, "<some code you choose>"); }  // handle result of speech recognition @Override public void onActivityResult(int requestCode, int resultCode, Intent data) {     // the resulting text is in the getExtras:     Bundle bundle = data.getExtras();     ArrayList<String> matches = bundle.getStringArrayList(RecognizerIntent.EXTRA_RESULTS)     // the recording url is in getData:     Uri audioUri = data.getData();     ContentResolver contentResolver = getContentResolver();     InputStream filestream = contentResolver.openInputStream(audioUri);     // TODO: read audio file from inputstream } 

Answer 2

Last time I checked, Google Keep set these extras:

• android.speech.extra.GET_AUDIO_FORMAT: audio/AMR
• android.speech.extra.GET_AUDIO: true

These are not documented as part of the Android documentation, so they do not constitute an Android API. Also, Google Keep does not rely on the recognizer intent to consider these extras. It would certainly be nice if such extras were popularized and documented by Google.

To find out which extras are set by Google Keep when it calls the RecognizerIntent, implement an app that responds to the RecognizerIntent and print out all the extras that it receives. You can also install Kõnele (http://kaljurand.github.io/K6nele/), which is an implementation of RecognizerIntent. When Kõnele is launched by Google Keep, then long-press the wrench-shaped settings icon. This shows some technical details about the caller, and includes also the incoming extras.

The answer by @Iftah explains how Google Keep returns the audio recording to the caller of RecognizerIntent.