Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Android MediaCodec not decoding all input buffers

In Android 4.4.2, I am using MediaCodec to decode mp3 files. I'm using queueInputBuffer() to queue an input mp3 encoded frames and dequeueOutputBuffer() to get the decoded frames. But decoder gives decoded output from 8th frame onwards ( based on the bufferInfo.presentationTimeUs) and skips initial 7 frames. This scenario occurs only for few streams but not for all the streams. Also, this sort of behavior is consistent over many runs.

I want the decoded output of all the frames and I don't want any frames to skipped. Can anyone help me to understand why the frames are skipped? I assure that stream is not corrupted. Because I am getting INFO_TRY_AGAIN till 7th frame, when valid buffer index is returned by `dequeueOutputBuffer', its presentation time is always 8th frame's.

Following is the queuing code :

Log.e(TAG, "audptOffset = "+audptOffset+"input PT = "+audpt);
                audcodec.queueInputBuffer(audInbufIndex, 0, audchunkSize, audpt, 0);

Following is how i call dequeue and write to AudioTrack:

if(!audoutputDone ){
                if(!waitForAudioRelease){
                    auoutBufIndex  = audcodec.dequeueOutputBuffer(auinfo, 100);
                    Log.e(TAG, "Output PT = " + auinfo.presentationTimeUs+"auoutBufIndex = "+auoutBufIndex);
                }
                if (auoutBufIndex >= 0) {
                    waitForAudioRelease = true;
                } else if (auoutBufIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
                    auddecOutBufArray = audcodec.getOutputBuffers();
                } else if (auoutBufIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                    MediaFormat newFormat = audcodec.getOutputFormat();
                    int sampleRate1 = newFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);
                    int channelCount1 =newFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
                    Log.e(TAG, "INFO_OUTPUT_FORMAT_CHANGED");
                    int minBufSize1 = AudioTrack.getMinBufferSize(sampleRate1, (channelCount1==2)?
                            AudioFormat.CHANNEL_OUT_STEREO:AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
                    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
                            sampleRate1, (channelCount1==2)?AudioFormat.CHANNEL_OUT_STEREO:AudioFormat.CHANNEL_OUT_MONO,
                            AudioFormat.ENCODING_PCM_16BIT, minBufSize1,
                            AudioTrack.MODE_STREAM);
                    audioTrack.play();
                    waitForAudioRelease = false;
                }
                audionowUs = System.currentTimeMillis();
                if (auoutBufIndex >= 0) {               
                    auwhenRealUs = (auinfo.presentationTimeUs/1000) + mStartTimeRealMs - (audptOffset/1000);
                    aulateByUs = audionowUs - auwhenRealUs;


                    if(!audioWaitTillStartTime){
                        while((mStartTimeRealMs+((auinfo.presentationTimeUs/1000) - (audptOffset/1000))) >= audionowUs){
                            try {
                                Thread.sleep(10);
                            } catch (InterruptedException e) {
                                // TODO Auto-generated catch block
                                e.printStackTrace();
                            }
                            audionowUs = System.currentTimeMillis();
                        }
                        Log.e(TAG,"Play is going to start PT Difference = "+((auinfo.presentationTimeUs/1000) - (audptOffset/1000)));
                    }

Adding More Logs :

02-22 17:46:03.164: E/CL(28650): received play command from server
02-22 17:46:03.209: E/RealTimeClient(28650): created decoder for audio/mpeg
02-22 17:46:03.234: E/Music(28650): Output PT = 0
02-22 17:46:03.234: E/Music(28650): pt of first frame received 1215000
02-22 17:46:03.234: E/Music(28650): Input PT = 1215000
02-22 17:46:03.234: E/Music(28650): Output PT = 0
02-22 17:46:03.234: E/Music(28650): Input PT = 1241122
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1267244
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1293367
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1319489
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.244: E/Music(28650): INFO_OUTPUT_FORMAT_CHANGED
02-22 17:46:03.249: I/Reverb(28650):  getpid() 28650, IPCThreadState::self()->getCallingPid() 28650
02-22 17:46:03.249: E/Reverb(28650): Reverb::StartElementHandler, wrong element or attributes: boolean
02-22 17:46:03.249: E/Music(28650): Input PT = 1345612
02-22 17:46:03.254: E/Music(28650): Output PT = 1293367
02-22 17:46:03.259: E/Music(28650): Input PT = 1371734
02-22 17:46:03.259: E/Music(28650): Input PT = 1397857
02-22 17:46:03.259: E/Music(28650): Input PT = 1423979
02-22 17:46:03.259: E/Music(28650): Input PT = 1450102
02-22 17:46:03.264: E/Music(28650): Input PT = 1476224
02-22 17:46:03.269: E/Music(28650): Input PT = 1502346
02-22 17:46:03.269: E/Music(28650): Input PT = 1528469
02-22 17:46:03.269: E/Music(28650): Input PT = 1554591
02-22 17:46:03.269: E/Music(28650): Input PT = 1580714
02-22 17:46:03.269: E/Music(28650): Input PT = 1606836
02-22 17:46:03.269: E/Music(28650): Input PT = 1632959
02-22 17:46:03.269: E/Music(28650): Input PT = 1659081
02-22 17:46:04.124: W/AudioTrack(28650): releaseBuffer() track 0x5e2faf28 name=0x3 disabled, restarting
02-22 17:46:04.129: E/Music(28650): Output PT = 1319489
02-22 17:46:04.129: E/Music(28650): Input PT = 1685204
02-22 17:46:04.159: E/Music(28650): Output PT = 1345612
like image 966
nmxprime Avatar asked Jan 25 '16 10:01

nmxprime


1 Answers

MPEG-1 Layer III (MP3) has dependent frames, you can't just start from any frame like Layer I or Layer II. Quoting from the provided link, "In the worst case, 9 input frames may be needed before being able to decode one single frame." This is most likely what you are seeing. Although each of the first 7 frames has a PTS associated with it, it isn't until you reach the 8th frame that the decoder is actually able to fully decode a frame and begin playback. Playback begins with the 8th frames PTS. You would need to painfully parse through the bytes of the stream in question by hand to fully verify this is what is happening, but I suspect you are in fact playing all of the frames.

like image 98
Kaleb Avatar answered Oct 27 '22 00:10

Kaleb