Decode of live RTSP stream: large video lag using MediaPlayer on Android

Question

I'm playing a Live RTSP stream from VLC on a PC to Android MediaPlayer class (both on same local network). It plays smoothly with no errors - the problem is that the decoded video on screen is between around 5 and 7 seconds behind live.

From debug and callbacks I can see that the live data is arriving on the device < 1s after starting mMediaPlayer.prepareAsync(). This is when the MediaPlayer class begins to work out what format the stream is with what dimensions etc. Then just before video is shown on screen (between 5 and 7 seconds later), onPrepared() is called where I call mMediaPlayer.start(). It looks like this start() plays the video that was originally captured from the start of the prepare stage.

I've tried seekTo(5000) both before and after start(), but it has no effect on the lag at all.

For a live video calling app, the setup delay of a few seconds is perfectly fine, but this lag once video is presented is unaccaptable to me.

public void surfaceCreated(SurfaceHolder holder)
{
   mMediaPlayer = new MediaPlayer();
   mMediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
   mMediaPlayer.setOnInfoListener(this);
   mMediaPlayer.setOnErrorListener(this);
   mMediaPlayer.setOnVideoSizeChangedListener(this);
   mMediaPlayer.setOnBufferingUpdateListener(this);
   mMediaPlayer.setDataSource("rtsp://192.168.1.4:5544/test");
   mMediaPlayer.setDisplay(holder);
   mMediaPlayer.setScreenOnWhilePlaying(true);
   mMediaPlayer.setOnPreparedListener(this);
   mMediaPlayer.setOnCompletionListener(this);
   mMediaPlayer.prepareAsync();
   ...
public void onPrepared(MediaPlayer mediaplayer)
{
   mMediaPlayer.start();
...

Any ideas how I can reduce this lag, or seek to the end of what is buffered by MediaPlayer? My device is 3.1, minSdkVersion is 2.2.

EDIT:

I've found some high and low water-marks in AwesomePlayer.cpp (2s and 8s). As a quick test I have hacked the libstagefright.so to make these 0.1s and 1s. This however made no effect on the delay. My search continues...

Answer 1

I'm not giving a final answer, but let me share what I have by now.

• I had a 5s latency issue playing locally on PC, from GStreamer to GStreamer. The latency was gone after adding the following parameters to pipelines:
  • on client - latency=0 parameter of rtspsrc;
  • on server - v4l2src's is-live=1 parameter and, of course, x264enc tune=zerolatency.

There is no way to control MediaPlayer/VideoView's codec/media source parameters. Neither in MediaCodec API, as far as I see.

So we need to go for GStreamer or ffmpeg.

• NDK access to hardware-accelerated codecs was introduced in Android 4.2 (JellyBean) (look for "androidmedia"), and people had some success utilizing it.
• NDK hardware-accelerated codec access in FFMPEG was implemented in 2011, in FFMPEG 0.9

Ease of use and portability are to be found out yet.