Using VLC imem to play an h264 video file from memory but receiving error "main stream error: cannot pre fill buffer"

Question

I have an h264 video file that is loaded into memory, and I attempt to play it with imem using the parameter "imem-cat=4" so that vlc will use an access module to demux the video, and vlc starts and receives my imem parameters successfully:

[0x7f38a0000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7fff5b4a9430), cookie("IMEM")

This category also means I don't have to provide DTS and PTS. The imem module for VLC isn't well documented, but I've found hints in several places e.g.

https://forum.videolan.org/viewtopic.php?t=111917

https://forum.videolan.org/viewtopic.php?f=32&t=93842

Play video using libVLC from memory in python

My imem-get function simply sets the buffer pointer to the video data on the first call, returning 0, on any further calls it returns 1 to indicate that there is no more data:

int MyImemGetCallback (void *data,
                   const char *cookie,
                   int64_t *dts,
                   int64_t *pts,
                   unsigned *flags,
                   size_t * bufferSize,
                   void ** buffer)
{

ImemData* imem = (ImemData*)data;
cookie = imem->cookieString;

if(imem == NULL || imem->allBuffered==true) //indicate all data has been get()ted
    return 1;

*buffer = (void*) imem->video;
bufferSize = (size_t*) &(imem->bytes);
imem->allBuffered=true;

return 0;
}

Unfortunately after the first call I receive the following errors:

[0x189cb18] main input debug: Creating an input for 'imem://'
[0x189cb18] main input debug: using timeshift granularity of 50 MiB, in path '/tmp'
[0x189cb18] main input debug: `imem://' gives access `imem' demux `' path `'
[0x189cb18] main input debug: creating demux: access='imem' demux='' location='' file='(null)'
[0x7f2808000e28] main demux debug: looking for access_demux module matching "imem": 20 candidates
[0x7f2808000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808000e28] main demux debug: no access_demux modules matched
[0x189cb18] main input debug: creating access 'imem' location='', path='(null)'
[0x7f2808001958] main access debug: looking for access module matching "imem": 25 candidates
[0x7f2808001958] access_imem access debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808001958] main access debug: using access module "access_imem"
[0x7f2808000e28] main stream debug: Using block method for AStream*
[0x7f2808000e28] main stream debug: starting pre-buffering
[0x7f2808000e28] main stream error: cannot pre fill buffer
[0x7f2808001958] main access debug: removing module "access_imem"
[0x189cb18] main input warning: cannot create a stream_t from access
[0x17d7298] main libvlc debug: removing all interfaces
[0x17d7298] main libvlc debug: exiting
[0x17d7298] main libvlc debug: no exit handler
[0x17d7298] main libvlc debug: removing stats

For some reason it seems vlc cannot access the video data, but the error messages aren't very helpful and normally refer to network streams rather than memory locations.

Has anyone successfully used imem in this way or have any ideas as to what the problem could be? The video plays in VLC perfectly from disk. Thanks for any help.

Edit

It looks like the item interface may not actually support playing in this way. However, libVLC provides libvlc_media_t and livblc_media_new_callbacks which may allow me achieve what I want. I'll report back if I get it working.

Answer 1

So I couldn't get Imem to work, however on the VLC forums I was pointed towards a new API available in version 3.0.0. I had to remove my current installs of vlc and libvlc-dev and add the VLC daily builds PPA to the Ubuntu install, then install those versions.

The API is libvlc_media_new_callbacks:

LIBVLC_API libvlc_media_t * libvlc_media_new_callbacks
(
    libvlc_instance_t *instance, 
    libvlc_media_open_cb open_cb, 
    libvlc_media_read_cb read_cb, 
    libvlc_media_seek_cb seek_cb, 
    libvlc_media_close_cb close_cb, 
    void *opaque
);

You must implement each of those callbacks to give VLC access to the in-memory stream. Although the documentation states that implementing the seek() callback is unnecessary, I couldn't get h264 video to play without it.

The open() callback should pass a pointer to your video data, I recommend a container class so you can store the index of the last byte read along side it.

The read() callback is used to read len bytes into a buffer for which a pointer is passed. Here you can write len or less bytes to the buffer and return the number of bytes copied, block until you have some bytes ready, return 0 for EOF or -1 for error.

The seek() callback is used to set the byte index at which the next read() will take place.

Finally close() doesn't return anything and is used to tidy up.

Here is an example of a read() implementation:

class MemVideoData
{
    public:
        MemVideoData(char *data, int data_bytes) : video(data), bytes(data_bytes), lastByteIndex(0) {}   //init
        ~MemVideoData() {}
        char* video;   //pointer to video in memory
        int bytes;
        int lastByteIndex;
};

ssize_t memVideo_read(void *opaque, unsigned char* buf, size_t len)
{
  //TODO: block if not end of stream but no bytes available
  MemVideoData *mVid = (MemVideoData*) opaque;    //cast and give context
  int bytesToCopy=0;
  int bytesSoFar = mVid->lastByteIndex;
  int bytesRemaining = mVid->bytes - mVid->lastByteIndex;
  
  if(bytesRemaining >= len) bytesToCopy = len;   //at least as many bytes remaining as requested
  else if (bytesRemaining < len) bytesToCopy = bytesRemaining;    //less that requested number of bytes remaining
  else return 0;   // no bytes left to copy
  
  char *start = mVid->video;
  std::memcpy(buf,&start[mVid->lastByteIndex], bytesToCopy);    //copy bytes requested to buffer.
  mVid->lastByteIndex = mVid->lastByteIndex + bytesToCopy;    //increment bytes read count
  
  return bytesToCopy;
}

As requested here is an example of an Open callback:

int VideoPlayer::memVideo_open(void* opaque, void** datap, uint64_t* sizep)
{
   //TODO: get mutex on MemVideoData object pointed to by opaque
   MemVideoData *mVid = static_cast<MemVideoData*> (opaque); //cast opaque to our video state struct
   *sizep = (uint64_t) mVid->bytesTotal;    //set stream length
   *datap = mVid; // point to entire object. Think this was intended to point to the raw video data but we use the MemVideoData object in read() and seek()
   mVid->lastByteReadIndex=0;
   return 0;
}