Stutter when updating Video Texture using multiple Pixel Buffer Objects

Question

I have an application that decodes a video file using FFMPEG (in a separate thread) and renders this texture using PBOs in another. All the PBO do-hickey happens in the following function:

void DynamicTexture::update()
{
    if(!_isDirty)
    {
        return;
    }

    /// \todo   Check to make sure that PBOs are supported
    if(_usePbo)
    {
        // In multi PBO mode, we keep swapping between the PBOs
        // We use one PBO to actually set the texture data that we will upload
        // and the other we use to update/modify. Once modification is complete,
        // we simply swap buffers

        // Unmap the PBO that was updated last so that it can be released for rendering

        glBindBuffer(GL_PIXEL_UNPACK_BUFFER, _pboIds[_currentPboIndex]);
        glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER);
        Util::GLErrorAssert();

        // bind the texture
        glBindTexture(GL_TEXTURE_2D, _textureId);
        Util::GLErrorAssert();

        // copy pixels from PBO to texture object
        // Use offset instead of pointer.
        glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _width, _height, 
                        (_channelCount==4)?GL_RGBA:GL_RGB,
                        GL_UNSIGNED_BYTE, 0);
        Util::GLErrorAssert();

        // Now swap the pbo index
        _currentPboIndex = (++_currentPboIndex) % _numPbos;

        // bind PBO to update pixel values
        glBindBuffer(GL_PIXEL_UNPACK_BUFFER, _pboIds[_currentPboIndex]);
        Util::GLErrorAssert();

        // map the next buffer object into client's memory
        // Note that glMapBuffer() causes sync issue.
        // If GPU is working with this buffer, glMapBuffer() will wait(stall)
        // for GPU to finish its job
        GLubyte* ptr = (GLubyte*)glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY);
        Util::GLErrorAssert();
        if(ptr)
        {
            // update data directly on the mapped buffer
            _currentBuffer = ptr;
            Util::GLErrorAssert();
        }
        else
        {
            printf("Unable to map PBO!");
            assert(false);
        }

        // It is good idea to release PBOs with ID 0 after use.
        // Once bound with 0, all pixel operations behave normal ways.
        glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
        Util::GLErrorAssert();

        // If a callback was registered, call it
        if(_renderCallback)
        {
            (*_renderCallback)(this);
        }
    }
    else
    {
        glBindTexture(GL_TEXTURE_2D, _textureId);
        glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
                        _width, _height, (_channelCount==4)?GL_RGBA:GL_RGB,
                        GL_UNSIGNED_BYTE,
                        &(_buffer[0]) 
                        );
        Util::GLErrorAssert();
    }

    // Reset the dirty flag after updating
    _isDirty = false;
}

In the decoding thread, I simply update _currentBuffer and set the _isDirty flag to true. This function is called in the render thread.

When I use a single PBO, i.e. when _numPbos=1 in the above code, then the rendering works fine without any stutter. However, when I use more than one PBO, there is a visible stutter in the video. You can find a sample of me rendering 5 videos with _numPbos=2 here. The more number of PBOs I use, the worse the stutter becomes.

Theoretically, the buffer that I am updating and the buffer than I am using for render are different, so there should be no glitch of this sort. I want to use double/triple buffering so as to increase rendering performance.

I am looking for some pointers/hints as to what could be going wrong.

Answer 1

I dont know, if it is your problem, but after you call this:

 glBindBuffer(GL_PIXEL_UNPACK_BUFFER, _pboIds[_currentPboIndex]);
 glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER);
 Util::GLErrorAssert();

You are calling

glBindTexture

But you are still operating with buffer at index _currentPboIndex.

In my code, I have two indices - index and nextIndex

In init I set

index = 0;
nextIndex = 1;

Than my update pipeline is like this:

index = (index + 1) % 2;
nextIndex = (nextIndex + 1) % 2;

uint32 textureSize = sizeof(RGB) * width * height;


GL_CHECK( glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pbo[nextIndex]) );
GL_CHECK( glBufferData(GL_PIXEL_UNPACK_BUFFER, textureSize, 0, GL_STREAM_DRAW_ARB) );
GL_CHECK( gpuDataPtr = glMapBufferRange(GL_PIXEL_UNPACK_BUFFER, 0, textureSize, GL_MAP_WRITE_BIT | GL_MAP_INVALIDATE_BUFFER_BIT) );

//update data gpuDataPtr

GL_CHECK( glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER_ARB) );

//bind texture

GL_CHECK( glBindBufferARB(GL_PIXEL_UNPACK_BUFFER, pbo[index]) ); 

GL_CHECK( glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 
            width, height, glFormat, GL_UNSIGNED_BYTE, 0) );

GL_CHECK( glBindBufferARB(type, 0) );