Using Media Foundation to encode Direct X surfaces

Question

I'm trying to use the MediaFoundation API to encode a video but I'm having problems pushing the samples to the SinkWriter.

I'm getting the frames to encode through the Desktop Duplication API. What I end up with is an ID3D11Texture2D with the desktop image in it.

I'm trying to create an IMFVideoSample containing this surface and then push that video sample to a SinkWriter.

I've tried going about this in different ways:

• I called MFCreateVideoSampleFromSurface(texture, &pSample) where texture is the ID3D11Texture2D, filled in the SampleTime and SampleDuration and then passed the created sample to the SinkWriter.
  SinkWriter returned E_INVALIDARG.

• I tried creating the sample by passing nullptr as the first argument and creating the buffer myself using MFCreateDXGISurfaceBuffer, and then passing the resulting buffer into the Sample.
  That didn't work either.

• I read through the MediaFoundation documentation and couldn't find detailed information on how to create the sample out of a DirectX texture.

I ran out of things to try.
Has anyone out there used this API before and can think of things I should check, or of any way on how I can go about debugging this?

Answer 1

First of all you should learn to use mftrace tool. Very likely, it will tell you the problem right away.

But my guess is, following problems are likely.

1. Probably, some other attributes are required besides SampleTime / SampleDuration.

2. Probably, SinkWriter needs a texture it can read on CPU. To fix that, when a frame is available, create a staging texture of the same format + size, call CopyResource to copy desktop to staging texture, then pass that staging texture to MF.

3. Even if you use a hardware encoder so the CPU never tries to read the texture data, I don’t think it’s a good idea to directly pass your desktop texture to MF.

When you set a D3D texture for sample, no data is copied anywhere, the sample merely retains the texture.

MF works asynchronously, it may buffer several samples in its topology nodes if they want to.

DD gives you data synchronously, you may only access the texture between AcquireNextFrame and ReleaseFrame calls.