DirectShow, Media Foundation, DXVA, what?

Question

I'm tasked with revising an application which uses DirectShow for video rendering. The old application works fine, but it's starting to show age.

As I have a gaming background, I figured I should just approach this as such. After having made a bunch of prototypes using different techniques, I was pretty sure I could pull of video-to-texture rendering and use DirectX to fulfill all the clients his requests.

However, I'm now at the point I have to pick a technique to go with and I couldn't be more lost...

• DirectShow
• Media Foundation
  • Media Session (< Windows 8)
  • Media Engine (> Windows 8)
• DXVA1
• DXVA2
• DXVA-HD

DirectShow is obviously the oldest, but still works pretty well, also mostly because of the HUGE amount of codecs available.

Media Foundation has been introduced with Windows Vista, and had something called "Media Session" up til Windows 8, after which they introduced the "Media Engine". If I'm correct codecs are called MFTs? Are there actually sites that have these available much like free-codecs.com?

On top of these 2 techniques, there is DXVA, which allows you to offload video decoding to the GPU. Currently, I ran into 3 different types. DXVA1 being an implementation that has been around since XP, DXVA2 being the newer API. But what is DXVA-HD? Is it newer than DXVA2 or is it an equivalent name?

I really want to move towards Media Foundation, the out-of-the-box codecs work really well with the formats the client wants to use. All I'm concerned is how easy it is to get codecs for it... Does Windows 8+ default media player still have both DirectShow and MF render paths and swaps them on the fly or is Microsoft completely phasing out DirectShow?

I also can't quite grasp how DXVA is supposed to work. Given I want to basically render videos inside a game, it would only make sense to make all rendering actually happen on the GPU and keep application logic on the CPU, but I can't really find any good examples on how to utilize it using Media Foundation. This is mostly caused by the fact I don't know the difference between DXVA2 and DXVA-HD. Windows 7 SDK came with samples for all 3 implementations...

If someone could answer my questions in bold, correct my misinformation and point me in the direction of a proper example(s) (DirectX + Media Foundation + DXVA-whatever), I would be really grateful! It could be as simple as "combine these 3 examples of the Windows 7 SDK to get what you want).

The target of the whole assignment is to get the application in top-notch form again and I want to absolutely sure I pick the techniques that will last for the next decade like DirectShow already has :P

Answer 1

If I'm correct codecs are called MFTs? Are there actually sites that have these available much like free-codecs.com?

Codecs are typically software items whcih either decompress from compressed format to uncompressed (decoders) or vice versa (encoders). Neither of the APIs operate with exactly codecs and instead define a general interface for a component with input and output, e.g. filter in DirectShow, DirectX Media Object (DMO), or its evolution - Media Foundnation Transform (MFT).

MFT is a base component of Media Foundation: encoders, decoders, effects, scalers, video capture devices - all have MFTs inside and higher level API is built on top of that.

Due to limited interest in Media Foundation overall third party MFTs are a rare thing. While you can find almost anything in DirectShow filter form factor, it is not the case with Media Foundation and MFTs.

But what is DXVA-HD? Is it newer than DXVA2 or is it an equivalent name?

DXVA-HD is a part of DXVA 2.0 family of APIs and covers hardware-accelerated video processing. Available as public low level interface, it is also integrated in DirectShow and Media Foundation stock components.

Does Windows 8+ default media player still have both DirectShow and MF render paths and swaps them on the fly or is Microsoft completely phasing out DirectShow?

Yes, WMP 12 still has both. Windows 10 Movies & TV player is MF only.

Given I want to basically render videos inside a game, it would only make sense to make all rendering actually happen on the GPU and keep application logic on the CPU, but I can't really find any good examples on how to utilize it using Media Foundation.

Both DirectShow and Media Foundation offer stock decoders with DXVA support (primarily for H.264 but not only). I would say that DirectShow implementation is rather inflexible and intended for use with EVR. Media Foundation's is more flexibly implemented and can be used outside of MF pipeline as an MFT. Both however are made to hide all complexity and utilize DXVA decoding automatically behind the scenes.

... and I want to absolutely sure I pick the techniques that will last for the next decade like DirectShow already has

DirectShow is still here as a legacy API, with much more generous offering in samples, code, public information, documentation, third party extensibility. Media Foundation is an intended replacement drawing very limited interest to it. Basically it does not offer much compared to DirectShow, Microsoft did not develop it to cover the feature set of DirectShow - what is often expected from newer API, does not offer useful developer tools. It looks like it was supposed that MF's being a newer thing should sell it on its own, but eventually it did not work out:

... The miracle of DirectShow is that, even though it was written in 1995, that original code still supports the many wild and varied multimedia formats that have been developed since. The latest baseclasses update I'm aware of is from 2009, in the Windows 7 SDK, but even that has only minor modifications from the original release.

[...]

But remember, even 7 years ago, DirectShow was already 15 years old. The baseclasses were released in source form from the start. They've had more than 2 decades of attention paid by programmers who care. The problems were ironed out a long time ago.

[...]

Now, having said all that, it's true that the folks in Redmond consider DirectShow to be a dead technology. They've moved on to Media Foundation, for reasons that utterly escape me. I have YET to find a problem that Media Foundation solves better than DirectShow. Its inherent limitations mean that there are problems it simply cannot solve. I will never understand why they chose to start over, instead of patching whatever they didn't like in DShow. The fundamental concepts are absolutely identical, although the vocabulary is different. It's just a waste of resources.

July 2019 Update

I would like to link a comment from Microsoft's James Dailey regarding on DirectShow related issue, and it also neatly outlines the status of DirectShow overall as of 2019:

Please keep in mind that while DirectShow is technically still supported we are in the process of deprecating this legacy technology. We ask that you please move to MediaFoundation as soon as possible.