Custom Wrapper Codec Integration into Android

Question

I need to develop a custom 'wrapper' video codec and integrate it into android (JB for now, ICS later). We want to use some custom decryption keys from the SIM (don't ask!). The best method (that would allow it to work alongside other non-encrypted media and to use the standard media player or other) seems to be to define our own mime-type, and link that to a custom wrapper codec that can do the custom decryption, and then pass the data on to a real codec. (Let's say the filetype is .mp4 for now.)

(An alternative might be to write our own media player, but we'd rather not go down that route because we really want the media to appear seamlessly alongside other media)

I've been trying to follow this guide: how to integrate a decoder into multimedia framework

1. I'm having trouble with OMX Core registration - I can build the libstagefright.so from the android source by typing make stagefright but in the guide he says to use the libstagefrighthw.so which seems appropriate for JB, but I'm not sure how to build this, it doesn't seem to get built from using make stagefright unless I'm doing something wrong?

2. The other problem is that even if I do get the custom wrapper codec registered, I'm not sure how to go about passing the data off to a real codec.

If anyone has any suggestions (or can give some baby step by step instructions!), I'd really appreciate it - the deadline is quite tight for the proof of concept and I know very little about codecs or the media framework...

Many Thanks. (p.s. I don't want to get into a mud fight about drm and analogue holes etc.., thanks)

Answer 1

In this post, I am using H.264 as an example, but the solution(s) can be extended to support other codecs like MPEG-4, VC-1, VP8 etc. There are 2 possible solutions to solve your problem, which I am enlisting below, each with their own pros and cons to help you take an informed decision.

Solution 1: Extend the codec to support new mode

In JellyBean, one could register the same OMX component with same MIME types as 2 different component names viz., OMX.ABC.XYZ and OMX.ABC.XYZ.secure. The former is used for normal playback and is the more commonly used component. The latter is used when the parser i.e. MediaExtractor indicates the presence of secure content. In OMXCodec::Create, after findMatchingCodecs returns a list of codecs, we can observe the choice to select .secure component as here.

Steps to follow:

1. In your platform, register another component with some new extension like OMX.H264.DECODER.decrypt or something similar. Change is required only in media_codecs.xml. The choice of whether to register a new factory method or have a common factory method is your choice.

2. From your parser, when you encounter the specific use-case, set a new flag like kKeyDecryptionRequired. For this you will have to define a new flag in Metadata.h and a corresponding quirk in OMXCodec.h.

3. Modify the OMXCodec::create method to append a .decrypt suffix similar to the .secure suffix as shown above.

4. With all changes in OMXCodec, Metadata, MediaExtractor modules, you will have to rebuild only libstagefright.so and replace the same on your platform.

Voila!! your integration should be complete. Now comes the main challenge inside the component. As part of the component implementation, you should be able to differentiate between an ordinary component creation and .decrypt component creation.

From a runtime perspective, assuming that your component is aware of the fact that it is a .decrypt component or not, you could handle the decryption as part of the OMX_EmptyThisBuffer call, where you could decrypt the data and then pass it to underlying codec.

Pros: Easy to integrate, Minimal changes in Android framework, Scalable to other codecs, No new MIME type registration required.

Cons: You need to track the future revisions of android, specifically on the new quirks, flags and choice of .decrypt extension. If Google decides to employ something similar, you will have to adapt / modify your solution accordingly.

Solution 2: Registration of new MIME Type

From your question, it is not clear if you were able to define the MIME type or not and hence, I am capturing the steps for clarity.

Steps to follow:

1. Register a new MIME type at MediaDefs as shown here. For example, you could employ a new MIME type as const char *MEDIA_MIMETYPE_VIDEO_AVC_ENCRYPT = "video/avc-encrypt";

2. Register your new component with this updated MIME type in media_codecs.xml. Please note that you will have to ensure that the component quirks are also handled accordingly.

3. In OMXCodec::setVideoOutputFormat method implementation, you will have to introduce the support for handling your new MIME type as shown for H.264 here. Please note that you will have to handle similar changes in OMXCodec to support the new MIME type.

4. In MediaExtractor, you will have to signal the MIME type for the video track using the newly defined type. With these changes, your component will be selected and created.

However, the challenge still remains: Where to perform the decryption? For this, you could as well employ the same solution as described in the previous section i.e. handle the same as part of OMX_EmptyThisBuffer call.

Pros: None that I can think of..

Cons: First, solution is not scalable. You will have to keep adding newer MIME types and keep modifying the Stagefright framework. Next, the changes in OMXCodec will require corresponding changes in MediaExtractor. Hence, even though your initial focus is on MP4 extractor, if you wish to extend the solution to other container formats like AVI, MKV, you will have to include the support for new MIME types in these extractors.

Lastly, some notes.

1. As a preferred solution, I would recommend Solution 1 as it is easy and simple.

2. I haven't touched upon ACodec based implementation of the codec. However, I do feel that Solution 1 would be a far more easier solution to implement even if such a support is required in future.

3. If you aren't modifying the OMX core, you shouldn't require to modify the libstagefrighthw.so. Just FYI, this is typically implemented by the vendors as part of their vendor specific modules as in vendor/<xyz>/hardware/.... You need to check with your platform provider on the sources for libstagefrighthw.so.