Adding watermark bitmap over video in android: 4.3's MediaMuxer or ffmpeg

Question

Here is my scenario:

• Download an avi movie from the web
• Open a bitmap resource
• Overlay this bitmap at the bottom of the movie on all frames in the background
• Save the video on extarnal storage
• The video length is 15 seconds usually

Is this possible to achieve using MediaMuxer ? Any info on the matter is gladly received

I've been looking to http://bigflake.com/mediacodec/#DecodeEditEncodeTest (Thanks @fadden) and it says there:

"Decoding the frame and copying it into a ByteBuffer with glReadPixels() takes about 8ms on the Nexus 5, easily fast enough to keep pace with 30fps input, but the additional steps required to save it to disk as a PNG are expensive (about half a second)"

So having almost 1 sec/frame is not acceptable. From what I am thinking one way would be to save each frame as PNG, open it, add the bitmap overlay on it and then save it. However this would take an enormous time to accomplish.

I wonder if there is a way to do things like this:

1. Open video file from external storage
2. Start decoding it
3. Each decoded frame will be altered with the bitmap overlay in memory
4. The frame is sent to an encoder.

On iOS I saw that there a way to take the original audio + original video + an image and add them in a container and then just encode the whole thing...

Should I switch to ffmpeg ? How stable and compatible is ffmpeg ? Am I risking compatibility issues with android 4.0+ devices ? Is there a way to use ffmpeg to acomplish this ? I am new to this domain and still doing research.

---

Years later edit:
Years have passed since the question and ffmpeg isn't really easy to add to a commercial software in terms of license. How did this evolved? Newer versions of android are more capable on this with the default sdk?

---

Some more time later edit

I got some negative votes for posting info as an answer so I'll edit the original question. Here is a great library which, from my testing does apply watermark to video and does it with progress callback making it a lot easier to show progress to the user and also uses the default android sdks. https://github.com/MasayukiSuda/Mp4Composer-android

This library generate an Mp4 movie using Android MediaCodec API and apply filter, scale, and rotate Mp4.

Sample code, could look like:

new mp4Composer(sourcePath, destinationPath)
        .filter(new GlWatermarkFilter(watermarkBitmap)
        .listener(){
              @Override 
              private void onProgress(double value){}

              @Override 
              private void onCompleted(double value){
                  runOnUiThread( () ->{
                     showSneakbar
                  }
              }

              @Override 
              private void onCancelled(double value){}

              @Override 
              private void onFailed(Exception e){}

        }).start();

Testing on emulator, seems to work fine on android 8+ while on older generates a black video file.However, when testing on real device seems to work.

Answer 1

I don't know much about the MediaMuxer but ffmpeg does support overlaying functionality. FFMPEG has various filters one of them is overlay filter. What I understand is you want to overlay an image (i.e. png) on the video, ffmpeg surely is a useful framework to do this job. You can set the output format you can set the co-ordinates of the image which is to be overplayed.

E.g.

ffmpeg -i input.avi -i logo.png -filter_complex 'overlay=10:main_h-overlay_h-10' output.avi

Above command adds overlays logo.png on the input.avi video file in bottom left corner.

More information about the filters is available at following website,

https://www.ffmpeg.org/ffmpeg-filters.html#overlay-1

If this is a solution to your problem you need the C code equivalent to the above command. You also need to see the performance of the ffmpeg because it a pure software framework.

Hope I have understood your question correctly and this helps.

Answer 2

If you need do this without ffmpeg on Android device:

Start from : https://github.com/google/grafika

The answer on your question between Play video (PlayMovieActivity.java) and Record Gl App (RecordFBOActivity.java) examples.

Steps:

1. Setup mInputWindowSurface as Video Encoder Input Surface.

2. Decode frame from video stream using MoviePlayer as video (external) texture.

3. Draw this video texture on Surface.

4. Draw watermark on the same Surface over video texture.

5. Notify MediaCodec that surface ready for encoding:

   mVideoEncoder.frameAvailableSoon(); 
   mInputWindowSurface.setPresentationTime(timeStampNanos);
   

   and then goto Step 2.

Don't forget to adjust speed of decoding. Just remove SpeedControlCallback which in example set to decode 60 FPS video.

Advantages of this way:

1. Media Codec use hardware decoder/encoder for video processing.

2. You can change bit rate of result video.