How to provide both audio data and video data to MediaMux

Question

I'm trying to get video raw data by Preview, get audio raw data by AudioRecord. Then I will send them to MediaCodec(I will set two Codec instance). After that I will send the video data and audio data to MediaMux to get a mp4 file. I have two questions:

1) I've used MediaMux to process the video data before. For video data, the MediaMux process it by frame, but the video record is continuous. How could the MediaMux handle the video and audio in synchronization.

2) I found only of variable for inputbuffer in the function writeSampleData. A nice guy on StackOverflow also provided a demo of MediaMux. But this demo only has one variable for source data. How could this one variable stand both video data and audio data?

Thank you!

Answer 1

I have had some success passing synced audio and video to MediaMuxer. I calculated the number of audio samples that should play for each frame of video: based on the audio sample rate and the video frame rate. I then had a loop that wrote one video frame and one block of audio in each iteration. Use the presentation time to ensure they will be synced on playback. Use the track index from addTrack in calls to writeSampleData to allow writing video and audio to two separate tracks.

Answer 2

In aggregate to answer @robin-royal, in order to encode both video and audio y should call twice the Addtrack method of MediaMuxer, One track index (int) to each one. so then when you call WriteSampleData in the MediaMuxer, the first parameter specifies the track index, e. g. if audioTrackIndex=2 and videoTrackIndex=1, if you call WriteSampleData with the first parameter equals 1, you would be writing video. (Sorry I have no privileges to answer him in the comment) thanks