Write seekable AAC audio stream into MP4 file with FFMPEG

Question

I'm trying to write an AAC audio stream into an mp4 file using the FFMPEG libraries. I am using a custom IO context that writes directly to a socket so I have to set ioContext->seekable = 0. To make this work I had to add the "movflags" empty_moov and frag_keyframe when writing the header.

After writing the output to a file on the other end of the socket, I can play the file in VLC or Windows Media Player. However, seeking to a specific position in the file is not working properly in both players. WMP also does not show the total duration and VLC only flashes it shortly when reaching the end of the audio.

Is there a way to add more metadata when muxing so the players are able to treat the file as if it was not written as a stream? Transfer via the socket is not interrupted abruptly, so I could write metadata at the end of the file. I also know the total duration in advance, so I could add it to the header of the file if it was possible. I cannot use the faststart flag because this would require output to a seekable file before writing to the socket.

Update: I learned that I can set the duration in AVFormatContext and I can set nb_frames and avg_frame_rate in AVStream. However, it doesn't solve my problem. When I set the codecContext flag AV_CODEC_FLAG_QSCALE, VLC seems to be able to estimate the total time. However, seeking still doesn't work.

Answer 1

If you want a seekable MP4 file make sure it definately has metadata as that's a neccessity for MPEG decoders to handle seeking (the MP4 metadata lists the start positions of each AAC frame's bytes.
This means lose that -movflags empty_moov & frag_keyframe since they add to the problem.

Consider:

• You don't need frag_keyframe since all audio frames are classed as keyframes.
• Don't force empty_moov since MP4 decoders can't seek if there is no metadata. (FFmpeg wll take care of metadata in fragmentation mode).

A quote from this excellent guide on streaming :

Writing a fragmented file has the advantage that the file is decodable even if the writing is interrupted (while a normal MOV/MP4 is undecodable if it is not properly finished), and it requires less memory when writing very long files (since writing normal MOV/MP4 files stores info about every single packet in memory until the file is closed). The downside is that it is less compatible with other applications.

Options you could try are:

• frag_duration [num] Create fragments that are num milliseconds long.
• frag_size [num] Create fragments that contain up to num bytes size of payload.