Should I use the MP3 or AAC codec for a .mp4 file?

Question

We have an encoding process in place using ffmpeg on Mac OS X. This process will take a source video and a couple files from that: .m3u8 video, .mp4 video and .mp3 audio file.

By default we've used the video from our m3u8 process which is a h264 (via libx264) video with AAC (via libfaac) audio.

We are mostly using these videos on mobile devices (hence the m3u8 files) but we also use the .mp4 files for Android, Windows Phone, etc. More and more we also need to offer these same videos on the web via either a flash player or HTML5 player.

Therefore, we'd like to have the best audio/video codec combo for all these uses ... where I'm confused is to what is "standard" for a .mp4 file?

If the .mp4 uses the mp3 codec then it plays just fine everywhere but QuickTime, in QuickTime the video plays but there is not audio (works just fine in VLC player tho.)

I've been told it's due to how QuickTime uses file extensions to assume information about the video instead of trying to actually get the codec data from the file? This does make some sense, if we encode the same file but use AAC for the audio codec then it works just fine in QuickTime.

So --- what's the "correct" or "ideal" audio/video codec combo --- is it best and safe to use AAC (i.e. will it work on a broad range of devices) even though it's not a "free" codec?

Answer 1

Compatibility issues set aside, you might want to take into account the quality loss induced by transcoding from mp3 to aac. As your original audio is mp3 encoded, you will have a better end result by remuxing the mp3 bitstream instead of transcoding to aac (even if aac is better than mp3 at a given bitrate).

If your workflow requires lowering the bitrate of the original mp3, then my point looses relevance.

Answer 2

AAC is preferred because many modern devices such as tablets and mobile phones might have hardware decoders for it which will result in better battery and lower CPU consumption. Same applies to h264.