Preserving Side Data information for 360 video transcoding using FFmpeg

Question

I am trying to transcode a 360 Video using ffmpeg (to play on Gear VR, for now), mainly to reduce the bitrate (this is a requirement). But the output file seems to remove the "Side Data" that has the 360 video metadata.

Input file ffprobe result (note the Side Data section):

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
Metadata:
major_brand     : mp42
minor_version   : 0
compatible_brands: mp42mp41
creation_time   : 2016-10-28T10:41:42.000000Z
Duration: 00:02:09.56, start: 0.000000, bitrate: 20116 kb/s
Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 2560x1280, 19797 kb/s, 50 fps, 50 tbr, 50k tbn, 100 tbc (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : Alias Data Handler
  encoder         : AVC Coding
Side data:
  spherical: equirectangular (0.000000/0.000000/0.000000) 
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : Alias Data Handler

Commands tried to transcode (values of bitrate and preset are just for testing):

ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0:s:v:0 -write_id3v2 1 -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0:s -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0 -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -y test.mp4

In each case the ffprobe test.mp4 seems to show the missing "Side Data" section:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
creation_time   : 2016-10-28T10:41:42.000000Z
encoder         : Lavf57.71.100
Duration: 00:02:09.57, start: 0.000000, bitrate: 708 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 2560x1280 [SAR 8:9 DAR 16:9], 507 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : SoundHandler

Any help or advice to get ffmpeg to preserve the Side Data would be appreciated

Answer 1

In recent ffmpeg versions the spherical packet side-data is supported but to write it in MP4 you have to set the standard compliance mode to unofficial or experimental:

ffmpeg -i R0010309_er.MP4 -c copy -strict unofficial spherical.mp4

ffprobe spherical.mp4 2>&1 | grep -A1 Side
Side data:
  spherical: equirectangular (0.000000/0.000000/0.000000) 

---

The relevant code in libavformat/movenc.c function mov_write_video_tag:

    if (track->mode == MODE_MP4 && mov->fc->strict_std_compliance <= FF_COMPLIANCE_UNOFFICIAL) {
        AVStereo3D* stereo_3d = (AVStereo3D*) av_stream_get_side_data(track->st, AV_PKT_DATA_STEREO3D, NULL);
        AVSphericalMapping* spherical_mapping = (AVSphericalMapping*)av_stream_get_side_data(track->st, AV_PKT_DATA_SPHERICAL, NULL);

        if (stereo_3d)
            mov_write_st3d_tag(pb, stereo_3d);
        if (spherical_mapping)
            mov_write_sv3d_tag(mov->fc, pb, spherical_mapping);
    }