Upper limit on FFMPEG/libx264 frame sizes?

Question

I'm trying to use FFMPEG to transcode a video with very large frames (4096 x 4096). It's a short video - just under two minutes, but the source video file is 15.8GB. (The video is an animation designed to be projected onto planetarium domes.)

The source file is a Quicktime MOV containing uncompressed Photo-JPEG images (with audio, too). When I tell FFMPEG to transcode my MOV file into an MP4 file, it simply says:

Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

Not very helpful, but yes, I've verified that all of the parameters are correct and/or defaults. In fact, if I change the requested output size to 2048x2048, then everything works just fine. That's why I suspect that there's an upper bound in the encoder.

I am kerflummoxed. Any suggestions that result in a successful transcode will be greatly appreciated. (At this point I am also willing to consider alternatives to FFMPEG.)

The Facts

I run FFMPEG from a script:

set srcFile=%1
set dstFile=%srcFile:.mov=.mp4%

ffmpeg -i %srcFile% -movflags faststart -q:a 0 -q:v 0 -pix_fmt yuv420p -sws_flags gauss -s 4096x4096 %dstFile%

The Output

    ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers
    built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --ena
ble-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 19.100 / 52. 19.100
  libavcodec     55.  0.100 / 55.  0.100
  libavformat    55.  0.100 / 55.  0.100
  libavdevice    54.  4.100 / 54.  4.100
  libavfilter     3. 45.103 /  3. 45.103
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'also-sprach-zarathustra-4k.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    creation_time   : 2013-01-11 23:41:30
  Duration: 00:01:40.57, start: 0.000000, bitrate: 1357694 kb/s
    Stream #0:0(eng): Audio: pcm_s16be (twos / 0x736F7774), 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      creation_time   : 2013-01-11 23:41:30
      handler_name    : Apple Alias Data Handler
    Stream #0:1(eng): Video: mjpeg (jpeg / 0x6765706A), yuvj420p, 4096x4096 [SAR 1:1 DAR 1:1], 1356281 kb/s, 30 fps, 30
tbr, 3k tbn, 3k tbc
    Metadata:
      creation_time   : 2013-01-11 23:41:30
      handler_name    : Apple Alias Data Handler
File 'also-sprach-zarathustra-4k.mp4' already exists. Overwrite ? [y/N] y
using SAR=1/1
[libx264 @ 003b9c80] frame MB size (256x256) > level limit (36864)
[libx264 @ 003b9c80] DPB size (4 frames, 262144 mbs) > level limit (2 frames, 184320 mbs)
[libx264 @ 003b9c80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI1 BMI2
x264 [error]: malloc of size 17305600 failed
Output #0, mp4, to 'also-sprach-zarathustra-4k.mp4':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    Stream #0:0(eng): Video: h264, yuv420p, 4096x4096 [SAR 1:1 DAR 1:1], q=-1--1, 90k tbn, 30 tbc
    Metadata:
      creation_time   : 2013-01-11 23:41:30
      handler_name    : Apple Alias Data Handler
    Stream #0:1(eng): Audio: none, 44100 Hz, stereo, s16, 128 kb/s
    Metadata:
      creation_time   : 2013-01-11 23:41:30
      handler_name    : Apple Alias Data Handler
Stream mapping:
  Stream #0:1 -> #0:0 (mjpeg -> libx264)
  Stream #0:0 -> #0:1 (pcm_s16be -> libvo_aacenc)
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

PS: Yes, for those playing at home, it is the theme to 2001 - A Space Odyssey. You can see a 1K YouTube version here: 2001 - A Fulldome Space Odyssey And yes, in a planetarium with a good sound system, it is absolutely EPIC.

Answer 1

I think you're actually running into the limitation of of x264's codec complexity levels.

Brief description of the levels from http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC

As the term is used in the standard, a "level" is a specified set of constraints that indicate a degree of required decoder performance for a profile. For example, a level of support within a profile specifies the maximum picture resolution, frame rate, and bit rate that a decoder may use. A decoder that conforms to a given level must be able to decode all bitstreams encoded for that level and all lower levels.

The max resolution for the highest level (5.2) is 4,096×2,304.

I would look into using HEVC which is specifically designed for these larger format 4K / Ultra-HD resolutions.

http://en.wikipedia.org/wiki/HEVC

Max Resolution for HEVC is currently 8,192x4,320