Is there a fisheye or dual fisheye to equirectangular filter for ffmpeg?

Question

Or a way to do it with an existing filter? So that you could take in video from a fisheye or dual fisheye camera (such as the Ricoh Theta) and directly output equirectangular, in real-time, to something like RTMP?

Answer 1

The Remap filter does just this:

This filter copies pixel by pixel a source frame to a target frame. It remaps the pixels to a new x,y destination based on two files ymap/xmap.

Basic command syntax is

ffmpeg -i fisheye_grid_input.jpg -i fisheye_grid_xmap.pgm -i fisheye_grid_ymap.pgm -filter_complex remap out.png 

Also included at that link are the mapping files for

Ricoh Theta S camera: input files for resolution 1920x960(1080)

Answer 2

In the latest ffmpeg, you can do this to convert fisheye video to equirectangular now

ffmpeg -y -i in.mp4 -vf v360=dfisheye:e:yaw=-90 -c:v libx265 -b:v 40000k -bufsize 5000k -preset ultrafast -c:a copy out.mp4

• y : overwrite output without wanring
• i xxx : input file
• vf yyy: use filter

  • yyy: filter parameters

  • v360 : filter name

    • dfisheye : double fisheye (rectangular image containing two spheres/fisheye); use "fisheye" to use single sphere/fisheye
    • e : abbreviation for "equirectangular"
    • yaw : view direction (=azimut) of center of equirectangular output (=look left/right); use "pitch" to look up/down
    • ih_fov : input horizontal Field Of View; half sphere is 180°, but some cameras arrive to 235°
    • iv_fov : input vertical Field Of View, usually identical to ih_fov
    • h_fov : output horizontal FOV
    • v_fov : output vertical FOV

Docs: https://ffmpeg.org/ffmpeg-filters.html#v360

Note: filter works fine both with image or video as input