FFmpeg hardware acceleration on Raspberry PI

Question

I am building a program that use ffmpeg to stream webcam content over internet. I would like to know if it is possible to use the GPU for the streaming part on the raspberry pi model 3. If yes, how could I implement this on ffmpeg?

Answer 1

You'll need some additional configure options:

• --enable-mmal – Enable Broadcom Multi-Media Abstraction Layer (Raspberry Pi) via MMAL. For hardware decoding of H.264, VC-1, MPEG-2, MPEG-4. As a dependency you'll need the linux-raspberrypi-headers (Arch Linux) or linux-headers-*-raspi2 (Ubuntu) package which provides the required header file mmal.h.

• --enable-omx-rpi – Enable OpenMAX IL code for Raspberry Pi. For hardware encoding of H.264 (encoder is named h264_omx) and MPEG-4 (mpeg4_omx). As a dependency you'll need the libomxil-bellagio (Arch Linux) or libomxil-bellagio-dev (Ubuntu) package which provides the required header file OMX_Core.h.

For Arch Linux users:

Copy the PKGBUILD file for the ffmpeg package (perhaps via the ABS if you prefer). Add the two new configure options shown above, and add the two mentioned packages to the depends line. Compile/install with the makepkg command.

_{Disclaimer: I don't have one of these devices to test any of this. Most of this info was acquired from the FFmpeg configure file.}

Answer 2

The ffmpeg package from apt now comes with hardware codecs enabled so you can just install that using:

sudo apt install ffmpeg

There are a few hardware enabled codecs on the Pi depending on which model you've got. Here's an excerpt from this detailed post/thread on the Raspberry Pi Forum:

Pi0-3 have hardware accelerated decode for H264, MPEG4, H263, and through optional codec licences for MPEG2 and VC1.

Pi4 has the same hardware accelerated decode for H264, but not the other codecs. It also has a separate block for HEVC.

There are a few APIs (v4l2m2m, VAAPI, OMX, MMAL,...) to access the hardware codecs, but the main one is now the Video Memory-To-Memory Interface based h264_v4l2m2m, and there's also the [older] h264_omx OMX based one, and others. For full list of codecs for encode and decode run:

ffmpeg -codecs

Note: If you have changed the gpu_mem setting in /boot/config.txt it needs to be greater than 16, otherwise you will get an error with all hardware codecs.