Video encoding in ios

Question

So I already know there are existing questions on StackOverflow about streaming live video on ios devices to a server but I'm going to ask about the encoding portion.

I'm a bit lost on which software encoding services are available to encode the raw (live) video footage from an ios device to send to a server like AWS. After using AVCam to capture videos, would I use something like ffmpeg to encode the raw video on the fly and then send the encoded video to the server using HTTP or RTSP? Or do I have the concepts wrong?

Answer 1

As of iOS8 access to the hardware backed encoder/decoder api is available. To get an overview of how this can be done watch the WWDC 2014 session on "Direct Access to Video Encoding and Decoding". In a nutshell you get the pixel buffers from the camera, feed them to the encoder and you get encoded blocks back that you can then feed on in turn to whatever library you are using for the network streaming. You might have to do some data conversion before you can just go and use it. As far as I remember ffmpeg does have rtsp support, so you should be able to use that after getting access to the encoded frames...

Answer 2

The iOS device captures the video in a MOV or MP4.

It is possible to capture raw data (YUV, BGRA etc.) from the iOS device camera using AVFoundation (see this and this and many others).

However it is not efficient to manually encode the raw data since apparently the only way to use hardware-acceleration on the device is by going trough AVAssetWrite which will output a file.

There is a way to achieve live streaming by reading back the file and packetize it using protocols such as RTSP and RTMP, but it's not very straight-forward.

There are a few implementations you can check out like:

http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html for RTSP

and

Streaming support library for the Livu iPhone application for RTMP (older lib for Livu)

The writer of the Livu App is also on StackOverflow, check his Q&A: https://stackoverflow.com/users/227021/steve-mcfarlin