Making a video with opencv and ffmpeg. How to find the right color format?

Question

I have a webcam video recorder program built with python, opencv and ffmpeg

It works ok except that the color of the video is more blue than the reality. The problem seems to come from color format of images.

It seems that OpenCv is giving BGR images and ffmpeg+libx264 is expecting YUV420p. I've read that YUV420p correspond to YCbCr.

opencv has no conversion from BGR to YCbCr. It only has a conversion to YCrCb.

I have made some searchs and tried different alternatives to try converting opencv image to something that could be ok for ffmpeg+libx264. None is working. At this point, I am a bit lost and I would appreciate any pointer that could help me to fix this color issue.

Answer 1

You are right, the default pixel format of OpenCV is BGR.

The equivalent format on the ffmpeg side would be BGR24, so you don't need to convert it to YUV420p if you don't want to.

This post shows how to use a python application to capture frames from the webcam and write the frames to stdout. The purpose is to invoke this app on the cmd-line and pipe the result directly to the ffmpeg application, which stores the frames on the disk. Quite clever indeed!

capture.py:

import cv, sys

cap = cv.CaptureFromCAM(0)
if not cap:
    sys.stdout.write("failed CaptureFromCAM")

while True :
    if not cv.GrabFrame(cap) : 
        break

    frame = cv.RetrieveFrame(cap)
    sys.stdout.write( frame.tostring() )

And the command to be executed on the shell is:

python capture.py | ffmpeg -f rawvideo -pix_fmt bgr24 -s 640x480 -r 30 -i - -an -f avi -r 30 foo.avi

Where:

• -r gives the frame rate coming off the camera
• -an says "don't encode audio"

I tested this solution on my Mac OS X with OpenCV 2.4.2.

EDIT:

In case you haven't tried to record from the camera and use OpenCV to write the video to an mp4 file on the disk, here we go:

import cv, sys

cap = cv.CaptureFromCAM(0)   # 0 is for /dev/video0
if not cap:
    sys.stdout.write("!!! Failed CaptureFromCAM")
    sys.exit(1)

frame = cv.RetrieveFrame(cap)
if not frame: 
    sys.stdout.write("!!! Failed to retrieve first frame")
    sys.exit(1)

# Unfortunately, the following instruction returns 0
#fps = cv.GetCaptureProperty(cap, cv.CV_CAP_PROP_FPS)
fps = 25.0      # so we need to hardcode the FPS
print "Recording at: ", fps, " fps"  

frame_size = cv.GetSize(frame)
print "Video size: ", frame_size  

writer = cv.CreateVideoWriter("out.mp4", cv.CV_FOURCC('F', 'M', 'P', '4'), fps, frame_size, True)
if not writer:
    sys.stdout.write("!!! Error in creating video writer")
    sys.exit(1)


while True :
    if not cv.GrabFrame(cap) : 
        break
    frame = cv.RetrieveFrame(cap)
    cv.WriteFrame(writer, frame)

cv.ReleaseVideoWriter(writer)
cv.ReleaseCapture(cap)

I've tested this with Python 2.7 on Mac OS X and OpenCV 2.4.2.

Answer 2

Have you tried switching the Cb/Cr channels in OpenCV using split and merge ?