opencv - video looks good but frames are rotated 90 degrees

Question

We save a video on a mobile client and send it to a server. On the server, I use the following code to save the frames:>

import skvideo.io
import cv2

haar = 
'/home/ubuntu/opencv/data/haarcascades/haarcascade_frontalface_alt.xml'
face_cascade = cv2.CascadeClassifier(haar)

ret = True

video = 'my_video.mov'
i = 0
while ret == True:
    cap = skvideo.io.VideoCapture(video)
    ret, frame = cap.read()
    cv2.imwrite('frame_'+str(i)+'.jpg',frame)
    i+=1

When we play the video on a windows media player or itunes, it looks good. I.e. the player knows how to orient it.

But skvideo.io does not know that, and those frames we saved are rotated 90 degrees counterclockwise.

How can we embed info into the video file (a .mov) file that that skvideo knows the correct orientation?

Answer 1

there was a glitch in skvideo, it was not reading the available metadata. For videos taken in mobile are rotated, but metadata includes such parameter. The skvideo team committed a fix, and current skvideo version 1.1.7 reads metadata from mobile, that indicates that video should be rorated. skvideo.io.vread then rotates the file:

1) use newer skvideo version, 1.1.7 which can be cloned at https://github.com/scikit-video/scikit-video

2) You can use following code to read all frames in the video, most likely metadata will be read

import skvideo.io
videogen = skvideo.io.vread(f.name)

That will rotate the video automatically if it was taken in portrait mode.

3) Created an issue on skvideo repo, take a look for further reference: https://github.com/scikit-video/scikit-video/issues/40

Answer 2

It looks like OpenCV does not record the rotation metadata of the video file with VideoCapture() as you can see by the propIds that it stores.

I'm not sure if scikit-video does. It looks like they have a metadata puller called ffprobe which might be able to pull the rotation. See here for an example of how to call and see the output. This shows a hefty list of metadata---no rotation---but that might just be because it's not set or of a movie type which doesn't have rotation metadata.

Another way to grab it would be to read the metadata directly from ffmpeg. I found an old StackOverflow answer that wrote a little python code to extract specifically the rotation metadata from a video using ffmpeg.

Answer 3

I installed 'ffmpeg' ('ffprobe'). My computer is Ubuntu 18.04. Then Python, get_rotation() function worked for me.

Function Code:

import subprocess
import shlex
import json
def get_rotation(self, file_path_with_file_name):
    cmd = "ffprobe -loglevel error -select_streams v:0 -show_entries stream_tags=rotate -of default=nw=1:nk=1"
    args = shlex.split(cmd)
    args.append(file_path_with_file_name)
    ffprobe_output = subprocess.check_output(args).decode('utf-8')
    if len(ffprobe_output) > 0:  # Output of cmdis None if it should be 0
        ffprobe_output = json.loads(ffprobe_output)
        rotation = ffprobe_output
    else:
        rotation = 0
    return rotation