Frame from video is upside down after extracting

Question

My problem here is that when I extracting a video into a frame using opencv, sometimes the frame that I get will flip up which happened to me for both my machine(window) and VM(ubuntu) But some of the video I tested, frames are not flip. So, I wonder what factor or what should be changed/added in my code to make the extract fixed without a flip

def extract_frame(video,folder):
   global fps

   os.mkdir('./green_frame/{folder}/'.format(folder=folder))
   vidcap = cv2.VideoCapture(video)
   success,image = vidcap.read()
   fps = vidcap.get(cv2.CAP_PROP_FPS)
   count = 0
   success = True
   while success:  #os.path.join(pathOut,(name+'.png'))
      cv2.imwrite(os.path.join('./green_frame/{folder}/'.format(folder=folder),"frame%d.png" % count), image)
      success,image = vidcap.read()
      print('Read a new frame: ', success) 
      count += 1

This is the example of frame I get from this code. Which my orginal video that I used is upside down like this:

So, in my case, what I have to changed to make it not flip like my first picture. Is it relate to the resolution or framerate of the video? I tested with a 1280x720 resolution video and all of the frame extracted are flipped upside down but a frame from video with a 568x320 is normal

Thank you

Edit: So, I look at the information of the video and I found out that in the metadata, it has rotate 180 for the video that extract to an upside down frame But when I check with a normal video that produced a non upside-down frame, it does not have rotate:180

So from this, how can I deal with a video that has a rotation?

Answer 1

For anyone still looking into this, I was just stuck on the same problem. Turns out some Android phones and iPhones take images/frames in landscape and convert them on the fly according to the exif 'rotate' tag to display the images/frames.

Weird design choice in OpenCV is that cv2.imread(img_file) already reads the image in correct orientation by reading the image's rotate tag, but the cv2.VideoStream's read() method does not do this.

So, to fix this I used ffmpeg to read the 'rotate' tag and rotate the video frame to its correct orientation.(Big thanks to the comments above, for pointing me in the right direction 👍)

Following is the code:

• Make sure you have ffmpeg for python. (pip install ffmpeg-python)

• Create a method to check if rotation is required by the video_file:

   import ffmpeg    
  
   def check_rotation(path_video_file):
       # this returns meta-data of the video file in form of a dictionary
       meta_dict = ffmpeg.probe(path_video_file)
  
       # from the dictionary, meta_dict['streams'][0]['tags']['rotate'] is the key
       # we are looking for
       rotateCode = None
       if int(meta_dict['streams'][0]['tags']['rotate']) == 90:
           rotateCode = cv2.ROTATE_90_CLOCKWISE
       elif int(meta_dict['streams'][0]['tags']['rotate']) == 180:
           rotateCode = cv2.ROTATE_180
       elif int(meta_dict['streams'][0]['tags']['rotate']) == 270:
           rotateCode = cv2.ROTATE_90_COUNTERCLOCKWISE
  
       return rotateCode
  

• Create a method to correct the rotation of the frame in video file:

   def correct_rotation(frame, rotateCode):  
       return cv2.rotate(frame, rotateCode) 
  

• Finally, do this in your main loop:

   # open a pointer to the video file stream
   vs = cv2.VideoCapture(video_path)
  
   # check if video requires rotation
   rotateCode = check_rotation(video_path)
  
   # loop over frames from the video file stream
   while True:
       # grab the frame from the file
       grabbed, frame = vs.read()
  
       # if frame not grabbed -> end of the video
       if not grabbed:
           break
  
       # check if the frame needs to be rotated
       if rotateCode is not None:
           frame = correct_rotation(frame, rotateCode)
  
       # now your logic can start from here
  

Hope this helps 🍻

Answer 2

The rotate tag is optional so the check_rotation will fail, This code fix it:

def check_rotation(path_video_file):
    # this returns meta-data of the video file in form of a dictionary
    meta_dict = ffmpeg.probe(path_video_file)
    # from the dictionary, meta_dict['streams'][0]['tags']['rotate'] is the key
    # we are looking for
    rotate_code = None
    rotate = meta_dict.get('streams', [dict(tags=dict())])[0].get('tags', dict()).get('rotate', 0)
    return round(int(rotate) / 90.0) * 90