Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

display cv2.VideoCapture image inside Pygame surface

I'm trying to use opencv (cv2) to stream a webcam feed into a pygame surface object. The problem is the colors aren't displaying correctly. I think it is the type casting, but I'm having trouble understanding the pygame surface documentation to know what it expects.

This code demonstrates what I'm talking about

import pygame
from pygame.locals import *
import cv2
import numpy

color=False#True#False
camera_index = 0
camera=cv2.VideoCapture(camera_index)
camera.set(3,640)
camera.set(4,480)

#This shows an image the way it should be
cv2.namedWindow("w1",cv2.CV_WINDOW_AUTOSIZE)
retval,frame=camera.read()
if not color:
    frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
cv2.flip(frame,1,frame)#mirror the image
cv2.imshow("w1",frame)

#This shows an image weirdly...
screen_width, screen_height = 640, 480
screen=pygame.display.set_mode((screen_width,screen_height))

def getCamFrame(color,camera):
    retval,frame=camera.read()
    if not color:
        frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
    frame=numpy.rot90(frame)
    frame=pygame.surfarray.make_surface(frame) #I think the color error lies in this line?
    return frame

def blitCamFrame(frame,screen):
    screen.blit(frame,(0,0))
    return screen

screen.fill(0) #set pygame screen to black
frame=getCamFrame(color,camera)
screen=blitCamFrame(frame,screen)
pygame.display.flip()

running=True
while running:
    for event in pygame.event.get(): #process events since last loop cycle
        if event.type == KEYDOWN:
            running=False
pygame.quit()
cv2.destroyAllWindows()

The ultimate goal I have is to create a small photo booth application for a DIY wedding next year. I'm new to programming, but I've managed to cobble this together. I was also trying to accomplish this with VideoCapture, which outputs a PIL, which I also couldn't get to work with the surface object. I want to use a pygame surface so I can animate and overlay count-down text, borders, etc.

Update: The issue was that the cv2 function camera.read() returns a BGR image, but the pygame.surfarray expects a RGB image. This is fixed with the line

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

Also, when converting to grayscale, the following code works:

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
frame=cv2.cvtColor(frame,cv2.COLOR_GRAY2RGB)

So, the function getCamFrame should now be

def getCamFrame(color,camera):
    retval,frame=camera.read()
    frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
    if not color:
        frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
        frame=cv2.cvtColor(frame,cv2.COLOR_GRAY2RGB)
    frame=numpy.rot90(frame)
    frame=pygame.surfarray.make_surface(frame)
return frame
like image 245
Adam McNeilly Avatar asked Oct 08 '13 06:10

Adam McNeilly


People also ask

Is display a surface pygame?

This module offers control over the pygame display. Pygame has a single display Surface that is either contained in a window or runs full screen. Once you create the display you treat it as a regular Surface.

Can pygame handle image formats?

The drawing functions are fine if you want to draw simple shapes on the screen, but many games have images (also called sprites). Pygame is able to load images onto Surface objects from PNG, JPG, GIF, and BMP image files.


1 Answers

No the color error lies in here

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)

so for normal screen color u simply change that to

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

That will do because it works for me

like image 101
Transformer Avatar answered Nov 14 '22 22:11

Transformer