Take an input stream from the desktop in OpenCV

Question

I am using Python 3.5.1 and OpenCV 3.0.0.

I am working on a python program that can play games, so it needs to 'see' what is going on, on the screen. How can this be achieved?

import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
    #work with frames here

Is there a int 'a' such that cv2.VideoCapture(a) will take the desktop screen as video input? I tried making it and I followed a rather clumsy approach, I captured screen repeatedly using:

import os
os.system('screencapture test.jpg')

Then opening test.jpg using cv2.imread. This was a very slow approach, on searching online I found this question Screen Capture with OpenCV and Python-2.7 which does the same thing, but more efficiently. But the fact still remains that it is capturing individual screenshots and processing them one by one and not a true video stream. I also found this How to capture the desktop in OpenCV (ie. turn a bitmap into a Mat)? which I think is close to what I am trying but is in C++, if someone can help me convert this to Python, I will highly appreciate it.

The main thing is that the program will be doing something like MarI/O, so speed is a concern, any help is appreciated, go easy on me, I am (relatively) new to OpenCV.

Thanks.

Answer 1

Just an update on this question in case anyone wants a solution.

Taking screenshot can be achieved by using module pyautogui

import pyautogui
import matplotlib.pyplot as plt

image = pyautogui.screenshot()
plt.imshow(image)

If you want to read it as a stream,

while(True):
    image = pyautogui.screenshot()
    #further processing
    if finished:
        break

According to the documentation,

On a 1920 x 1080 screen, the screenshot() function takes roughly 100 milliseconds

So this solution can be used if your application does not demand high fps rate.