ImageGrab.grab() method is too slow

Question

So i need to get a bunch of screenshots every second, like 5. I am using it to program a bot for a game. However imagegrab method takes like 0.3 seconds, which is terribly slow for me. Even after specifying the bbox values it still takes like 0.3 seconds. I think should mention that I am on a mac. Is there a better way for me

I even tried the os.system("screencapture filename.png") which has a runtime of 0.15-0.2 seconds which is nice but I want to go faster.

Answer 1

Another solution is to use Python MSS.

from mss import mss
from PIL import Image

def capture_screenshot():
    # Capture entire screen
    with mss() as sct:
        monitor = sct.monitors[1]
        sct_img = sct.grab(monitor)
        # Convert to PIL/Pillow Image
        return Image.frombytes('RGB', sct_img.size, sct_img.bgra, 'raw', 'BGRX')

img = capture_screenshot()
img.show()

This function can return screenshots at up to 27 fps on my slow laptop.

Answer 2

Question : what could preform better than ImageGrab.grab() on a Mac

I did a test between mss, pil, and pyscreenshot and measured the average time it took to take a various sized image grabs and reported the time in milliseconds.

Answer

It appears that mss far exceeds the others on a mac. To capture a 800x400 slice of the screen mms takes 15ms whereas the other two take 300-400ms. That's the difference between 66fps for mms or 3fps for the other two.

Code

# !pip install image
# !pip install opencv-python
# !pip install pyscreenshot

import numpy as np
from time import time


resolutions = [
    (0, 0, 100,100),(0, 0, 200,100),
    (0, 0, 200,200),(0, 0, 400,200),
    (0, 0, 400,400),(0, 0, 800,400)
]


import numpy as np
import pyscreenshot as ImageGrab
import cv2


def show(nparray):
    import cv2
    cv2.imshow('window',cv2.cvtColor(nparray, cv2.COLOR_BGR2RGB))
    # key controls in a displayed window
    # if cv2.waitKey(25) & 0xFF == ord('q'):
        # cv2.destroyAllWindows()


def mss_test(shape) :
    average = time()
    import mss
    sct = mss.mss()
    mon = {"top": shape[0], "left": shape[1], "width": shape[2]-shape[1], "height": shape[3]-shape[0]}
    for _ in range(5):
        printscreen =  np.asarray(sct.grab(mon))
    average_ms = int(1000*(time()-average)/5.)
    return average_ms, printscreen.shape




def pil_test(shape) :
    average = time()
    from PIL import ImageGrab
    for _ in range(5):
        printscreen =  np.array(ImageGrab.grab(bbox=shape))
    average_ms = int(1000*(time()-average)/5.)
    return average_ms, printscreen.shape




def pyscreenshot_test(shape):
    average = time()
    import pyscreenshot as ImageGrab
    for _ in range(5):
        printscreen = np.asarray( ImageGrab.grab(bbox=shape) )
    average_ms = int(1000*(time()-average)/5.)
    return average_ms, printscreen.shape


named_function_pair = zip("mss_test,pil_test,pyscreenshot_test".split(","),
    [mss_test,pil_test,pyscreenshot_test])

for name,function in named_function_pair:
    results = [ function(res) for res in resolutions ]
    print("Speed results for using",name)
    for res,result in zip(resolutions,results) :
        speed,shape = result
        print(res,"took",speed,"ms, produced shaped",shape)

Output

Speed results for using mss_test
(0, 0, 100, 100) took 7 ms, produced shaped (200, 200, 4)
(0, 0, 200, 100) took 4 ms, produced shaped (200, 400, 4)
(0, 0, 200, 200) took 5 ms, produced shaped (400, 400, 4)
(0, 0, 400, 200) took 6 ms, produced shaped (400, 800, 4)
(0, 0, 400, 400) took 9 ms, produced shaped (800, 800, 4)
(0, 0, 800, 400) took 15 ms, produced shaped (800, 1600, 4)

Speed results for using pil_test
(0, 0, 100, 100) took 313 ms, produced shaped (100, 100, 4)
(0, 0, 200, 100) took 321 ms, produced shaped (100, 200, 4)
(0, 0, 200, 200) took 334 ms, produced shaped (200, 200, 4)
(0, 0, 400, 200) took 328 ms, produced shaped (200, 400, 4)
(0, 0, 400, 400) took 321 ms, produced shaped (400, 400, 4)
(0, 0, 800, 400) took 320 ms, produced shaped (400, 800, 4)

Speed results for using pyscreenshot_test
(0, 0, 100, 100) took 85 ms, produced shaped (200, 200, 4)
(0, 0, 200, 100) took 101 ms, produced shaped (200, 400, 4)
(0, 0, 200, 200) took 122 ms, produced shaped (400, 400, 4)
(0, 0, 400, 200) took 163 ms, produced shaped (400, 800, 4)
(0, 0, 400, 400) took 236 ms, produced shaped (800, 800, 4)
(0, 0, 800, 400) took 400 ms, produced shaped (800, 1600, 4)

Further Observations

Although all three libraries were sent identical screen areas to grab, both mss and pyscreenshot grabbed physical pixels of the mac screen where as pil grabbed the logical pixels. This only happens if you have your Mac display resolution turned down from its highest resolution. In my case I have a retinal display set to "balanced" which means that each logical pixel is actually 2x2 physical pixels.