Capturing game screenshots for use by a Python script

Question

I need to take screenshots of games for a Python script (a bit of machine learning, AI with video games), and I have been led to FRAPS. FRAPS and other similar software can capture screenshots as I want--yet as far as I can tell, I need to know what the files are to be named, and more importantly, I need to either distinguish it from other screenshots or delete these screenshots as soon as I'm done with them. FRAPS' naming system (as well as Bandicam's) requires that I know the time, to the thousandth of a second (as far as I can tell) when the shot was taken. The program will be running in a dynamic, time-sensitive situation, so I want to just have a set naming scheme like I can do in Greenshot (that is, unless it's really easy to find the file being saved, I'm not interested in a convolutional file search during the game, as I might otherwise be open to. Taking FRAPS gameplay videos already lags enough to break the functionality of the script synchronization on the game with a random error). I am both concerned with general system performance during the game, and with the algorithmic responsiveness. After all, I already know the next step is to find an image on the screen or a part of it, which will be taxing.

I would be satisfied with anything --software, module, or code-- which can place an image file (I've been planning to use .png, but any lossless file type will do) into the program directory as the image. Saving it to the clipboard would do just as well, as it can be saved and manipulated from there. I don't have a long-term use for most of these screenshots anyway. It would also be preferable if it can just take screenshots of a section of the screen.

I doubt it matters, but here's some condensed pseudocode of what my program looks like. I don't really need any troubleshooting for it.

    import os,sys,subprocess,mss,mss.tools,time as sleepy,keyboard,sys,pyscreenshot as Grabby
    from random import randint as spinny
    from PIL import Image
    from pyautogui import * # Just to help with what I've been using, a lot of these aren't currently in use
    class nameOfGame:
        magicNumbers = {'whereStuffIs': (500,500),'generalDelay':0.05,'Hotkey':'f10','fileName':'lmno.png'}
        def __init__(self,magicNumbers):
            os.startfile(gameLocation)

        def Play(self):
            if(self.goAhead()):

moveTo(self.magicNumbers['whereStuffIs'])                
click(self.magicNumbers['whereStuffIs']) #How do I format this correctly on StackOverflow?

        def ScreenShot(self):
            if(softWareScreenSshot):# Outside software is getting the screenshot
                press(self.magicNumbers['Hotkey'])
            else: # Python module or other method...?
                PseudoModule.takeAScreenshot()
            load(self.magicNumbers['fileName'])

        def goAhead(self):
            return True # This is a placeholder for AI.

How do people typically get this sort of AI? If at all possible, I don't want to use any higher level coding for this, as I'm mostly doing this as a personal challenge. I've learned so much getting to this point and I don't want a program to do everything for me, just the screenshot. To that end, though, I don't mind getting messy with scripting if there's a way to hodge-podge my way through this problem. However, from what I've read, I think I need to tap into the game while it's running to do this, which may or may not be practical. Thanks in advance!

Answer 1

I see the question is rather old, but still unanswered. You can use the python library mss for grabbing the screen. As I can see you already imported it, but did not use it.

In their documentation is already a sample:

import time
import cv2
import mss
import numpy

with mss.mss() as sct:
  # Part of the screen to capture
  monitor = {"top": 40, "left": 0, "width": 800, "height": 640}

  while "Screen capturing":
      last_time = time.time()
      # Get raw pixels from the screen, save it to a Numpy array
      img = numpy.array(sct.grab(monitor))

      # Display the picture
      cv2.imshow("OpenCV/Numpy normal", img)

      # Display the picture in grayscale
      # cv2.imshow('OpenCV/Numpy grayscale',
      #            cv2.cvtColor(img, cv2.COLOR_BGRA2GRAY))

      print("fps: {}".format(1 / (time.time() - last_time)))

      # Press "q" to quit
      if cv2.waitKey(25) & 0xFF == ord("q"):
          cv2.destroyAllWindows()
          break

As I can see, you also found PyAutoGui. It has a locate function to find areas on the screen, but it's rather slow. They write in the documentation

On a 1920 x 1080 screen, the locate function calls take about 1 or 2 seconds.

I assume with narrowing down the screenshot to the area of interest you can speed things up.

If you need more informations there are many tutorials out there. Here's for example a git repository for using Python to play GTA V. See the links there for related youtube videos and articles.