OpenAI gym: How to get pixels in CartPole-v0

Question

I would like to access the raw pixels in the OpenAI gym CartPole-v0 environment without opening a render window. How do I do this?

Example code:

import gym
env = gym.make("CartPole-v0")
env.reset()
img = env.render(mode='rgb_array', close=True) # Returns None
print(img)
img = env.render(mode='rgb_array', close=False) 
          # Opens annoying window, but gives me the array that I want
print(img.shape)

PS. I am having a hard time finding good documentation for OpenAI gym. Is it just me, or does it simply not exist?

Edit: I don't need to ever open the render video.

Answer 1

I was curious about same so I started looking into the source code and this is what I found.

Open AI uses pyglet for displaying the window and animations.

For showing the animation everything is drawn on to window and then rendered.

And then pyglet stores what is being displayed on to a buffer.

Dummy version of how code is written in open AI

import pyglet
from pyglet.gl import *
import numpy as np

display = pyglet.canvas.get_display()
screen = display.get_screens()
config = screen[0].get_best_config()

pyglet.window.Window(width=500, height=500, display=display, config=config)

# draw what ever you want

#get image from the buffer

buffer = pyglet.image.get_buffer_manager().get_color_buffer()

image_data=buffer.get_image_data()

arr = np.frombuffer(image_data.get_data(),dtype=np.uint8)

print(arr)
print(arr.shape)

output: [0 0 0 ... 0 0 0]
(1000000,)

so basically every image we get is from buffer of what is being displayed on the window. So if we don't draw anything on window we get no image so that window is required to get the image. so you need to find a way such that windows is not displayed but its values are stored in buffer. I know its not what you wanted but I hope it might lead you to a solution.