Open AI Gym Runs in Realtime instead of as fast as possible

Question

Ok so there must be some option in OpenAI gym that allows it to run as fast as possible? I have a linux environment that does exactly this(run as fast as possible), but when I run the exact setup on Windows, it instead runs it only in real-time.

The specific environment I'm working on is in Montezuma's Revenge Atari game. I run the exact same code but on my linux setup, it was able to run the game much faster. Just so you know my linux computer has worse specs than my windows.

here is some code for those who want it:

for i in range(episode_count):
    ob = env.reset()
    ob = np.expand_dims(ob, axis=0)
    time = 0
    while True:
        time += 1
        action = agent.act(ob, reward, done)
        new_ob, reward, done, _ = env.step(action)
        new_ob = np.expand_dims(new_ob, axis=0)
        agent.remember(ob, action, reward, new_ob, done)

        ob = new_ob
        env.render()
        if done or time >= 1000:
            print("episode: {}/{}, time: {}, e: {:.3}"
                  .format(i, episode_count, time, agent.epsilon))
            if len(agent.memory) > batch_size:
                agent.replay(batch_size)
            # agent.save("./save/montazuma-dqn.h5")
            break

The same thing runs on two setups, get different results in run speed.

Answer 1

For anyone looking at this in the future, it's because of self.env.render(). Took me some time to figure out what was slowing my code down. It ends up rendering each action takes time, which will slow your code down