CartPole-v0 stuck at a score of exactly 200 [closed]

Question

I was working on CartPole-v0 provided by openai gym. I noticed that my program never scores greater 200. At some point it is stuck at 200 and no more. I was wondering if there is any configuration I might have missed in openai gym. Has anybody found this problem?

Answer 1

CartPole-v0 gives a reward of 1.0 for every step your agent is "alive".

The environment is registered with these lines of code:

register(
    id='CartPole-v0',
    entry_point='gym.envs.classic_control:CartPoleEnv',
    max_episode_steps=200,
    reward_threshold=195.0,
)

which, in the current version of the repository, can be found here.

That max_episode_steps=200 means that an episode automatically terminates after 200 steps. So, the maximum score you can get is 200.