Determining maximum valid setrecursionlimit value for Python

Question

In the Python 2 documentation, the sys library contains the following (bolded part is my edit):

sys.setrecursionlimit(limit)

Set the maximum depth of the Python interpreter stack to limit. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python.

The highest possible limit is platform-dependent. A user may need to set the limit higher when she has a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash.

What does this mean? Is this just a general "make sure you have enough memory to handle the extra stack space" statement, or is there a specific "per stack frame" size that can be used to calculate the memory value required? What happens to Python when it can't acquire the space?

Answer 1

Why let's find out:

me@host$ docker run -m 4MB --cpuset-cpus=0 -it --rm python:3.5.1
Python 3.5.1 (default, Dec  9 2015, 00:12:22)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, struct
>>> maxint = 2 ** (struct.Struct('i').size * 8 - 1) - 1
>>> sys.setrecursionlimit(maxint)
>>> def goodbye_world():
...  goodbye_world()
...
>>> goodbye_world()
me@host$

Welp. Looks like it crashes Python. Pretty quickly, too, when you only give it 4MB of RAM.