Why does Python have a limit on the number of static blocks that can be nested?

Question

The number of statically nested blocks in Python is limited to 20. That is, nesting 19 for loops will be fine (although excessively time consuming; O(n^19) is insane), but nesting 20 will fail with:

SyntaxError: too many statically nested blocks 

What is the underlying reason for having such a limit? Is there a way to increase the limit?

Answer 1

This limit not only applies to for loops, but to all other control flow blocks as well. The limit for the number of nested control flow blocks is defined inside of code.h with a constant named CO_MAXBLOCKS:

#define CO_MAXBLOCKS 20 /* Max static block nesting within a function */ 

This constant is used to set the maximum size for the stack Python uses to execute exceptions and loops named blockstack. This limit is imposed upon all frame objects and is shown in frameobject.h:

int blockstack[CO_MAXBLOCKS];       /* Walking the 'finally' blocks */ 

The most likely reason for this limit is to keep memory usage at a sane level when executing nested blocks. It's probably similar to the limit Python imposes on recursive calls. This limit can be seen being enforced in compile.c:

if (c->u->u_nfblocks >= CO_MAXBLOCKS) {     PyErr_SetString(PyExc_SyntaxError,                     "too many statically nested blocks");     return 0; } 

A more concrete answer as to why Python has this specfic limit and why they cannot get rid of it, was given by Michael Hudson in a 2004 Python mailing list letter:

Spot on. This has to do with the 'blockstack', very much an internal detail to Python's implementation. We'd like to get rid of it (not because we want to let people write code with more than 20 nested for loops :-) but it's not especially easy (finally: blocks are the biggest problem).

Note that in Python 2.6 and lower, breaking the maximum number of nested loops would've cause a SystemError not a SyntaxError. This was changed however in Python 3 and back-patched to Python 2.7 so a SyntaxError would be raised instead. This was documented in #issue 27514:

Issue #27514: Make having too many statically nested blocks a SyntaxError instead of SystemError.

The reason for this change in exception types was given by Serhiy Storchaka :

[...] SystemError is not an exception that should be raised. SystemError is for errors that can't be occurred in normal case. It should only be caused by incorrect use of C API or hacking Python internals. I think SyntaxError is more appropriate in this case [...].