Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a limit to the size of a Python module?

Is there a limit to the size of a Python module?

It seems to me that the Python bytecode instruction POP_JUMP_IF_FALSE takes a 1-byte operand, telling it the instruction index to jump to.

Quoting some of the relevant CPython code from ceval.c (comment mine):

case TARGET(POP_JUMP_IF_FALSE): {
    PREDICTED(POP_JUMP_IF_FALSE);
    PyObject *cond = POP();
    int err;
    if (cond == Py_True) {
        Py_DECREF(cond);
        FAST_DISPATCH();
    }
    if (cond == Py_False) {
        Py_DECREF(cond);
        JUMPTO(oparg);  # <--- this
        FAST_DISPATCH();
    }

Does this mean a Python module cannot contain more than 255 bytecode instructions? What am I missing here?

like image 912
Aviv Cohn Avatar asked Apr 20 '19 19:04

Aviv Cohn


1 Answers

Note: I am no expert in Python and definitely not in interpreting bytecode, this is just what I found after experimenting a while.

Note: I am using Python 3.7.3 if you use a different version you might get different disassembly output (credit goes to @dunes for pointing this out).

# module.py
x = 0
while True:
  if x == 0:
    continue

Will produce the following instructions: (via python3 -m dis module.py)

  1           0 LOAD_CONST               0 (0)
              2 STORE_NAME               0 (x)

  2           4 SETUP_LOOP              14 (to 20)

  3     >>    6 LOAD_NAME                0 (x)
              8 LOAD_CONST               0 (0)
             10 COMPARE_OP               2 (==)
             12 POP_JUMP_IF_FALSE        6

  4          14 JUMP_ABSOLUTE            6
             16 JUMP_ABSOLUTE            6
             18 POP_BLOCK
        >>   20 LOAD_CONST               1 (None)
             22 RETURN_VALUE

At offset 12 is the POP_JUMP_IF_FALSE instruction. After adding a whole bunch of code at the top of the file (I just repeated x = 0 many times):

271        1080 SETUP_LOOP              20 (to 1102)

272     >> 1082 LOAD_NAME                0 (x)
           1084 LOAD_CONST               0 (0)
           1086 COMPARE_OP               2 (==)
           1088 EXTENDED_ARG             4
           1090 POP_JUMP_IF_FALSE     1082

273        1092 EXTENDED_ARG             4
           1094 JUMP_ABSOLUTE         1082
           1096 EXTENDED_ARG             4
           1098 JUMP_ABSOLUTE         1082
           1100 POP_BLOCK
        >> 1102 LOAD_CONST               1 (None)
           1104 RETURN_VALUE

The compiler added a EXTENDED_ARG instruction at offset 1088 which allows for a bigger operand.

like image 153
asynts Avatar answered Oct 13 '22 23:10

asynts