How to create a code object in python?

Question

I'd like to create a new code object with the function types.CodeType() .
There is almost no documentation about this and the existing one says "not for faint of heart"
Tell me what i need and give me some information about each argument passed to types.CodeType ,
possibly posting an example.

Note:
In normal use cases you will just need the builtin function compile()
You should use types.CodeType() only if you want to create new instructions that couldn't be obtained writing normal source code and that require direct access to bytecode.

Answer 1

–––––––––––
Disclaimer :
Documentation in this answer is not official and may be incorrect.

This answer is valid only for python version 3.x

–––––––––––

In order to create a code object you have to pass to the function CodeType() the following arguments:

CodeType(         argcount,             #   integer         kwonlyargcount,       #   integer         nlocals,              #   integer         stacksize,            #   integer         flags,                #   integer         codestring,           #   bytes         consts,               #   tuple         names,                #   tuple         varnames,             #   tuple         filename,             #   string         name,                 #   string         firstlineno,          #   integer         lnotab,               #   bytes         freevars,             #   tuple         cellvars              #   tuple         ) 

Now i will try to explain what is the meaning of each argument.

argcount
Number of arguments to be passed to the function (*args and **kwargs are not included).

kwonlyargcount
Number of keyword-only arguments.

nlocals
Number of local variables ,
namely all variables and parameters(*args and **kwargs included) except global names.

stacksize The amount of stack (virtual machine stack) required by the code ,
if you want to understand how it works , see official Documentation.

flags
A bitmap that says something about the code object:
1 –> code was optimized
2 –> newlocals: there is a new local namespace(for example a function)
4 –> the code accepts an arbitrary number of positional arguments (*args is used)
8 –> the code accepts an arbitrary number of keyworded arguments (*kwargs is used)
32 –> the code is a generator

othes flags are used in older python versions or are activated to say what is imported from __ future __

codestring
A sequence of bytes representing bytecode instructions
if you want a better understanding , see Documentation (same as above)

consts
A tuple containing literals used by the bytecode (for example pre-computed numbers, tuples,and strings)

names
A tuple containing names used by the bytecode
this names are global variables, functions and classes or also attributes loaded from objects

varnames
A tuple containing local names used by the bytecode (arguments first, then local variables)

filename
It is the filename from which the code was compiled.
It can be whatever you want,you are free to lie about this. ;)

name
It gives the name of the function. Also this can be whatever you want,but be careful:
this is the name shown in the traceback,if the name is unclear,the traceback could be unclear,
just think about how lambdas can be annoying.

firstlineno
The first line of the function (for debug purpose if you compiled source code)

lnotab
A mapping of bytes that correlates bytecode offsets to line numbers.
(i think also this is for debug purpose,there is few documentation about this)

freevars
A tuple containing the names of free variables.
Free variables are variables declared in the namespace where the code object was defined, they are used when nested functions are declared;
this doesn't happen at module level because in that case free variables are also global variables.

cellvars
A tuple containing names of local variables referenced by nested functions.

––––––––––––
Examples :
following examples should clarify the meaning of what has been said above.

Note: in finished code objects attributes mentioned above have the co_ prefix,
and a function stores its executable body in the __code__ attribute

––––––––––––
1st Example

def F(a,b):     global c     k=a*c     w=10     p=(1,"two",3)  print(F.__code__.co_argcount) print(F.__code__.co_nlocals , F.__code__.co_varnames) print(F.__code__.co_stacksize) print(F.__code__.co_flags) print(F.__code__.co_names) print(F.__code__.co_consts) 

Output:

2 5 ('a', 'b', 'k', 'w', 'p') 3 67 ('c' ,) (None, 10, 1, 'two'. 3, (1, 'two', 3)) 

1. there are two arguments passed to this function ("a","b")

2. this function has two parameters("a","b") and three local variables("k","w","p")

3. disassembling the function bytecode we obtain this:

   3         0 LOAD_FAST                0 (a)             #stack:  ["a"]            3 LOAD_GLOBAL              0 (c)             #stack:  ["a","c"]           6 BINARY_MULTIPLY                            #stack:  [result of a*c]           7 STORE_FAST               2 (k)             #stack:  []  4        10 LOAD_CONST               1 (10)            #stack:  [10]          13 STORE_FAST               3 (w)             #stack:  []  5        16 LOAD_CONST               5 ((1, 'two', 3)) #stack:  [(1,"two",3)]          19 STORE_FAST               4 (p)             #stack:  []          22 LOAD_CONST               0 (None)          #stack:  [None]          25 RETURN_VALUE                               #stack:  [] 

   as you can notice chile executing the function we never have more than three elements in the stack (tuple counts as its lenght in this case)

4. flag's value is dec 67 = bin 1000011 = bin 1000000 +10 +1 = dec 64 +2 +1 ,so we understand that

   • the code is optimized(as most of the automatically generated code is)
   • while executing the function bytecode local namespace changes
   • 64? Actually i don't know what is its meaning

5. the only global name that is used in the function is "c" , it is stored in co_names

6. every explicit literal we use is stored in co_consts:

   • None is the return value of the function
   • we explicitly assign the number 10 to w
   • we explicitly assign (1, 'two', 3) to p
   • if the tuple is a constant each element of that tuple is a constant,so 1,"two",3 are constants

––––––––––––
2nd example

ModuleVar="hi"  def F():     FunctionVar=106     UnusedVar=ModuleVar      def G():         return (FunctionVar,ModuleVar)      print(G.__code__.co_freevars)     print(G.__code__.co_names)  F() print(F.__code__.co_cellvars) print(F.__code__.co_freevars) print(F.__code__.co_names) 

Output:

('FunctionVar',) ('ModuleVar',) ('FunctionVar',) () ('print', '__code__', 'co_freevars', 'co_names', 'ModuleVar') 

the meaning of the output is this:

first and second line are printed when F is executed,so they show co_freevars and co_names of G code:
"FunctionVar" is in the namespace of F function,where G was created,
"ModuleVar" instead is a module variable,so it is considered as global.

following three lines are about co_cellvars,co_freevars and co_names attributes of F code:
"FunctionVar" is referenced in the G nested function ,so it is marked as a cellvar,
"ModuleVar" is in the namespace where F was created,but it is a module variable,
so it is not marked as freevar,but it is found in global names.
also the builtin function print is marked in names , and all the names of attributes used in F.

––––––––––––
3rd example

This is a working code object initialization,
this is unuseful but you can do everything you want with this function.

MyCode= CodeType(         0,         0,         0,         3,         64,         bytes([101, 0, 0,    #Load print function                101, 1, 0,    #Load name 'a'                101, 2, 0,    #Load name 'b'                23,           #Take first two stack elements and store their sum                131, 1, 0,    #Call first element in the stack with one positional argument                1,            #Pop top of stack                101, 0, 0,    #Load print function                101, 1, 0,    #Load name 'a'                101, 2, 0,    #Load name 'b'                20,           #Take first two stack elements and store their product                131, 1, 0,    #Call first element in the stack with one positional argument                1,            #Pop top of stack                100, 0, 0,    #Load constant None                83]),         #Return top of stack         (None,),         ('print', 'a', 'b'),         (),         'PersonalCodeObject',         'MyCode',         1,         bytes([14,1]),         (),         () )  a=2 b=3 exec(MyCode) # code prints the sum and the product of "a" and "b" 

Output:

5 6