Exporting a python module (to string or py file)

Question

Summary: I want to take a variable of type 'module' and export it.

I'm importing a python module from a .py file using import and making changes to it. I need to export the module back to a file or obtain a string representation of the complete module that could then be written to disk.

I have been unable to find any way to export a python module or a way to convert objects within the module to strings in structured, plain text python-executable format. (not json, pickling, etc)

Detailed Question & Use Case: This requirement is part of an internal build process; there are no security requirements and only our own modules, and not built in modules, are being modified. A python script runs with business logic to modify a number of other scripts. This process uses information only available at build time. As a result I do not have the option to import a module with varying data at runtime.

The initial system used a template with placeholder strings that would be replaced but the current requirements require more complex modifications to object declarations where programatically modifying the object is far easier than string replacements.

What I've Done With the master generator script, written in python, I can import multiple modules (which have only variable declarations and no executable code) and make all the substitutions that I need. I'm left with a variable of type module that I need to export back to a file to later be executed.

@abarnert had some good ideas. I was unaware of the repr function. That got me the information but without any formatting. This led me to look at pprint which is as close as I've gotten so far to what I'd prefer.

Example example.py

    sample = {
    'level1_dict_1' : {
        'key1' : 'value1',
        'key2' : {
            'level2_dict' : {
                'key1' : 'value3',
                'key2' : ['list1','list2','list3','list4'],
            }
        }
    },
    'level1_dict_2' : {
        'key1' : 'value1',
        'key2' : 'value2',
    },
}

Greatly simplified (and without any business logic applied) I basically want to do the following:

with open("example.py","w") as outfile:
    example = __import__('example') # Import module 
    example.sample['level1_dict_1']['key2']['level2_dict']['key2'][2] = "newlistitem3"  # Change 1 property in a list nested a few levels deep
    outfile.write("sample = \n" + pprint.pformat(example.sample)) # 

I'd love to have the same formatting as my source file but pprint, while readable, has different formatting than I would prefer. This might be as close as I can get to what I need though.

pprint output:

sample = 
{'level1_dict_1': {'key1': 'value1',
                   'key2': {'level2_dict': {'key1': 'value3',
                                            'key2': ['list1',
                                                     'list2',
                                                     'newlistitem3',
                                                     'list4']}}},
 'level1_dict_2': {'key1': 'value1', 'key2': 'value2'}}

EDITS & Clarifications: - My goal is to load a module, modify it, and save it back as an executable python file. That is the reason I'm objecting to pickle,json,etc. I need to produce a single executable py file. - Rewrote use case for clarification - Added examples and more information on things I've tried

Answer 1

What you're asking for isn't possible. You're trying to create source code out of object code. While there are decompiler projects that can do this to a greater or lesser extent, a fully general solution is intractable.

If you just want to get the original source to the module, that's easy. For example, you can use inspect.getsource, then write the resulting string to a file. Or just use inspect.getfile and copy from the resulting pathname.

And you can of course modify the original source while copying it. For example:

source = inspect.getsource(foo)
with open('newfoo.py', 'wb') as f:
    f.write(source)
    f.write('spam = Spam(3)\neggs = Eggs(spam)\n')

But you can't modify foo and then re-generate its source.

---

However, there are probably much better ways to do what you really need to do. For example:

• Use JSON, pickle (mode 0), YAML, etc. Despite what you claim, these are structured, readable plain-text formats, just like Python source.

• Use repr. For string, numeric, and builtin-constant literals, and lists and dicts containing nothing but (recursively) the above types, repr is round-trippable. For example:

---

with open('newfoo.py', 'wb') as f:
    for name, value in foo.__dict__.items():
        f.write('{} = {!r}\n'.format(name, value))

• If you're pretty sure that all of your values are repr-able, but not positive, there are a number of ways to check or sanitize your output before writing it out. For example:

---

with open('newfoo.py', 'wb') as f:
    for name, value in foo.__dict__.items():
        if ast.literal_eval(repr(value)) != value:
            raise ValueError('Tried to save {}'.format(value))
        f.write('{} = {!r}\n'.format(name, value))

• Create data files in some nice format that's ideal for your code generator, human readers, etc. instead of trying to generate Python source, then write simple Python code that reads the data files and generates the objects at runtime.