Roundtrip IPython transform_cell

Question

Consider the following example:

>>> from IPython.core.inputtransformer2 import TransformerManager
>>> src = """\
... foo = !ls
... !!ls
... """
>>> TransformerManager().transform_cell(src)
"foo = get_ipython().getoutput('ls')\nget_ipython().getoutput('ls')\n"

Both !ls and !!ls got transformed to get_ipython().getoutput('ls').

If I only saw the output of TransformerManager().transform_cell, would there be a way to retrieve the original source?

Is there a way to tell whether get_ipython().get_output(<some string>) came from !<some string> or !!<some string>?

Answer 1

Let's take a closer look at how TransformerManager works under the hood to recognize ipython special commands:

TransformerManager tries to match each line to one of the following structures:

• magic assignements (%%my ipython magic)
• system assigns (foo = !cmd)
• escaped commands (!cmd and !!cmd)

System assigns

In the case of system assigns, the first ! after the = sign is stripped, and whatever follows is used as the argument of get_ipython().getoutput.

ex: foo = !ls -> foo = get_ipython().getoutput('ls')

Escaped commands

In the case of so-called "escaped commands", that is, system calls without assignements, a difference is made between !cmd, which gets translated to get_ipython().system(cmd) and !!cmd, which becomes get_ipython().getoutput(cmd).

We can therefore figure out the number of exclamation marks used by using some basic regex to detect whether an assignement is present or not:

import re

def get_number_exclamation_marks(transformed_line):
    """
    Takes the output of TransformerManager for one ipython line, and returns the
    number of exclamation marks used in the system call. If no system call
    is present, returns 0.
    """
    if re.search(r"= *get_ipython\(\)\.getoutput\(", transformed_line)\
    or re.search(r"get_ipython\(\)\.system\(", transformed_line):
        return 1
    if re.search(r"get_ipython\(\)\.getoutput\(", transformed_line):
        return 2
    return 0

NB: The case foo=!!cmd is not treated by the above function, as it is invalid ipython syntax.