How to print each line of a script as it is run only for the top-level script being run?

Question

python's trace module will allow you to run a script printing each line of code as it is run in both the script and all imported modules like so:

 python -m trace -trace myscript.py

Is there a way to do the same thing, but only print the top-level calls, i.e. only print the lines in myscript.py as they are run?

I am trying to debug an abort trap failure and I can't figure out where it's dying. Unfortunately, using the full --trace takes forever - the script normally takes 2-3 minutes to run, and the full trace has been going for hours.

Answer 1

I stumbled into this problem and found grep to be a quick and dirty solution:

python -m trace --trace my_script.py | grep my_script.py

My script runs in finite time though. This probably won't work well for more complex scripts.

Answer 2

Maybe lptrace can be useful for you.

Answer 3

The current accepted answer is good enough if only one file is being used to run all the code but when a whole project of files is being used is insufficient. One way to trace multiple files would be done listing all the files by hand in the following manner:

python -m trace --trace src/main.py | grep "main.py\|file2.py"

For small projects this may suffice but for big projects with a lot of files this may become annoying to list all the necessary file names and can be prone to error. In this manner, ideally you would need to construct a regex for your file names and using find list all the needed files for you to monitor.

Following this idea, I constructed a regex that matches all file names that do not start with a lower dash (to avoid matching on __init__.py files), do not contain any upper case letters in their name nor folder name and have the .py extension. To list all this files in an src folder the match those criteria the command is:

find src/ -type f -regextype sed -regex '\([a-z_]\+/\)\+[a-z][a-z_]\+.py'

Now for using those files in grep we just need to print them with a backward slash and a pipe between them. To print the names in that structure use the following command:

find src/ -type f -regextype sed -regex '\([a-z_]\+/\)\+[a-z][a-z_]\+.py' -printf '%f\\\\|' | awk '{ print substr ($0, 1, length($0)-3)}'

Lastly, we construct the command to match all files with the tracing:

python -m trace --trace src/main.py | grep "`find src/ -type f -regextype sed -regex '\([a-z_]\+/\)\+[a-z][a-z_]\+.py' -printf '%f\\\\|' | awk '{ print substr ($0, 1, length($0)-3)}'`"

This can be used to match filenames in specific folders of the project and the regex can be modified to include other important details that you might have with the names of the files you want to monitor.