Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Profile Python import times

Is there a way to find out which imports are taking the longest in Python? Looking at the output of python -m cProfile <script>, it doesn't seem to include import statements (understandably given potentially huge dependency trees). Initially I thought it did since I saw a row for __import__() calls, but I think this might actually be because code somewhere is explicitly calling it, toy programs with only import statements don't have a row for it.

Right now I'm just using:

start = time.time()
import <module>
print '%s / time: %f' % (<module>, time.time()-start)

around each module, but it doesn't profile it recursively to see which import within an import might be inflating the time.

like image 417
Andrew Nguyen Avatar asked Jan 12 '16 23:01

Andrew Nguyen


People also ask

What does @profile do in Python?

Introduction to the profilers cProfile and profile provide deterministic profiling of Python programs. A profile is a set of statistics that describes how often and for how long various parts of the program executed. These statistics can be formatted into reports via the pstats module.

What is lazy import in Python?

“Lazy” means that the import of a module (execution of the module body and addition of the module object to sys. modules ) should not occur until the module (or a name imported from it) is actually referenced during execution.

How do you profile a Python code using cProfile?

The syntax is cProfile. run(statement, filename=None, sort=-1) . You can pass python code or a function name that you want to profile as a string to the statement argument. If you want to save the output in a file, it can be passed to the filename argument.

Does the order of imports matter in Python?

Import order does not matter. If a module relies on other modules, it needs to import them itself. Python treats each . py file as a self-contained unit as far as what's visible in that file.


1 Answers

This is a totally legitimate question. For instance, it makes sense to try to accelerate cold start for CLI applications. Python 3.7 now offers an option to print the import time: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPROFILEIMPORTTIME

You can either run:

python -X importtime myscript.py

or:

PYTHONPROFILEIMPORTTIME=1 myscript.py

EDIT: to view those results I recommend tuna.

like image 82
Régis B. Avatar answered Oct 22 '22 10:10

Régis B.