How do you get the Python profiler to work?

Question

I'm trying to follow the instructions here: http://docs.python.org/2/library/profile.html#module-cProfile

Specifically, this part:

import cProfile, pstats, io
pr = cProfile.Profile()
pr.enable()
... do something ...
pr.disable()
s = io.StringIO()
ps = pstats.Stats(pr, stream=s)
ps.print_results()

I've already determined that print_results is not a real method of the Stats class, nor does it seem to really exist anywhere. Here is my current code:

import cProfile, pstats, io
def foo(request):
    pr = cProfile.Profile()
    pr.enable()
    pass
    pr.disable()
    s = io.StringIO()
    ps = pstats.Stats(pr, stream = s)
    f = open('/profstats', 'a')
    ps.print_stats()
    f.write(s.getvalue())
    s.close()
    f.close()

Current result is: TypeError at /inspection-summary/ unicode argument expected, got 'str'

(Output looks like this because I am using Django to call the code in question).

So does anyone know how I can get the profiler to actually, well, work? I just want it to profile like it's supposed to, then print the results to a file so I can view the results later after execution. I can get dump_stats to work, but the file it produces is garbage.

Answer 1

Indeed, the API of the profile/pstats modules look rather ad-hoc. I think the line ps.print_results() is supposed to be a generic one, i.e. it should be written as ps.call_some_methods_to_print_the_result(), but this is not clear indeed. As for dump_stats() it actually saves a binary file that can be reloaded later.

Here is an example that works for me:

import cProfile, pstats
pr = cProfile.Profile()
pr.enable()
...
pr.disable()

f = open('x.prof', 'a')
sortby = 'cumulative'
pstats.Stats(pr, stream=f).strip_dirs().sort_stats(sortby).print_stats()
f.close()

Valid values ofsortby are: calls, cumulative, file, line, module, name, nfl (for name/file/line), pcalls, stdname, time.

Answer 2

The issue in the example in the 2.7 manual seems to be the use of StringIO. When I use a real file as suggested by Armin Rigo, that change avoids the error. Consulting the doc re StringIO I note

The StringIO object can accept either Unicode or 8-bit strings, but mixing the two may take > some care. If both are used, 8-bit strings that cannot be interpreted as 7-bit ASCII (that > > use the 8th bit) will cause a UnicodeError to be raised when getvalue() is called.

getvalue() is not being called here, the statement that fails is in pstats.py, probably the first print attempt in the print_stats() execution with several more to follow:

 print >> self.stream, indent, self.total_calls, "function calls",

I don't see which of the print arguments is causing the problem and I don't see how to get StringIO to accept whatever print_stats is trying to give it -- HOWEVER if you just omit the stream entirely, the output comes out on stdout anyway:

pr.enable()
(do the thing)
pr.disable()
pstats.Stats(pr).print_stats()

If stdout is good enough, that's it.