How do I profile my Perl programs?

Question

I need to improve the performance of my Perl application. How can I find the slow spots?

---

This is a question from the official perlfaq. We're importing the perlfaq to Stack Overflow.

Answer 1

(This is the official perlfaq answer, minus any subsequent edits)

The Devel namespace has several modules which you can use to profile your Perl programs. The Devel::DProf module comes with Perl and you can invoke it with the -d switch:

$ perl -d:DProf program.pl

After running your program under DProf, you'll get a tmon.out file with the profile data. To look at the data, you can turn it into a human-readable report with the dprofpp program that comes with Devel::DProf:

$ dprofpp

You can also do the profiling and reporting in one step with the -p switch to dprofpp:

$ dprofpp -p program.pl

The Devel::NYTProf (New York Times Profiler) does both statement and subroutine profiling. It's available from CPAN and you also invoke it with the -d switch:

$ perl -d:NYTProf some_perl.pl

Like DProf, it creates a database of the profile information that you can turn into reports. The nytprofhtml command turns the data into an HTML report similar to the Devel::Cover report:

$ nytprofhtml

CPAN has several other profilers that you can invoke in the same fashion. You might also be interested in using the C to measure and compare code snippets.

You can read more about profiling in Programming Perl, chapter 20, or Mastering Perl, chapter 5.

perldebguts documents creating a custom debugger if you need to create a special sort of profiler. brian d foy describes the process in The Perl Journal, "Creating a Perl Debugger", and "Profiling in Perl".

Perl.com has two interesting articles on profiling: "Profiling Perl", by Simon Cozens, and "Debugging and Profiling mod_perl Applications", by Frank Wiles.

Randal L. Schwartz writes about profiling in "Speeding up Your Perl Programs" for Unix Review and "Profiling in Template Toolkit via Overriding" for Linux Magazine.

Answer 2

I've switched over to using Devel::NYTProf, which is all the best profiling for Perl combined, initially by the folks at the NYTimes.

Answer 3

There's a very simple way to find the slow spots so you can improve the performance of your program - random-pausing.

Basically, the idea is, rather than measure to see what part is slow, you let its slowness expose it to you.

Run the program with the debug flag -d, and while it's running, interrupt it manually, and display the call stack (T). Do this a few times, like 5 or 10. Look for any statement that appears on more than one stack and that isn't strictly necessary, because the time it is responsible for is roughly the percent of stacks that show it.

This finds not only hotspots, but lines where functions are being called expensively. It works just as well whether the program is I/O or CPU bound, and it doesn't matter what else is going on in the machine.

You can do it more than once, until you can no longer find anything that can be speeded up.