Profiling for wall-time on Linux

Question

I have an application that I want to profile wrt how much time is spent in various activities. Since this application is I/O intensive, I want to get a report that will summarize how much time is spent in every library/system call (wall time).

I've tried oprofile, but it seems it gives time in terms of Unhalted CPU cycles (thats cputime, not real time)

I've tried strace -T, which gives wall time, but the data generated is huge and getting the summary report is difficult (and awk/py scripts exist for this ?)

Now I'm looking upto SystemTap, but I don't find any script that is close enough and can be modified, and the onsite tutorial didn't help much either. I am not sure if what I am looking for can be done.

I need someone to point me in the right direction. Thanks a lot!

Answer 1

Judging from this commit, the recently released strace 4.9 supports this with:

strace -w -c

They call it "syscall latency" (and it's hard to see from the manpage alone that's what -w does).

Answer 2

Are you doing this just out of measurement curiosity, or because you want to find time-drains that you can fix to make it run faster?

If your goal is to make it run as fast as possible, then try random-pausing. It doesn't measure anything, except very roughly. It may be counter-intuitive, but what it does is pinpoint the code that will result in the greatest speed-up.