Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

One could use a profiler, but why not just halt the program? [closed]

If something is making a single-thread program take, say, 10 times as long as it should, you could run a profiler on it. You could also just halt it with a "pause" button, and you'll see exactly what it's doing.

Even if it's only 10% slower than it should be, if you halt it more times, before long you'll see it repeatedly doing the unnecessary thing. Usually the problem is a function call somewhere in the middle of the stack that isn't really needed. This doesn't measure the problem, but it sure does find it.

Edit: The objections mostly assume that you only take 1 sample. If you're serious, take 10. Any line of code causing some percentage of wastage, like 40%, will appear on the stack on that fraction of samples, on average. Bottlenecks (in single-thread code) can't hide from it.

EDIT: To show what I mean, many objections are of the form "there aren't enough samples, so what you see could be entirely spurious" - vague ideas about chance. But if something of any recognizable description, not just being in a routine or the routine being active, is in effect for 30% of the time, then the probability of seeing it on any given sample is 30%.

Then suppose only 10 samples are taken. The number of times the problem will be seen in 10 samples follows a binomial distribution, and the probability of seeing it 0 times is .028. The probability of seeing it 1 time is .121. For 2 times, the probability is .233, and for 3 times it is .267, after which it falls off. Since the probability of seeing it less than two times is .028 + .121 = .139, that means the probability of seeing it two or more times is 1 - .139 = .861. The general rule is if you see something you could fix on two or more samples, it is worth fixing.

In this case, the chance of seeing it in 10 samples is 86%. If you're in the 14% who don't see it, just take more samples until you do. (If the number of samples is increased to 20, the chance of seeing it two or more times increases to more than 99%.) So it hasn't been precisely measured, but it has been precisely found, and it's important to understand that it could easily be something that a profiler could not actually find, such as something involving the state of the data, not the program counter.

like image 287
Mike Dunlavey Avatar asked Nov 05 '08 19:11

Mike Dunlavey


People also ask

How to use profiler in IntelliJ?

It is easy to start profiling your application – just open the Profiler tool window from the bottom toolbar, select a running process, and right-click it to choose which profiler to attach. You can then stop profiling and see the results in the same tool window.

Does IntelliJ have a profiler?

How IntelliJ IDEA profiler works For CPU and allocation profiling, IntelliJ IDEA provides integration with the following profilers: Java Flight Recorder – a standard profiling tool shipped as part of the JDK. Async Profiler – a very accurate profiler that can also collect native call and memory allocation data.

How do I run a performance profiler in Visual Studio?

Open the Performance Profiler by choosing Debug > Performance Profiler (or Alt + F2). For more information on using the CPU Usage or Memory usage tool in the Performance Profiler vs. the debugger-integrated tools, see Run profiling tools with or without the debugger.

What is profiling in Python?

Introduction to the profilers cProfile and profile provide deterministic profiling of Python programs. A profile is a set of statistics that describes how often and for how long various parts of the program executed. These statistics can be formatted into reports via the pstats module.


2 Answers

On Java servers it's always been a neat trick to do 2-3 quick Ctrl-Breakss in a row and get 2-3 threaddumps of all running threads. Simply looking at where all the threads "are" may extremely quickly pinpoint where your performance problems are.

This technique can reveal more performance problems in 2 minutes than any other technique I know of.

like image 75
krosenvold Avatar answered Oct 07 '22 06:10

krosenvold


Because sometimes it works, and sometimes it gives you completely wrong answers. A profiler has a far better record of finding the right answer, and it usually gets there faster.

like image 24
Paul Tomblin Avatar answered Oct 07 '22 06:10

Paul Tomblin