Understanding the Page Response Time (range) values in load tests in Visual Studio 2012

Question

I'm writing several Web Tests / Load Tests using Visual Studio 2012 Ultimate.

I have read several MSDN articles (a lot of them actually...) about load/web testing but I'm still a little bit confused when I try to interpret the results.

I still don't understand the difference between the values reported by the Key Indicators graph and the Page Response Time graph

And most of my confusion is reading the values reported by the load test.

I'm confused about the Range column

So according to MSDN

Range: This column displays the y-axis range of the performance counter. By default, this value will automatically adjust as the range of sample data changes. An automatically adjusted range will always be the next power of 10 greater than Max value. This includes negative powers of 10. A graph can contain a variety of counters, each of which has a different range. Therefore, the y-axis is not labeled with any specific range, but is instead labeled with values from 0‒100 that represent a percentage of the total range for each counter. For example, for a counter that has a range of 1,000, a data point of 60 on the y-axis would correspond to a value of 600 for the counter.

So the formula I was using was:

(Range x Value) / 100

To get the real value, and this works fine (I think...)

Well now what I don't understand is why in all the graphs the range appears to be consistent and represented in powers like 1, 10, 100, 1000, 10000....

But in the case of the Page Response Time graph the range is like 29, 20, 23, 5... weird numbers like those, and I cannot change the range on any plot for this graph, however I can change the range for all the other graphs

The values reported on the Page Response Time differ a lot from the ones reported by the Key Indicators graph even when I'm reading the Page response time indicator on both. I understand that on the Key Indicators graph I have the Avg Response Time from all the pages and that on the Page Response Time graph I have the values per page, but even though, adding the values don't match, there's a huge difference from both values and that's why I'm so confused

This is an example of my load tests:

As you can see I have threshold rules to report when the page takes more than 7 seconds to load

What doesn't make any sense to me is that the threshold rule is violated on the Page Response Time graph (because the values reported are bigger than the threshold value) but that does not happen on the Key Indicators graph (here the values make sense). I'm pretty sure the problem is related to the Range column but I don't get it, I don't know how to interpret it

For example if you look at the screenshot, on the Key Indicators graph the first warning icon from left to right reports:

The value 6.30 exceeds the warning threshold value of 6

But on the Page Response Time graph at the same time, the value reported is:

The value 20.9 exceeds the critical threshold value of 7

I have been working on this for several days any help would be truly appreciated

Answer 1

The Avg. Page Time in the Page Response Time section is split out by individual pages. CrmDomainS... is a slow page, having an average of 20 seconds in your screenshot, and it exceeded the threshold soon after the test started. LoggingServic... is an extremely fast page, having an average of just 0.020 seconds.

However, the Avg. Page Time in the Key Indicators section is average over ALL pages. That means it is an average over the slow CrmDomainS... page and the fast LoggingServic... page. This overall average did not exceed the threshold until 1:15. Clearly the fast page is strongly pulling the average down, and thus makes that average value basically meaningless -- useless for any kind of analysis.

If the performance of the LoggingServic... is not especially important, you could consider setting Record Results to False on those requests, which would prevent the statistics from being captured and throwing off the Key Indicators average. In fact if you did this you should then expect to see that the Avg. Page Time would be equal between Key Indicators and Page Response Time! (assuming there aren't any other pages)

The Range column is not related, just a red herring.