How to analyze a JMeter summary report?

Question

I get the following result when I run a load test. Can any one help me to read the report?

the number of thread = '500 '
ramp up period = '1'

Sample      =       '500'
Avg     =       '20917'
min         =       '820'
max         =       '48158'
Std Deviation   =       '10563.178194669255'
Error %     =       '0.046'
throughput      =       '10.375381295262601'
KB/Sec          =       `247.05023046315702` 
Avg. Bytes      =       '24382.664'

Answer 1

Short explanation looks like:

1. Sample - number of requests sent
2. Avg - an Arithmetic mean for all responses (sum of all times / count)
3. Minimal response time (ms)
4. Maximum response time (ms)
5. Deviation - see Standard Deviation article
6. Error rate - percentage of failed tests
7. Throughput - how many requests per second does your server handle. Larger is better.
8. KB/Sec - self expalanatory
9. Avg. Bytes - average response size

If you having troubles with interpreting results you could try BM.Sense results analysis service

Answer 2

The JMeter docs say the following:

The summary report creates a table row for each differently named request in your test. This is similar to the Aggregate Report , except that it uses less memory. The thoughput is calculated from the point of view of the sampler target (e.g. the remote server in the case of HTTP samples). JMeter takes into account the total time over which the requests have been generated. If other samplers and timers are in the same thread, these will increase the total time, and therefore reduce the throughput value. So two identical samplers with different names will have half the throughput of two samplers with the same name. It is important to choose the sampler labels correctly to get the best results from the Report.

• Label - The label of the sample. If "Include group name in label?" is selected, then the name of the thread group is added as a prefix. This allows identical labels from different thread groups to be collated separately if required.
• # Samples - The number of samples with the same label
• Average - The average elapsed time of a set of results
• Min - The lowest elapsed time for the samples with the same label
• Max - The longest elapsed time for the samples with the same label
• Std. Dev. - the Standard Deviation of the sample elapsed time
• Error % - Percent of requests with errors
• Throughput - the Throughput is measured in requests per second/minute/hour. The time unit is chosen so that the displayed rate is at least 1.0. When the throughput is saved to a CSV file, it is expressed in requests/second, i.e. 30.0 requests/minute is saved as 0.5.
• Kb/sec - The throughput measured in Kilobytes per second
• Avg. Bytes - average size of the sample response in bytes. (in JMeter 2.2 it wrongly showed the value in kB)

Times are in milliseconds.

Answer 3

A Jmeter Test Plan must have listener to showcase the result of performance test execution.

• Listeners capture the response coming back from Server while Jmeter runs and showcase in the form of – tree, tables, graphs and log files.

• It also allows you to save the result in a file for future reference. There are many types of listeners Jmeter provides. Some of them are: Summary Report, Aggregate Report, Aggregate Graph, View Results Tree, View Results in Table etc.

Here is the detailed understanding of each parameter in Summary report.

By referring to the figure:

image

Label: It is the name/URL for the specific HTTP(s) Request. If you have selected “Include group name in label?” option then the name of the Thread Group is applied as the prefix to each label.

Samples: This indicates the number of virtual users per request.

Average: It is the average time taken by all the samples to execute specific label. In our case, the average time for Label 1 is 942 milliseconds & total average time is 584 milliseconds.

Min: The shortest time taken by a sample for specific label. If we look at Min value for Label 1 then, out of 20 samples shortest response time one of the sample had was 584 milliseconds.

Max: The longest time taken by a sample for specific label. If we look at Max value for Label 1 then, out of 20 samples longest response time one of the sample had was 2867 milliseconds.

Std. Dev.: This shows the set of exceptional cases which were deviating from the average value of sample response time. The lesser this value more consistent the data. Standard deviation should be less than or equal to half of the average time for a label.

Error%: Percentage of Failed requests per Label.

Throughput: Throughput is the number of request that are processed per time unit(seconds, minutes, hours) by the server. This time is calculated from the start of first sample to the end of the last sample. Larger throughput is better.

KB/Sec: This indicates the amount of data downloaded from server during the performance test execution. In short, it is the Throughput measured in Kilobytes per second.

For more information: http://www.testingjournals.com/understand-summary-report-jmeter/