I'd like my test to fail if it runs slower than 0.5 seconds but the average time is merely printed in the console and I cannot find a way to access it. Is there a way to access this data?
Code
//Measures the time it takes to parse the participant codes from the first 100 events in our test data.
func testParticipantCodeParsingPerformance()
{
var increment = 0
self.measureBlock
{
increment = 0
while increment < 100
{
Parser.parseParticipantCode(self.fields[increment], hostCodes: MasterCalendarArray.getHostCodeArray()[increment])
increment++
}
}
print("Events measured: \(increment)")
}
Test Data
[Tests.ParserTest testParticipantCodeParsingPerformance]' measured [Time, seconds] average: 0.203, relative standard deviation: 19.951%, values: [0.186405, 0.182292, 0.179966, 0.177797, 0.175820, 0.205763, 0.315636, 0.223014, 0.200362, 0.178165]
One solution to slow tests is to trade integration for speed. If the system under test calls for a database query, replace the query with a mock that returns a prefab dataset instead.
Ideally, you'd run all the tests in your test suite for at least 10 minutes, if not longer, and that's too long to run for every commit. If you're like us, by the time you've run a load test for 30 or 60 minutes, you've mentally moved on and have switched contexts. Your head's no longer in the code.
The most common performance testing mistake is to use improper think time and pacing delays. Some either forget to add them or use unrealistic user think time. Many people hit their application with hundreds or thousands of requests per second without any think time and then wonder why the response time is slow.
Slow tests affect development. Engineering teams lose momentum and become frustrated because they can't meet their goals. A slow test suite puts the brakes on CI/CD, making release and deployment more difficult. This often means that organizations can't ship out products on time, and risk losing their competitive edge.
You need to set a baseline for your performance test. Head to the Report Navigator:
and select your recent test run. You'll see a list of all your tests, but the performance ones will have times associated with them. Click the time to bring up the Performance Result popover:
The "Baseline" value is what you're looking for--set it to 0.5s and that will inform Xcode that this test should complete in half a second. If your test is more than 10% slower than the baseline, it'll fail!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With