Asynchronous Performance Tests with XCTest

Question

I have started to explore the new XCTest APIs for asynchronous and performance testing. In isolation, the Apple examples from WWMC work well, but I have been unable to figure out how to combine them. The best I have been able to come up with is the following, but I receive the following error when it runs:

API violation - call made to wait without any expectations having been set.

XCTestExpectation *clsQueryReturnedExpectation = [self expectationWithDescription:@"clsQuery returned"];

PFCLSClient *theClient = [[PFCLSClient alloc] init];

[self measureMetrics:@[XCTPerformanceMetric_WallClockTime] automaticallyStartMeasuring:YES forBlock: ^{
   [theClient getStructureOfType:clsImageTypeSVG ForID:idString success: ^(NSDictionary *structureInfo) {
       [clsQueryReturnedExpectation fulfill];
} failure: ^(NSError *error) {
       XCTFail();
       [clsQueryReturnedExpectation fulfill];
}];

   [self waitForExpectationsWithTimeout:5 handler: ^(NSError *error) {
        [self stopMeasuring];
   }];
}];

Has anyone been able to accomplish something similar?

Thx

Answer 1

With some help from Apple, I have a solution. Silly oversight on my part as this is very easy to solve. To get to work, all you need to do is put the creating of the expectation object (clsQueryReturnedExpectation) inside the measureMetrics block so it is created afresh each time the performance test is run.

PFCLSClient *theClient = [[PFCLSClient alloc] init];

[self measureMetrics:@[XCTPerformanceMetric_WallClockTime] automaticallyStartMeasuring:YES forBlock: ^{
    XCTestExpectation *clsQueryReturnedExpectation = [self expectationWithDescription:@"clsQuery returned"];  
    [theClient getStructureOfType:clsImageTypeSVG ForID:idString success: ^(NSDictionary *structureInfo) {
       [clsQueryReturnedExpectation fulfill];
    } failure: ^(NSError *error) {
       XCTFail();
       [clsQueryReturnedExpectation fulfill];
    }];

   [self waitForExpectationsWithTimeout:5 handler: ^(NSError *error) {
        [self stopMeasuring];
   }];
}];