Best way to handle multiple NSURL connections

Question

I am trying to create an xls sheet programmatically. To fill the sheet, I am making the multiple NSURLConnection around 100. Right now, my approach is :

1. Make a connection and store the data into an array . This array has 100 objects.
2. Now take the first object and call the connection . Store the data. And make the second connection with 2nd object in the array. This continues till the last object in the array.

It takes on average 14 seconds to finish the 100 connections. Is there any way to implement the NSURLConnection to get the response in a faster way?

Till yesterday I followed the basic approach like:

Declaring the properties:

@property (nonatomic,strong) NSURLConnection *getReportConnection;
@property (retain, nonatomic) NSMutableData *receivedData;
@property (nonatomic,strong) NSMutableArray *reportArray;

Initializing the array in viewDidLoad:

reportArray=[[NSMutableArray alloc]init];

Initializing the NSURLConnection in a button action :

/initialize url that is going to be fetched.
NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"****/%@/crash_reasons",ID]];

//initialize a request from url
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
[request addValue:tokenReceived forHTTPHeaderField:@"**Token"];

[request setHTTPMethod:@"GET"];
[request setValue:@"application/x-www-form-urlencoded" forHTTPHeaderField:@"Content-Type"];

//initialize a connection from request
self.getReportConnection = [[NSURLConnection alloc] initWithRequest:request delegate:self];

Processing the received data:

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData*)data{
if (connection==_getVersionConnection) {

    [self.receivedData_ver appendData:data];

    NSString *responseString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];

    NSError *e = nil;
    NSData *jsonData = [responseString dataUsingEncoding:NSUTF8StringEncoding];

    NSDictionary *JSON = [NSJSONSerialization JSONObjectWithData:jsonData options: NSJSONReadingMutableContainers error: &e];
    [JSON[@"app_versions"] enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop) {
        if (![obj[@"id"] isEqual:[NSNull null]] && ![reportArray_ver containsObject:obj[@"id"]]) {

            [reportArray_ver addObject:obj[@"id"]];

        }
        NSLog(@"index = %lu, Object For title Key = %@", (unsigned long)idx, obj[@"id"]);
    }];

    if (JSON!=nil) {
        UIAlertView *alert=[[UIAlertView alloc]initWithTitle:@"Version Reports succesfully retrieved" message:@"" delegate:self cancelButtonTitle:@"Ok" otherButtonTitles: nil];
        [alert show];
    }
 }

}

Calling the another connection after one finishes:

// This method is used to process the data after connection has made successfully.
- (void)connectionDidFinishLoading:(NSURLConnection *)connection{
   if (connection==getReportConnection) {

             //check and call the connection again
    }
}

And today, I tried the NSURLConnection with sendAsync to fire all the connections one after other using loop,and it worked pretty well.

   self.receivedData_ver=[[NSMutableData alloc]init];
__block NSInteger outstandingRequests = [reqArray count];
 for (NSString *URL in reqArray) {

    NSMutableURLRequest *request=[NSMutableURLRequest requestWithURL:[NSURL URLWithString:URL]
                                                         cachePolicy:NSURLRequestUseProtocolCachePolicy
                                                     timeoutInterval:10.0];

    [request setHTTPMethod:@"GET"];
    [request setValue:@"application/x-www-form-urlencoded" forHTTPHeaderField:@"Content-Type"];


[NSURLConnection sendAsynchronousRequest:request
                                   queue:[NSOperationQueue mainQueue]
                       completionHandler:^(NSURLResponse *response,
                                           NSData *data,
                                           NSError *connectionError) {

                           [self.receivedData appendData:data]; //What is the use of appending NSdata into Nsmutable data? 

                           NSString *responseString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];

                           NSError *e = nil;
                           NSData *jsonData = [responseString dataUsingEncoding:NSUTF8StringEncoding];

                           NSDictionary *JSON = [NSJSONSerialization JSONObjectWithData:jsonData options: NSJSONReadingMutableContainers error: &e];
                           NSLog(@"login json is %@",JSON);

                           [JSON[@"app_versions"] enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop) {


                               if (![obj[@"id"] isEqual:[NSNull null]] && ![reportArray_ver containsObject:obj[@"id"]]) {

                                   [reportArray_ver addObject:obj[@"id"]];

                               }

                               NSLog(@"index = %lu, Object For title Key = %@", (unsigned long)idx, obj[@"id"]);
                           }];


                          outstandingRequests--;

                           if (outstandingRequests == 0) {
                               //all req are finished
                               UIAlertView *alert=[[UIAlertView alloc]initWithTitle:@"Version Reports succesfully retrieved" message:@"" delegate:self cancelButtonTitle:@"Ok" otherButtonTitles: nil];
                               [alert show];
                           }

                       }];
}

This time it took half the time to complete the 100 requests than the old procedure, Is there any faster way exists other than the asynReq?.What is the best scenario to use NSURLconnection and NSURLConnection with asyncReq?

Answer 1

A couple of observations:

1. Use NSURLSession rather than NSURLConnection (if you are supporting iOS versions of 7.0 and greater):

   for (NSString *URL in URLArray) {
       NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:URL];
   
       // configure the request here
   
       // now issue the request
   
       NSURLSessionTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
           // check error and/or handle response here
       }];
       [task resume];
   }
   

2. If you absolutely have to issue 100 requests, then issue them concurrently like your sendAsynchronousRequest implementation (or my dataTaskWithRequest), not sequentially. That's what achieves the huge performance benefit.

   Note, though, that you have no assurances that they'll completely in the order that you issued them, so you will want to use some structure that supports that (e.g. use NSMutableDictionary or pre-populate the NSMutableArray with placeholders so you can simply update the entry at a particular index rather than adding an item to the array).

   Bottom line, be aware that they may not finish in the same order as requested, so make sure you handle that appropriately.

3. If you keep 100 separate requests, I'd suggest that you test this on a really slow network connection (e.g. use the Network Link Conditioner to simulate really bad network connection; see NSHipster discussion). There are problems (timeouts, UI hiccups, etc.) that only appear when doing this on slow connection.

4. Rather than decrementing a counter of number of pending requests, I'd suggest using dispatch groups or operation queue dependencies.

   dispatch_group_t group = dispatch_group_create();
   
   for (NSString *URL in URLArray) {
       dispatch_group_enter(group);
   
       NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:URL];
   
       // configure the request here
   
       // now issue the request
   
       NSURLSessionTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
           // check error and/or handle response here
   
           // when all done, leave group
   
           dispatch_group_leave(group);
       }];
       [task resume];
   }
   
   dispatch_group_notify(group, dispatch_get_main_queue(), ^{
       // do whatever you want when all of the requests are done
   });
   

5. If possible, see if you can refactor the web service so you are issuing one request that returns all of the data. If you're looking for further performance improvement, that's probably the way to do it (and it avoids a lot of complexities involved when issuing 100 separate requests).

6. BTW, if you use delegate based connection, like you did in your original question, you should not be parsing data in didReceiveData. That should only be appending data to a NSMutableData. Do all of the parsing in connectionDidFinishLoading delegate method.

   If you go to block-based implementation, this issue goes away, but just an observation on your code snippets.

Answer 2

Using sendAsynchronous is a great way to improve code organization. I'm sure with some careful scrutiny, we could improve the speed at the margin, but the way to noticeably improve speed is to not make 100 requests.

If the response bodies are small, create an endpoint that answers a conjunction of the results.

If the response bodies are large, then you're requesting more data than the user needs at the moment. Hold up the UI only on what user needs to see, and get the rest silently (... or, maybe better than silently, lazily).

If you don't control the server, and the response bodies are small, and the user needs all or most of to carry on with the app, then you can start working on performance at the margins and UI tricks to amuse user while the app works, but usually one of those constraints -- usually the latter -- can be relaxed.