iPhone iOS5 CLGeocoder how to geocode a large (200) set of addresses?

Question

I got a large set of about 200 addresses for which I need to know their latitude and longitude. I've created a method that parses the addresses, and now I'm trying to get coordinates for these addresses using CLGeocoder.

My current approach is to create geocoders in parallel and let them do their magic. I noticed that each one of them seems to take a separate thread. (so I saw up to 100 threads at one point).

The problem that I'm running into is that at some point (after about 50 addresses), geocodes stop returning any place marks, and the

NSLog(@"Address not recognized: *%@*",[htc objectForKey:kAddressKey]);

gets called. Is this a limitation on a number of threads or a built-in CLGeocoder limitation? Could it be that I'm not cleaning up geocoders properly and need some sort of an autorelease statement(ARC)?

-(void)geocodeArray:(NSMutableArray*)array {

    NSMutableDictionary* htc = nil;
    objectsToGeocode = array.count;

    NSDictionary *htcDictionary =nil;
     for (int i = 0; i<array.count;i++) {
         htcDictionary = [array objectAtIndex:i];

        //create an updated dictionary that would hold the reverse geocoding location
        htc = [[NSMutableDictionary alloc] initWithDictionary:htcDictionary];
        NSLog(@"geocoding: %@",[htc objectForKey:kAddressKey]);

        CLGeocoder* geoCoder = [[CLGeocoder alloc] init];
        [geoCoder geocodeAddressString:[htc objectForKey:kAddressKey] completionHandler:^(NSArray *placemarks, NSError *error) {

            if(placemarks.count>0)
            {
                NSLog(@"Found placemarks for %@",[htc objectForKey:kAddressKey]);
                CLPlacemark* placemark =  [placemarks objectAtIndex:0];
                MyLocation *annotation = [[MyLocation alloc]
                                          initWithName:[htcDictionary objectForKey:kNameKey]
                                          address:[htcDictionary objectForKey:kAddressKey]
                                          coordinate:placemark.location.coordinate] ;
                annotation.faxNumber = [htc objectForKey:kFaxKey];
                annotation.phoneNumber = [htc objectForKey:kPhoneKey];
                annotation.website = [htc objectForKey:kWebsiteKey];
                annotation.type = [htc objectForKey:kFacilityTypeKey];
                [_mapView addAnnotation:annotation];  


                double placemarkToUserDistance = [self._mapView.userLocation.location distanceFromLocation:placemark.location] ;
                //convert distance to miles
                placemarkToUserDistance =placemarkToUserDistance/ 1000/ kKilometersPerMile;

                [htc setObject:[NSNumber numberWithDouble:placemarkToUserDistance] forKey:kDistanceToUserKey];
                [htc setObject:[NSNumber numberWithDouble:placemark.location.coordinate.latitude] forKey:kLatitudeKey];
                 [htc setObject:[NSNumber numberWithDouble:placemark.location.coordinate.longitude] forKey:kLongitudeKey];
                NSAssert([htc objectForKey:kLatitudeKey]!=nil,@"kLatitudeKey is not saved!");
                NSAssert([htc objectForKey:kLongitudeKey]!=nil,@"kLongitudeKey is not saved!");

            }else {
                NSLog(@"Address not recognized: *%@*",[htc objectForKey:kAddressKey]);
            }


            [self.dataSource addObject:htc];

            if(++geocodingCount >=objectsToGeocode){
                NSLog(@"%@",self.dataSource);
                    [self saveGeocoding];

            }

        } ];


        //        [temp addObject:htcDictionary];
    }

}

To test if this is a threading issue, I created this method, which splits my large dataset into 5 arrays, and attempts to geocode them in chunks. I noticed that the first request passes, as well as a part of a second one. But once the magic number of (~50) is reached, the geocoding stops.

Any ideas of what may be happening? Is this an Apple imposed limit on the number of geocoding operations? Should I increase the delay between requests or try to run the app 5 separate times and piece together the results by hand?

-(void)geocodeDatasource
{

        //I'm trying to build a file with coordinates of addresses and include it with the app
        geocodingCount = 0;
        self.dataSource = [[NSMutableArray alloc] initWithCapacity:self.arrayForGeocodingInitialJSON.count+5];
        haveToEmailInitialResults = YES;


    //attempt to geocode in batches

     float numberOfArrays = 5.0;
    NSMutableArray* array1 = [[NSMutableArray alloc] initWithCapacity:arrayForGeocodingInitialJSON.count/numberOfArrays];
    NSMutableArray* array2 = [[NSMutableArray alloc] initWithCapacity:arrayForGeocodingInitialJSON.count/numberOfArrays];
    NSMutableArray* array3 = [[NSMutableArray alloc] initWithCapacity:arrayForGeocodingInitialJSON.count/numberOfArrays];
    NSMutableArray* array4 = [[NSMutableArray alloc] initWithCapacity:arrayForGeocodingInitialJSON.count/numberOfArrays];
    NSMutableArray* array5 = [[NSMutableArray alloc] initWithCapacity:arrayForGeocodingInitialJSON.count/numberOfArrays];

    for(int i = 0 ;i<arrayForGeocodingInitialJSON.count;i++)
    {
        id object = [arrayForGeocodingInitialJSON objectAtIndex:i];
        if(i<arrayForGeocodingInitialJSON.count*(1/numberOfArrays))
        {
            [array1 addObject:object];
        }else if(i>=arrayForGeocodingInitialJSON.count/numberOfArrays && i<arrayForGeocodingInitialJSON.count*(2/numberOfArrays))
        {
            [array2 addObject:object];
        }else if(i>=arrayForGeocodingInitialJSON.count*(2/numberOfArrays) && i<arrayForGeocodingInitialJSON.count*(3/numberOfArrays))
        {
            [array3 addObject:object];
        }else if(i>=arrayForGeocodingInitialJSON.count*(3/numberOfArrays) && i<arrayForGeocodingInitialJSON.count*(4/numberOfArrays))
        {
            [array4 addObject:object];
        }else if(i>=arrayForGeocodingInitialJSON.count*(4/numberOfArrays) && i<arrayForGeocodingInitialJSON.count)
        {
            [array5 addObject:object];
        }



    }

    //simple delays eliminate the need for extra variables and notifications
        [self geocodeArray:array2];

        [self performSelector:@selector(geocodeArray:) withObject:array1 afterDelay:15];
        [self performSelector:@selector(geocodeArray:) withObject:array3 afterDelay:30];
        [self performSelector:@selector(geocodeArray:) withObject:array4 afterDelay:45];
        [self performSelector:@selector(geocodeArray:) withObject:array5 afterDelay:45];
}

Thank you!

Answer 1

You can't immediately geocode large sets. iOS throttles you. I have seen that iOS limits you to 50 geocodes at a time, with the "time" factor being an unknown.

I've had a similar problem, but as I needed only to present the geocoding data in a sequence to the user that takes time, I queued all my geocodings.

Effectively, I geocode a chunk of 25 - display the results to the user 1 at a time at about an interval of a half second between each. When I have fewer than 4 left to display, I will geocode the next 25. This continues until everything is geocoded (or in my case, indefinitely).

If you need to have everything geocoded at once, you'll need to chain your geocodings together with delays between each chunk and show some sort of busy indicator until you are done. Which could be some time with large sets of geocodings.