NSURLConnection crashing under 10.5.7

Question

I have a little app that downloads stock prices and was working perfectly (for years) until my recent upgrade to 10.5.7. After the upgrade, the program would crash on this call:

NSString *currinfo = [NSString stringWithContentsOfURL:[NSURL URLWithString:[NSString stringWithFormat:@"http://finance.yahoo.com/d/quotes.csv?s=%@&f=l1c1p2", escsymbol]]];

Oddly, the crash doesn't happen right away. This line of code is called many times, with no problems, and then the program eventually fails after 1-2 hours due to a crash on this call.

I originally had a long post here describing my attempts to investigate this problem. I received two suggestions: (i) make the call asynchronous (probably better anyway) and (ii) use NSZombieEnabled to investigate the possibility that an Objective-C object is getting deallocated early (this comment was made in response to stack traces showing failure in objc_msgSend).

I spent a good deal of time making the call asynchronous (using [[NSURLConnection alloc] initWithRequest:theRequest delegate:self]) and this didn't help. The program still failed eventually, usually after 10-15 minutes. During this interval before failure, many asynchronous calls were made without any problem, data was returned, etc. Everything was fine. Then the program crashed suddenly again.

I then turned on NSZombieEnabled. Sure enough, when the program eventually crashed I got the message:

-[CFArray count]: message sent to deallocated instance 0x16b90bd0

"info malloc 0x16b90bd0" then yielded:

0: 0x93db810c in malloc_zone_malloc
1: 0x946bc3d1 in _CFRuntimeCreateInstance
2: 0x9464a138 in __CFArrayInit
3: 0x946cd647 in _CFStreamScheduleWithRunLoop
4: 0x932d1267 in _Z16_scheduleRStreamPKvPv
5: 0x946bf15c in CFSetApplyFunction
6: 0x932b0e2b in CFNSchedulingSetScheduleReadStream
7: 0x9331a310 in _ZN12HTTPProtocol19createAndOpenStreamEv
8: 0x9332e877 in _ZN19URLConnectionLoader24loaderScheduleOriginLoadEPK13_CFURLRequest
9: 0x9332d739 in _ZN19URLConnectionLoader26LoaderConnectionEventQueue33processAllEventsAndConsumePayloadEP20XConnectionEventInfoI12XLoaderEvent18XLoaderEventParamsEl
10: 0x9332dbdd in _ZN19URLConnectionLoader13processEventsEv
11: 0x932d8dbf in _ZN17MultiplexerSource7performEv
12: 0x946ba595 in CFRunLoopRunSpecific
13: 0x946bac78 in CFRunLoopRunInMode
14: 0x9058c530 in +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:]
15: 0x90528e0d in -[NSThread main]
16: 0x905289b4 in __NSThread__main__
17: 0x93de8155 in _pthread_start
18: 0x93de8012 in thread_start

I am no expert in reading stack traces, but doesn't this trace indicate a problem in Apple code, rather than my code? Or could I somehow be responsible for de-allocation of the CFArray in question? Is there any way for me to further investigate the cause of the problem?

---

(Here's the rest of my original post)

Seeing that stringWithContentsOfURL is deprecated, I switched to this code:

pathURL = [NSURL URLWithString:[NSString stringWithFormat:@"http://finance.yahoo.com/d/quotes.csv?s=%@&f=l1c1p2", escsymbol]];

NSURLRequest *request = [NSURLRequest requestWithURL:pathURL cachePolicy:NSURLRequestReturnCacheDataElseLoad timeoutInterval:30.0];

responseData = [ NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];

NSString *currinfo = nil;

if ([error code]) {  dNSLog((@"%@ %d %@ %@ %@", [ error domain], [ error code], [ error localizedDescription], request, @"file://localhost/etc/gettytab"));  }

This didn't help. The program still crashes on the sendSynchronousRequest line after an arbitrary length of time, with this information in the debugger:

0   0x93db7286 in mach_msg_trap
1   0x93dbea7c in mach_msg
2   0x946ba04e in CFRunLoopRunSpecific
3   0x946bac78 in CFRunLoopRunInMode
4   0x932b53eb in CFURLConnectionSendSynchronousRequest
5   0x905dca4b in +[NSURLConnection sendSynchronousRequest:returningResponse:error:]

...etc.

The real crash might actually be in a different thread:

0   libobjc.A.dylib                 0x965c3688 objc_msgSend + 24
1   com.apple.CoreFoundation        0x946cc581 _CFStreamSignalEventSynch + 193
2   com.apple.CoreFoundation        0x946ba595 CFRunLoopRunSpecific + 3141
3   com.apple.CoreFoundation        0x946bac78 CFRunLoopRunInMode + 88
4   com.apple.Foundation            0x9058c530 +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:] + 320
5   com.apple.Foundation            0x90528e0d -[NSThread main] + 45
6   com.apple.Foundation            0x905289b4 __NSThread__main__ + 308
7   libSystem.B.dylib               0x93de8155 _pthread_start + 321
8   libSystem.B.dylib               0x93de8012 thread_start + 34

which I presume is the thread spawned to download the URL. By the way, the error handling code works fine--when I intentionally cause an error by disconnecting from the internet, the error is just reported in the console and the program doesn't crash.

This is incredibly frustrating. I would be very happy to spend as much time as necessary tracking down the problem, but I'm kind of at the limits of my knowledge with gdb and especially with assembly language. I don't know how to find out what is the actual problem for the Foundation code. At first I thought that maybe the autoreleased NSString escsymbol is somehow being deallocated, but sending it a retain message didn't help. If this were the case, how could I prove it?

Is anybody else having this problem?

Answer 1

I think Eugene's reply in this thread is properly describing the problem; after some testing, here is what I've concluded seems to be going on, with hopefully some details to help others stuck with this issue:

Redirecting URLs will periodically cause failures. This happens both in the sync and async usages of NSURLConnection. I've created a test project to track down this bug, and this crash will consistently occur (typically between 25-500 iterations). Running the same test on 10.5.6 or without redirecting URLs does not fail (have run it up to 20,000 iterations).

There are two possible work-arounds:

1. Don't use redirecting URLs:
   Obviously this isn't always possible, but if it is a known URL, you can still use the simple calls (like stringWithContentsOfURL:) and this will work fine. Dennis, in your case, the proper server URL is download.finance.yahoo.com, not finance.yahoo.com, so I believe this would fix your particular problem. Using curl you can see that you get a 301 redirect when you hit the latter address.
2. Use the async calls and implement connection:willSendRequest:redirectResponse:
   If you implement even the most basic handling of this delegate method, everything seems to work. If you leave out this call (and thus let the system use its default implementation), you'll get the crash. For a basic implementation, just return the request that is passed in:

- (NSURLRequest *)connection:(NSURLConnection *)connection 
             willSendRequest:(NSURLRequest *)request 
            redirectResponse:(NSURLResponse *) redirectResponse
{
    return request;
}

All of this seems to suggest to me that there is something broken in Apple's implementation in 10.5.7, but if anyone else has any insights as to what might be going on, please chime in.

I have submitted a bug with my test project to Apple as rdar://6936109 and referenced tjw's report (Radar 6932684).

Answer 2

This seems to be related to redirects. My app downloading about 500 megs of data directly (few hundred separate files) doesn't crash. Same app downloading smaller set of urls, all of which are redirected will randomly crash several times in exactly that point (restart resumes, and restarting over and over again will actually result in successful download).

EDIT: BTW, Colin's suggestion about reimplementing redirects doesn't seem to work for NSURLDownload :(.

EDIT2: Ok, this seems to be a race condition. Adding cerr << "redirect" << endl; in the callback "fixes" it for me with NSURLDownload. Sleeping for 1 second or locking a local static mutex have no effect however...