Application sticks on OSSpinLockLockSlow

Question

Update 2: I found a workaround which is to synchronize MOC deallocating and saving. Please see the updated project. https://github.com/shuningzhou/MOCDeadLock.git

Note: I made it fail more aggressively. Don't run it on a real device!

Update: A sample project to demonstrate this issue. https://github.com/shuningzhou/MOCDeadLock.git

XCode 6.2: Unable to reproduce.

XCode 6.3: Reproducible.

XCode 6.4 beta: Reproducible.

========================== The Issue ===============================

Our app randomly stuck on OSSpinLockLockSlow after upgrading to XCode 6.3. In our project, we used NSOperation and NSOperationQueue to fetch data from our server and used Core Data for data persistence.

This issue never happened before! You can see from the stack trace that no calls are made by our code. I am not sure where to start debugging this. Could someone provide some guidance?

Thank you in advance!

Please see the stack trace

Edit:

We are using AFNetworking and our NSOperations are subclasses of AFHTTPRequestOperation. We added some custom properties and overrode the method -(void)start:

- (void)start;
{
    //unrelated code...

    NSString *completionQueueID = [NSString uuid];
    const char *cString = [completionQueueID cStringUsingEncoding:NSASCIIStringEncoding];
    self.completionQueue = dispatch_queue_create(cString, DISPATCH_QUEUE_SERIAL);

    //unrelated code....

    [super start];
}

For Core Data, We are following the thread-confinement pattern. We have separate managed object context for each thread, and the contexts share a static persistent store coordinator.

Edit 2:

More info: I found that this issue happens when the system exits multiple threads at the same time. We store the Managed Object Context in the thread dictionary, and they get released when the threads exit.

[[[NSThread currentThread] threadDictionary] setObject:dataManager forKey:@"IHDataManager"];

CPU usage is around 20%.

Answer 1

I have been experiencing precisely this issue. As per your stack trace, I have a bunch of threads stalled with _OSSpinLockLockSlow.

It appears to be a livelock situation with the spinlocks chained up together. Including some networking threads and core data. But as Rob pointed out, symptoms of livelock should include high CPU usages (spinlocks are all endlessly spinning). In my case (and in yours) this is not the case, CPU usage is low - simulator 'percent used' 20%, simulator overall in activity monitor 0.6% - so maybe it's a deadlock ;-)

Like you, I am using a thread-confinement pattern, separate managed object context per thread, single persistent store.

Following your observation that the hang always seems to follow deallocing of a bunch of threads, I checked that behaviour and can confirm that is the case.

This got me wondering why I had so many threads active. It turned out I was using gcd with a concurrent background queue:

 dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND,0),^{
        modelClass = [WNManagedObject classForID:mongoID];
         dispatch_async(dispatch_get_main_queue(),^{
         ...
         });
        });

This snippet is part of some networking/JSON parsing code. 'classForID' was causing slight UI jitters on the main thread, so I backgrounded it.

In effect the concurrent background queue was spitting out a whole bunch of short-lived threads. This was completely unnecessary. Refactoring as a single serial queue fixed the thread excesses, which got rid of the spinlock issue. Finally I realised I didn't need to get the class at all, so this code has since been exorcised.

Problem fixed, but no explanation as to why this should suddenly become an issue with 8.3

I suspect that the same issue is touched on in this question (although Cocoalumberjack gets the blame there):
syscall_thread_switch iOS 8.3 race - CocoaLumberjack bug? how to debug this?

..and in this Cocoalumberjack bug report
https://github.com/CocoaLumberjack/CocoaLumberjack/issues/494

I am also using CocoaLumberjack but it does not feature in any of the problem threads, so I think that is a red herring. The underlying cause seems to be excess thread creation.

I have seen the issue in the simulator and on devices when tethered to XCode, but I have not experienced it when running independently of XCode. It is new to me in iOS 8.3 / XCode 6.3.1

Not really an answer, more of a diary of my own workaround for this weird issue, but maybe you'll find it useful.

Answer 2

If question is still actual - this is a bug in iOS: OpenRadar crash report
Also you may find this blog post useful: blog post

I think you should replace OSSpinLocks with something else to fix this in your app.

We encountered this bug in our Unity3d game. We didnt fixed this yet in our app because we do not have access to most of the native iOS code (we write our game on C# and we use a lot of 3-rd party native plugins). So I cannot recommend you something concrete about replacing OSSpinLock. Sorry for my English.

Update
Many Apple frameworks and libraries uses OSSpinLock internally, so you dont need to use it explicity to run into this issue.