Trying to Understand Asynchronous Operation Subclass

Question

I am trying to get started with using Operations in a side project rather than having closure-based callbacks littered throughout my networking code to help eliminate nested calls. So I was doing some reading on the subject, and I came across this implementation:

open class AsynchronousOperation: Operation {

    // MARK: - Properties

    private let stateQueue = DispatchQueue(label: "asynchronous.operation.state", attributes: .concurrent)

    private var rawState = OperationState.ready

    private dynamic var state: OperationState {
        get {
            return stateQueue.sync(execute: {
                rawState
            })
        }
        set {
            willChangeValue(forKey: "state")
            stateQueue.sync(flags: .barrier, execute: {
                rawState = newValue
            })
            didChangeValue(forKey: "state")
        }
    }

    public final override var isReady: Bool {
        return state == .ready && super.isReady
    }

    public final override var isExecuting: Bool {
        return state == .executing
    }

    public final override var isFinished: Bool {
        return state == .finished
    }

    public final override var isAsynchronous: Bool {
        return true
    }


    // MARK: - NSObject

    private dynamic class func keyPathsForValuesAffectingIsReady() -> Set<String> {
        return ["state"]
    }

    private dynamic class func keyPathsForValuesAffectingIsExecuting() -> Set<String> {
        return ["state"]
    }

    private dynamic class func keyPathsForValuesAffectingIsFinished() -> Set<String> {
        return ["state"]
    }


    // MARK: - Foundation.Operation

    public final override func start() {
        super.start()

        if isCancelled {
            finish()
            return
        }

        state = .executing
        execute()
    }


    // MARK: - Public

    /// Subclasses must implement this to perform their work and they must not call `super`. The default implementation of this function throws an exception.
    open func execute() {
        fatalError("Subclasses must implement `execute`.")
    }

    /// Call this function after any work is done or after a call to `cancel()` to move the operation into a completed state.
    public final func finish() {
        state = .finished
    }
}

@objc private enum OperationState: Int {

    case ready

    case executing

    case finished
}

There are some implementation details of this Operation subclass that I would like some help in understanding.

1. What is the purpose of the stateQueue property? I see it being used by get and set of the state computed property, but I can't find any documentation that explains the sync:flags:execute and sync:execute methods that they use.

2. What is the purpose of the three class methods in the NSObject section that return ["state"]? I don't see them being used anywhere. I found, in NSObject, class func keyPathsForValuesAffectingValue(forKey key: String) -> Set<String>, but that doesn't seem to help me understand why these methods are declared.

Answer 1

You said:

1. What is the purpose of the stateQueue property? I see it being used by get and set of the state computed property, but I can't find any documentation that explains the sync:flags:execute and sync:execute methods that they use.

This code "synchronizes" access to a property to make it thread safe. Regarding why you need to do that, see the Operation documentation, which advises:

Multicore Considerations

... When you subclass NSOperation, you must make sure that any overridden methods remain safe to call from multiple threads. If you implement custom methods in your subclass, such as custom data accessors, you must also make sure those methods are thread-safe. Thus, access to any data variables in the operation must be synchronized to prevent potential data corruption. For more information about synchronization, see Threading Programming Guide.

Regarding the exact use of this concurrent queue for synchronization, this is known as the "reader-writer" pattern. This basic concept of reader-writer pattern is that reads can happen concurrent with respect to each other (hence sync, with no barrier), but writes must never be performed concurrently with respect to any other access of that property (hence async with barrier).

For example, you might implement a reader-writer for thread-safety on an array like so:

class ThreadSafeArray<T> {
    private var values: [T]
    private let queue = DispatchQueue(label: "...", attributes: .concurrent)
    
    init(_ values: [T]) {
        self.values = values
    }
    
    func reader<U>(block: () throws -> U) rethrows -> U {
        return try queue.sync {
            try block()
        }
    }
    
    func writer(block: @escaping (inout [T]) -> Void) {
        queue.async(flags: .barrier) {
            block(&self.values)
        }
    }
    
    // e.g. you might use `reader` and `writer` like the following:
    
    subscript(_ index: Int) -> T {
        get { reader { values[index] } }
        set { writer { $0[index] = newValue } }
    }
    
    func append(_ value: T) {
        writer { $0.append(value) }
    }
    
    func remove(at index: Int) {
        writer { $0.remove(at: index)}
    }
}

Obviously, the use of reader-writer in this Operation subclass is even simpler, but the above illustrates the pattern.

You also asked:

2. What is the purpose of the three class methods in the NSObject section that return ["state"]? I don't see them being used anywhere. I found, in NSObject, class func keyPathsForValuesAffectingValue(forKey key: String) -> Set<String>, but that doesn't seem to help me understand why these methods are declared.

These are just methods that ensure that changes to the state property trigger KVO notifications for properties isReady, isExecuting and isFinished. The KVO notifications of these three keys is critical for the correct functioning of asynchronous operations. Anyway, this syntax is outlined in the Key-Value Observing Programming Guide: Registering Dependent Keys.

The keyPathsForValuesAffectingValue method you found is related. You can either register dependent keys using that method, or have the individual methods as shown in your original code snippet.

---

BTW, here is a revised version of the AsynchronousOperation class you provided, namely:

1. You must not call super.start(). As the start documentation says (emphasis added):

   If you are implementing a concurrent operation, you must override this method and use it to initiate your operation. Your custom implementation must not call super at any time.

2. Add @objc required in Swift 4.

3. Renamed execute to use main, which is the convention for Operation subclasses.

4. It is inappropriate to declare isReady as a final property. Any subclass should have the right to further refine its isReady logic (though we admittedly rarely do so).

5. Use #keyPath to make code a little more safe/robust.

6. You don't need to do manual KVO notifications when using dynamic property. The manual calling of willChangeValue and didChangeValue is not needed in this example.

7. Change finish so that it only moves to .finished state if not already finished.

Thus:

public class AsynchronousOperation: Operation {
    
    /// State for this operation.
    
    @objc private enum OperationState: Int {
        case ready
        case executing
        case finished
    }
    
    /// Concurrent queue for synchronizing access to `state`.
    
    private let stateQueue = DispatchQueue(label: Bundle.main.bundleIdentifier! + ".rw.state", attributes: .concurrent)
    
    /// Private backing stored property for `state`.
    
    private var _state: OperationState = .ready
    
    /// The state of the operation
    
    @objc private dynamic var state: OperationState {
        get { return stateQueue.sync { _state } }
        set { stateQueue.async(flags: .barrier) { self._state = newValue } }
    }
    
    // MARK: - Various `Operation` properties
    
    open         override var isReady:        Bool { return state == .ready && super.isReady }
    public final override var isExecuting:    Bool { return state == .executing }
    public final override var isFinished:     Bool { return state == .finished }
    public final override var isAsynchronous: Bool { return true }

    // KVN for dependent properties
    
    open override class func keyPathsForValuesAffectingValue(forKey key: String) -> Set<String> {
        if ["isReady", "isFinished", "isExecuting"].contains(key) {
            return [#keyPath(state)]
        }
        
        return super.keyPathsForValuesAffectingValue(forKey: key)
    }
    
    // Start
    
    public final override func start() {
        if isCancelled {
            state = .finished
            return
        }
        
        state = .executing
        
        main()
    }
    
    /// Subclasses must implement this to perform their work and they must not call `super`. The default implementation of this function throws an exception.
    
    open override func main() {
        fatalError("Subclasses must implement `main`.")
    }
    
    /// Call this function to finish an operation that is currently executing
    
    public final func finish() {
        if !isFinished { state = .finished }
    }
}

Answer 2

When using an updated code snippet from Rob's answer, one should be aware of possibility of a bug, caused by this change:

7. Change finish so that it only moves to .finished state if isExecuting.

The above goes against Apple docs:

In addition to simply exiting when an operation is cancelled, it is also important that you move a cancelled operation to the appropriate final state. Specifically, if you manage the values for the finished and executing properties yourself (perhaps because you are implementing a concurrent operation), you must update those properties accordingly. Specifically, you must change the value returned by finished to YES and the value returned by executing to NO. You must make these changes even if the operation was cancelled before it started executing.

This will cause a bug in a few cases. For example, if Operation Queue with "maxConcurrentOperationCount = 1" gets 3 async operations A B and C, then if all operations are cancelled during A, C will not get executed and the queue will be stuck on operation B.

Answer 3

About your first question: stateQueue lock your operation when writing a new value to you operation state by:

    return stateQueue.sync(execute: {
            rawState
    })

And

    stateQueue.sync(flags: .barrier, execute: {
        rawState = newValue
    })

as your operation is asynchronous so before read or write one state another state can be called. Like you want to write isExecution but in the mean time isFinished already called. So to avoid this scenario stateQueue lock the operation state to be read and write until it finished its previous call. Its work like Atomic. Rather use dispatch queue you can use an extension to NSLock to simplify executing critical code from Advanced NSOperations sample code in WWDC 2015 https://developer.apple.com/videos/play/wwdc2015/226/ from https://developer.apple.com/sample-code/wwdc/2015/downloads/Advanced-NSOperations.zip and you can implement like following:

private let stateLock = NSLock()

private dynamic var state: OperationState {
    get {
        return stateLock.withCriticalScope{ rawState } 
    }
    set {
        willChangeValue(forKey: "state")

        stateLock.withCriticalScope { 
            rawState = newValue
        }
        didChangeValue(forKey: "state")
    }
}

About your second question: Its a KVO notification for the read only property isReady, isExecuting, isFinished to manage the operation state. You can read this: http://nshipster.com/key-value-observing post till the end for better understanding about KVO.