Adding to array in parallel

Question

I'm using Grand Central Dispatch to transforms elements of one array into another. I call dispatch_apply on the source array, transform it into zero or more items, then add them to the destination array. This is a simplified example:

let src = Array(0..<1000)
var dst = [UInt32]()

let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
dispatch_apply(src.count, queue) { i in
    dst.append(arc4random_uniform(UInt32(i))) // <-- potential error here
}

print(dst)

I sometimes get an error on the append line. The error is always one of:

1. malloc: *** error for object 0x107508f00: pointer being freed was not allocated
2. fatal error: UnsafeMutablePointer.destroy with negative count
3. fatal error: Can't form Range with end < start

I guess this is due to append not being thread-safe. What did I do wrong and how to fix it?

Answer 1

As you have discovered, Swift may relocate the array for memory efficiency. When you run multiple append, it's bound to happen. My guess is that appending to the array is a very low cost operation compared to your transformation. You can create a serial queue just to extend the dst array:

let src = Array(0..<1000)
var dst = [UInt32]()

let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
let serialQueue = dispatch_queue_create("mySerialQueue", DISPATCH_QUEUE_SERIAL)
let serialGroup = dispatch_group_create()

dispatch_apply(src.count, queue) { i in
    let result = arc4random_uniform(UInt32(i)) // Your long-running transformation here
    dispatch_group_async(serialGroup, serialQueue) {
        dst.append(result)
    }
}

// Wait until all append operations are complete
dispatch_group_wait(serialGroup, DISPATCH_TIME_FOREVER)

Answer 2

In case anyone is interested in Swift 4 (I suppose Swift 3 onwards) solution (slightly different approach):

let source = Array(0..<1000)

// If inside class/struct, make this lazy or execute the block inside init
var destination: [UInt32] = {
    var copy = Array<UInt32>.init(repeating: 0, count: source.count)

    DispatchQueue.concurrentPerform(iterations: source.count) { i in
        // Do stuff to get the i'th element 
        copy[i] = UInt32.random(in: 0..<i) 
    }

    return copy
}()

If you choose to avoid the tie between your array size and the number of concurrent operations:

(e.g. you may choose to do it in two iterations 0..<500 in parallel with 500..<1000)

let iterations = 2  

DispatchQueue.concurrentPerform(iterations: iterations) { index in

    // First iteration would start at 0 whereas second (index = 1) starts at 500
    let start = index * source.count / iterations

    // First and second iterations will have 500 and 1000 upper bounds respectively 
    let end = (index + 1) * source.count / iterations

    for i in start..<end {
        // Do stuff to get the i'th element
        copy[i] = UInt32.random(in: 0..<source.count)
    }
}