Filling an array concurrently

Question

I am stumbling upon an issue with concurrency and arrays in Swift 5. To reproduce the issue I have simplified my code to the following fragment:

import Dispatch

let group = DispatchGroup()
let queue = DispatchQueue(
  label: "Concurrent threads",
  qos: .userInitiated,
  attributes: .concurrent
)

let threadCount = 4
let size = 1_000
var pixels = [SIMD3<Float>](
  repeating: .init(repeating: 0),
  count: threadCount*size
)

for thread in 0..<threadCount {
  queue.async(group: group) {
    for number in thread*size ..< (thread+1)*size {
      let floating = Float(number)
      pixels[number] = SIMD3<Float>(floating, floating, floating)
    }
  }
}

print("waiting")
group.wait()
print("Finished")

When I execute this in debug mode using Xcode Version 10.2 beta 4 (10P107d) it always crashes with an error like:

Multithread(15095,0x700008d63000) malloc: *** error for object 0x104812200: pointer being freed was not allocated
Multithread(15095,0x700008d63000) malloc: *** set a breakpoint in malloc_error_break to debug

I have the feeling that this is some bug in the compiler because when I run the code in release mode it runs just fine. Or am I doing something wrong here?

Answer 1

There are pointers inside an Array that absolutely can change under your feet. It is not raw memory.

Arrays are not thread-safe. Arrays are value types, which means that they support copy-on-write in a thread-safe way (so you can freely pass an array to another thread, and if it is copied there, that is ok), but you can't mutate the same array on multiple threads. An Array is not a C buffer. It's not promised to have contiguous memory. It's not even promised to allocate memory at all. Array could choose internally to store "I'm currently all zeros" as a special state and just return 0 for every subscript. (It doesn't, but it's allowed to.)

For this specific problem, you'd typically use vDSP methods like vDSP_vramp, but I understand this is just an example, and there may not be a vDSP method that solves the problem. Typically, though, I'd still focus on Accelerate/SIMD methods rather than dispatching to queues.

But if you are going to dispatch to queues, you'll need an UnsafeMutableBuffer to take control of the memory (and ensure that the memory even exists):

pixels.withUnsafeMutableBufferPointer { pixelsPtr in
    DispatchQueue.concurrentPerform(iterations: threadCount) { thread in
        for number in thread*size ..< (thread+1)*size {
            let floating = Float(number)
            pixelsPtr[number] = SIMD3(floating, floating, floating)
        }
    }
}

The "Unsafe" indicates that it's now your problem to make sure all the accesses are legal and that you're not creating race conditions.

Note the use of .concurrentPerform here. As @user3441734 reminds us, pixelsPtr is not promised to be valid once .withUnsafeMutablePointer completes. .concurrentPerform is guaranteed not to return until all blocks are complete, so the pointer is guaranteed to be valid.

This could be done with DispatchGroup as well, but the .wait would need to be inside of withUnsafeMutableBufferPointer.