Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Processing vec in parallel: how to do safely, or without using unstable features?

I have a massive vector that I want to be able to load/act on in parallel, e.g. load first hundred thousand indices in one thread, next in another and so on. As this is going to be a very hot part of the code, I have come up with this following proof of concept unsafe code to do this without Arcs and Mutexes:

let mut data:Vec<u32> = vec![1u32, 2, 3];
let head = data.as_mut_ptr();
let mut guards = (0..3).map(|i|
  unsafe {
    let mut target = std::ptr::Unique::new(head.offset(i));
    let guard = spawn(move || {
      std::ptr::write(target.get_mut(), 10 + i as u32);
    });
    guard
  });

Is there anything I have missed here that can make this potentially blow up?

This uses #![feature(unique)] so I don't see how to use this in stable. Is there a way to do this sort of thing in stable (ideally safely without using raw pointers and overhead of Arc's and Mutex's)?

Also, looking at documentation for Unique, it says

It also implies that the referent of the pointer should not be modified without a unique path to the Unique reference

I am not clear what "unique path" means.

like image 955
sgldiv Avatar asked Jul 27 '15 02:07

sgldiv


1 Answers

Today the rayon crate is the de facto standard for this sort of thing:

use rayon::prelude::*;

fn main() {
    let mut data = vec![1, 2, 3];
    data.par_iter_mut()
        .enumerate()
        .for_each(|(i, x)| *x = 10 + i as u32);
    assert_eq!(vec![10, 11, 12], data);
}

Note that this is just one line different from the single-threaded version using standard iterators, which would replace par_iter_mut with iter_mut.

See also Writing a small ray tracer in Rust and Zig.

like image 56
Jack O'Connor Avatar answered Sep 20 '22 06:09

Jack O'Connor