Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What are idiomatic ways to send data between threads?

I want to do some calculation in a separate thread, and then recover the data from the main thread. What are the canonical ways to pass some data from a thread to another in Rust?

fn main() {
    let handle = std::thread::spawn(|| {
        // I want to send this to the main thread:
        String::from("Hello world!")
    });

    // How to recover the data from the other thread?

    handle.join().unwrap();
}
like image 355
Boiethios Avatar asked Nov 27 '19 17:11

Boiethios


People also ask

How can I share data between two threads?

All static and controlled data is shared between threads. All other data can also be shared through arguments/parameters and through based references, as long as the data is allocated and is not freed until all of the threads have finished using the data.

How do you share data between Rust threads?

Using Mutexes to Allow Access to Data from One Thread at a Time. Mutex is an abbreviation for mutual exclusion, as in, a mutex allows only one thread to access some data at any given time. To access the data in a mutex, a thread must first signal that it wants access by asking to acquire the mutex's lock.

Which variables are ways to communicate data among threads?

The easiest way for multiple threads to communicate is through memory. If two threads can access the same memory location, the cost of that access is little more than the memory latency of the system.

How do you move data between threads in python?

You can protect data variables shared between threads using a threading. Lock mutex lock, and you can share data between threads explicitly using queue. Queue.


2 Answers

There are lots of ways to send send data between threads -- without a clear "best" solution. It depends on your situation.


Using just thread::join

Many people do not realize that you can very easily send data with only the thread API, but only twice: once to the new thread and once back.

use std::thread;

let data_in = String::from("lots of data");
let handle = thread::spawn(move || {
    println!("{}", data_in);  // we can use the data here!

    let data_out = heavy_compuations();
    data_out // <-- simply return the data from the closure
});

let data_out = handle.join().expect("thread panicked :(");
println!("{}", data_out);  // we can use the data generated in the thread here!

(Playground)

This is immensely useful for threads that are just spawned to do one specific job. Note the move keyword before the closure that makes sure all referenced variables are moved into the closure (which is then moved to another thread).


Channels from std

The standard library offers a multi producer single consumer channel in std::sync::mpsc. You can send arbitrarily many values through a channel, so it can be used in more situations. Simple example:

use std::{
    sync::mpsc::channel,
    thread,
    time::Duration,
};

let (sender, receiver) = channel();
thread::spawn(move || {
    sender.send("heavy computation 1").expect("receiver hung up :(");
    thread::sleep(Duration::from_millis(500));
    sender.send("heavy computation 2").expect("receiver hung up :(");
});

let result1 = receiver.recv().unwrap();
let result2 = receiver.recv().unwrap();

(Playground)

Of course you can create another channel to provide communication in the other direction as well.


More powerful channels by crossbeam

Unfortunately, the standard library currently only provides channels that are restricted to a single consumer (i.e. Receiver can't be cloned). To get more powerful channels, you probably want to use the channels from the awesome crossbeam library. Their description:

This crate is an alternative to std::sync::mpsc with more features and better performance.

In particular, it is a mpmc (multi consumer!) channel. This provides a nice way to easily share work between multiple threads. Example:

use std::thread;

// You might want to use a bounded channel instead...
let (sender, receiver) = crossbeam_channel::unbounded();
for _ in 0..num_cpus::get() {
    let receiver = receiver.clone();  // clone for this thread
    thread::spawn(move || {
        for job in receiver {
            // process job
        }
    });
}

// Generate jobs
for x in 0..10_000 {
    sender.send(x).expect("all threads hung up :(");
}

(Playground)

Again, adding another channel allows you to communicate results back to the main thread.


Other methods

There are plenty of other crates that offer some other means of sending data between threads. Too many to list them here.

Note that sending data is not the only way to communicate between threads. There is also the possibility to share data between threads via Mutex, atomics, lock-free data structures and many other ways. This is conceptually very different. It depends on the situation whether sending or sharing data is the better way to describe your cross thread communication.

like image 124
Lukas Kalbertodt Avatar answered Oct 07 '22 15:10

Lukas Kalbertodt


The idiomatic way to do so is to use a channel. It conceptually behaves like an unidirectional tunnel: you put something in one end and it comes out the other side.

use std::sync::mpsc::channel;

fn main() {
    let (sender, receiver) = channel();

    let handle = std::thread::spawn(move || {
        sender.send(String::from("Hello world!")).unwrap();
    });

    let data = receiver.recv().unwrap();
    println!("Got {:?}", data);

    handle.join().unwrap();
}

The channel won't work anymore when the receiver is dropped.

They are mainly 3 ways to recover the data:

  • recv will block until something is received
  • try_recv will return immediately. If the channel is not closed, it is either Ok(data) or Err(TryRevcError::Empty).
  • recv_timeout is the same as try_recv but it waits to get a data a certain amount of time.
like image 35
Boiethios Avatar answered Oct 07 '22 16:10

Boiethios