Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I use Rayon with an existing iterator?

Tags:

rust

rayon

I turn a regex into a HashSet after doing some filtering. I am trying to use it with Rayon, but I can't figure out how to make Rayon work with an existing iterator without converting it to a vector first. Is this possible?

let re = Regex::new("url=\"(?P<url>.+?)\"").unwrap();
let urls: HashSet<String> = re.captures_iter(&contents)
    .map(|m| Url::parse(m.name("url").unwrap().as_str()))
    .filter(|parsed_url| parsed_url.is_ok())
    .map(|parsed_url| parsed_url.unwrap())
    .filter(|parsed_url| parsed_url.has_host())
    .map(|parsed_url| parsed_url.into_string())
    .collect();
like image 610
user964375 Avatar asked Feb 22 '18 08:02

user964375


1 Answers

This is possible now with ParallelBridge:

use rayon::iter::ParallelBridge;
use rayon::prelude::ParallelIterator;
use std::sync::mpsc::channel;

let rx = {
    let (tx, rx) = channel();

    tx.send("one!");
    tx.send("two!");
    tx.send("three!");

    rx
};

let mut output: Vec<&'static str> = rx.into_iter().par_bridge().collect();
output.sort_unstable();

assert_eq!(&*output, &["one!", "three!", "two!"]);
like image 70
Jesse Grosjean Avatar answered Sep 20 '22 22:09

Jesse Grosjean