1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
|
---
title: "Parseq"
description: "A parallel sequential iterator for Rust."
published: 2022-12-20
---
I've published my own parallel sequential iterator library for Rust, [Parseq](https://crates.io/crates/parseq).
Parseq provides an extension trait adding a [`map_parallel`](https://docs.rs/parseq/0.1.0/parseq/trait.ParallelIterator.html#method.map_parallel) method to the standard iterator trait.
It's a drop-in-replacement for the standard [`map`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#tymethod.next) method.
use std::time::Duration;
use parseq::ParallelIterator;
let mut iter = (0..3)
.into_iter()
.map_parallel(|i| {
// Insert heavy computation here ...
std::thread::sleep(Duration::from_millis((i % 3) * 10));
i
});
assert_eq!(iter.next(), Some(0));
assert_eq!(iter.next(), Some(1));
assert_eq!(iter.next(), Some(2));
assert_eq!(iter.next(), None);
See the [repository](https://git.skreutz.com/parseq.git) for a real world example, and [docs.io](https://docs.rs/parseq/latest/parseq/) for the documentation.
Parseq is of course not the first crate providing a parallel map function for iterators.
But I think it comes with a unique set of features:
* Parseq utilizes a configurable number of worker threads
* Parseq preserves the order of the original iterator
* Parseq is lazy in the sense that it doesn't consume from the original iterator before [`next`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#tymethod.next) is called for the first time
* Parseq doesn't [`fuse`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.fuse) the original iterator
* Parseq uses constant space: linear in the number of threads and the size of the buffer, not in the length of the possibly infinite original iterator
* Parseq propagates panics from the given closure
In contrast, [Rayon](https://crates.io/crates/rayon), the de-facto standard solution for Rust, doesn't preserve the order of the original iterator.
[Pariter](https://crates.io/crates/pariter), a good sequential alternative addressing this exact issue, doesn't support infinite iterators; though it provides more features than Parseq.
Therefore, I invite you to give [Parseq](https://crates.io/crates/parseq) a try, and send me some feedback.
|