diff options
Diffstat (limited to 'posts/parseq.md')
-rw-r--r-- | posts/parseq.md | 42 |
1 files changed, 42 insertions, 0 deletions
diff --git a/posts/parseq.md b/posts/parseq.md new file mode 100644 index 0000000..2e3b607 --- /dev/null +++ b/posts/parseq.md @@ -0,0 +1,42 @@ +--- +title: "Parseq" +description: "A parallel sequential iterator for Rust." +published: 2022-12-20 +--- + +I've published my own parallel sequential iterator library for Rust, [Parseq](https://crates.io/crates/parseq). +Parseq provides an extension trait adding a [`map_parallel`](https://docs.rs/parseq/0.1.0/parseq/trait.ParallelIterator.html#method.map_parallel) method to the standard iterator trait. +It's a drop-in-replacement for the standard [`map`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#tymethod.next) method. + + use std::time::Duration; + use parseq::ParallelIterator; + + let mut iter = (0..3) + .into_iter() + .map_parallel(|i| { + // Insert heavy computation here ... + std::thread::sleep(Duration::from_millis((i % 3) * 10)); + i + }); + + assert_eq!(iter.next(), Some(0)); + assert_eq!(iter.next(), Some(1)); + assert_eq!(iter.next(), Some(2)); + assert_eq!(iter.next(), None); + +See the [repository](https://git.skreutz.com/parseq.git) for a real world example, and [docs.io](https://docs.rs/parseq/latest/parseq/) for the documentation. + +Parseq is of course not the first crate providing a parallel map function for iterators. +But I think it comes with a unique set of features: + +* Parseq utilizes a configurable number of worker threads +* Parseq preserves the order of the original iterator +* Parseq is lazy in the sense that it doesn't consume from the original iterator before [`next`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#tymethod.next) is called for the first time +* Parseq doesn't [`fuse`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.fuse) the original iterator +* Parseq uses constant space: linear in the number of threads and the size of the buffer, not in the length of the possibly infinite original iterator +* Parseq propagates panics from the given closure + +In contrast, [Rayon](https://crates.io/crates/rayon), the de-facto standard solution for Rust, doesn't preserve the order of the original iterator. +[Pariter](https://crates.io/crates/pariter), a good sequential alternative addressing this exact issue, doesn't support infinite iterators; though it provides more features than Parseq. + +Therefore, I invite you to give [Parseq](https://crates.io/crates/parseq) a try, and send me some feedback. |