I have a home made Iterator that I want to loop over, do some costly processing in a threadpool and gather the results in the input order.
I use this chain for that:
Iterator > enumerate() > rayon par_bridge() > map()
When I use this chain and collect()
into a vector there's no problem, but when I use a for loop I get an error that something isn't an Iterator.
I don't understand the difference.
Here's a MRE:
use std::vec::IntoIter;
use rayon::prelude::*;
fn main() {
// collecting();
for_loop();
}
fn collecting() {
let some_vec = (1..100).collect::<Vec<u8>>();
let x_iter = AA {y: some_vec.into_iter()};
let with_index: Vec<(usize, u8)> = x_iter
.enumerate()
.par_bridge()
.map(|(i, x)| {(i, x - 1)})
.collect();
println!("{:?}", with_index);
}
fn for_loop() {
let some_vec = (1..100).collect::<Vec<u8>>();
let x_iter = AA {y: some_vec.into_iter()};
let mut with_index: Vec<(usize, u8)> = Vec::with_capacity(99);
for x in x_iter
.enumerate()
.par_bridge() // with this line commented out it works fine
.map(|(i, x)| {(i, x - 1)})
{
with_index.push(x);
}
println!("{:?}", with_index);
}
struct AA {
y: IntoIter<u8>
}
impl Iterator for AA {
type Item = u8;
fn next(&mut self) -> Option<u8> {
self.y.next()
}
}
The collecting
function works fine.
The for_loop
function gives this error:
rayon::iter::Map<IterBridgestd::iter::Enumerate<AA>, {closure@src/main.rs:30:14: 30:22}> is not an iterator
Please explain why I'm getting this error.
There is an error in expectation here: The compiler is complaining that rayon::iter::Map<_>
is not an iterator, and therefor can't be used in for
-loop, because - well - rayon::iter::Map<_>
is not an iterator: The type does not implement std::iter::IntoIterator
, which is required to be used in a for
-loop. It does implement rayon::iter::ParallelIterator
, but this only allows for parallel iteration using the means of rayon
itself. There is no way for Rust compiler to desugar a rayon::iter::Map<_>
, and perform the operations you want; even if it (somehow) did, the items from rayon
's iterators become available in unspecified order, and are processed on different threads, all of which would need simultaneous mutable access to with_index
, which can never work.
You may want to look at rayon::iter::ParallelIterator::for_each()
. Yet even with, you'll notice that you can't .push()
to a single Vec
from within the closure given to for_each()
, due to the problem mentioned above.