The following code splits a stream of objects into chunks of 1000, processes them on materialisation and returns the total number of objects at the end.
In all cases the number returned is correct unless the stream size happens to be 1. In the case the stream size is 1, the number returned is 0.
Any help would be greatly appreciated. I have also had to hack the return call in the case there are no records in the stream to be 0. I'd like to fix this too.
AtomicInteger recordCounter = new AtomicInteger(0);
try (StreamEx<MyObject> stream = StreamEx.of(myObjects)) {
stream.groupRuns((prev, next) -> recordCounter.incrementAndGet() % 1000 != 0)
.forEach((chunk) ->
{
//... process each chunk
}
);
} catch(Exception e) {
throw new MyRuntimeException("Failure streaming...", e);
} finally {
myObjects.close();
}
return recordCounter.get() == 0 ? 0 : recordCounter.incrementAndGet();
In the end I went with Guava's Iterators.partition() to split my stream of objects into chunks:
MutableInt recordCounter = new MutableInt();
try {
Iterators.partition(myObjects.iterator(), 1000)
.forEachRemaining((chunk) -> {
//process each chunk
...
recordCounter.add(chunk.size());
});
} catch (Exception e) {
throw new MyRuntimeException("Failure streaming...", e);
} finally {
myObjects.close();
}
return recordCounter.getValue();