The following class is designed to handle data that becomes available in chunks, while being able to write the processing function imperatively. I'm using Fiber
to suspend execution when necessary to wait for more input:
class Filter
def initialize
@fiber = Fiber.new do
run
end
@fiber.resume
end
def <<(chunk)
@fiber.resume(chunk)
end
def each_two
loop do
a = Fiber.yield
b = Fiber.yield
yield a + b
end
end
def run
each_two do |chunk|
puts chunk.inspect
end
end
end
filter = Filter.new
filter << "Hello"
filter << ", "
filter << "world"
filter << "!\n"
$ ruby filter.rb
"Hello, "
"world!\n"
I can even wrap each_two
in an Enumerator
:
def run
pairs = enum_for(:each_two)
pairs.each do |chunk|
puts chunk.inspect
end
end
But if I use external iteration, it breaks:
def run
pairs = enum_for(:each_two)
loop do
puts pairs.next.inspect
end
end
$ ruby filter.rb
nil
nil
filter.rb:20:in 'block in Filter#each_two': undefined method '+' for an instance of Fiber (NoMethodError)
yield a + b
^
from <internal:kernel>:168:in 'Kernel#loop'
from filter.rb:17:in 'Filter#each_two'
from filter.rb:in 'Enumerator#each'
I suspect that this is due to Enumerator
using Fiber
as an implementation detail. It's unfortunate that this is a leaky abstraction. Is there a way to make them work together nicely?
(I suspect this question is related to External iteration is broken but internal iteration works, but I'm not sure.)
With external iteration each_two
runs in a different fiber which is where Fiber.yield
returns control (instead of the main fiber) where iterator does its thing and resumes.
You can use transfer
instead to navigate between fibers. Transfer to main fiber where it waits and then transfer back to enum fiber:
class Filter
def initialize
@main_fiber = Fiber.current
@fiber = Fiber.new do
run
end
@fiber.transfer
end
def <<(chunk)
@enum_fiber.transfer(chunk)
end
def each_two
@enum_fiber = Fiber.current
loop do
a = @main_fiber.transfer
b = @main_fiber.transfer
yield a + b
end
end
def run
pairs = enum_for(:each_two)
loop do
p pairs.next
end
end
end
f = Filter.new
f << "h"
f << "i"
# => "hi"