I am creating a SharedMemory object, where data is being written to it in an I/O bound process for subsequent processing by a separate compute-bound process. In subsequent processing of the data I want to be able to read and write with a file-like object, so I'd like to copy to a io.BytesIO object. I'm not sure how to use the buf memory view object of the SharedMemory.
How can I copy data between a SharedMemory object and a BytesIO object?
Given the following example setup:
from io import BytesIO
from multiprocessing.shared_memory import SharedMemory
b = BytesIO()
s = SharedMemory(create=True, size=1024)
you can write the contents of the memory view to the bytes I/O object directly with
>>> b.write(s.buf)
1024 # N.B. bytes written equals size of shared memory
This works for any file-like object, not just BytesIO.
To do the reverse, you can extract the contents of the ByteIO object into a bytes string, and then write those bytes to a desired spot in the shared memory by indexing the memoryview object:
b.seek(0) # change stream position to start
contents = b.read() # read contents into bytes object
start_idx = 123 # example location
s.buf[start_idx : start_idx + len(contents)] = contents