I have a process where I need to convert a numpy recarray to bytes, and after that reconstruct the recarray from the bytes.
However, I am not sure how to do recover the array from bytes.
Does anyone know how could I do it?
Example code:
import numpy as np
import pandas as pd
df = pd.DataFrame(np.zeros(500))
rec = df.to_records()
rec_s = rec.tostring() # this returns a bytes object
# perform some computation
new_rec = <method to recover from bytes>(rec_s)
Note: I don't actually need to use numpy recarry, just some structure that will allow me to transform the pandas dataframe into a bytes object, and also recover it.
In [497]: arr = np.ones(3, dtype='i,i,f')
In [498]: arr
Out[498]:
array([(1, 1, 1.), (1, 1, 1.), (1, 1, 1.)],
dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<f4')])
In [499]: astr = arr.tostring()
In [500]: astr
Out[500]: b'\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?'
Recover it using the same dtype:
In [502]: np.frombuffer(astr, arr.dtype)
Out[502]:
array([(1, 1, 1.), (1, 1, 1.), (1, 1, 1.)],
dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<f4')])
If the source was 2d, you'd have to reshape as well