What is the best way to push a polars dataframe to redis ?
Currently I am trying to read the _io.BytesIO buffer returned by write_ipc
redis_instance.set("key",df.write_ipc(file = None, compression="lz4").read())
This doesn't seem to work as the key in redis is empty
write_ipc returns a BytesIO object. As it has just written, the position is at the end of the stream. BytesIO.read() will start reading from the position onwards, which is in your code empty by definition.
You can instead use getvalue, which will always read all contents. See also this Stackoverflow question on BytesIO Alternatively, you can reset the position by calling the method seek(0) prior to callling read().