I am trying to stream a csv to azure blob storage, the csv is generated directly from python scripts without local copy, i have the following code, df is the csv file:
with open(df,'w') as f:
stream = io.BytesIO(f)
stream.seek(0)
block_blob_service.create_blob_from_stream('flowshop', 'testdata123', stream)
then i got the error massage:
stream = io.BytesIO(f) TypeError: a bytes-like object is required, not '_io.TextIOWrapper'
i think the problem has been the format incorrect, can you please identify the problem. thanks.
You opened df
for write, then tried to pass the resulting file object as the initializer of io.BytesIO
(which is supposed to to take actual binary data, e.g. b'1234'
). That's the cause of the TypeError
; open files (read or write, text or binary) are not bytes
or anything similar (bytearray
, array.array('B')
, mmap.mmap
, etc.), so passing them to io.BytesIO
makes no sense.
It looks like your goal is to read from df
, and you shouldn't need io.BytesIO
at all for that. Just change the mode from (text) write, 'w'
, to binary read, 'rb'
. Then pass the resulting file object to your API directly:
with open(df, 'rb') as f:
block_blob_service.create_blob_from_stream('flowshop', 'testdata123', f)
Update: Apparently df
was your actual data, not a file name to open at all. Given that, you should really skip the stream API (which is pointless if the data is already in memory) and just use the bytes
based API directly:
block_blob_service.create_blob_from_bytes('flowshop', 'testdata123', df)
or if df
is str
, not bytes
:
block_blob_service.create_blob_from_bytes('flowshop', 'testdata123', df.encode('utf-8'))