I'm using the warcio library to read and write warc files.
When trying to write a record of a response object from requests.get(URL,stream=False), warcio is writing only HTTP headers to the record but not the payload. However, when stream mode is enabled it works fine.
Is there a way store the payload when stream mode is not enabled?
I've found a workaround but not sure if it's the correct way. Instead of making request object streamable, I've made the payload streamable
BytesIO(response.text.encode()) and this seems to work.