pythonpython-3.xdjangostreamgzipstream

Download files as stream without downloading complete files into memory


I want to download files as stream from a remote server and make them zip stream and return it to frontend as stream without waiting all files to complete downloads.

Is it possible in python and framework of python ?? I tried as below but its not working i am using Django framework:

 zip_buffer = io.BytesIO()

 with zipstream.ZipFile(zip_buffer,"w", zipfile.ZIP_DEFLATED, False) as zf:
     url = "https://download-file/zy"
     r = requests.get(url, stream=True)
     zip_inf = zipstream.ZipInfo(file_name)
     zf.write_iter(zip_inf, r.content)
response = StreamingHttpResponse(zip_buffer.getvalue(), content_type='application/octet-stream')
            response['Content-Disposition'] = 'attachment; filename=zip-files.zip'
            return response

Solution

  • I tried by using aiozipstream and fast api for streamingResponse. this solution worked for me.

     files = []
     urls = ['url1','ur2']
     for url in urls:
        files.append({'stream': file_content_generator(url), 'name': url.split('/')[-1]})
     zf = ZipStream(files, chunksize=32768)
    
     response = StreamingResponse(zf.stream(), media_type='application/zip')
        return response
    
    def file_content_generator(url):
         with httpx.stream('GET', url, timeout=None) as r:
              yield from r.iter_bytes()