python-3.xreadlinespython-zipfile

How to read a ZIpFile using a specific char as newline separator, instead of \n?


I've got this code:

from zipfile import ZipFile
...
with ZipFile(flu) as zf:
    for file in zf.namelist():
        if not file.endswith('.sql'):
            continue
        with zf.open(file,"r") as f:
            ...
        for row_b in f:
            ...

I want to use the char ';' as newline separator when opening the zip file, instead of '\n'. In this way I can run the SQL statements included in the zip file also if they are formatted in more than one line.

I find out this documentation and this documentation and it seems that I can specify the newline separator when opening a stream. I cannot understand how to do it when opening a zip file.


Solution

  • There is no option to configure the line ending character for ZipExtFile (what zip.open(...) returns).

    However, you can create your own buffered reader with a simple function:

    from io import BytesIO
    def delimited_read(f, sep, block_size=16384):
        buf = bytearray()
        while True:
            idx = buf.find(sep)
            while idx == -1:
                block = f.read(block_size)
                if block == b'':
                    break
                start = len(buf)
                buf.extend(block)
                idx = buf.find(sep, start)
            if idx == -1:
                if len(buf) > 0:
                    yield bytes(buf)
                return
            else:
                yield bytes(buf[0:idx])
                buf = buf[idx+1:]
    

    And you can simply use it as:

    for line in delimited_read(f, b';'):
      print(line)