javamultithreadingcompressiongzipoutputstream

GZIPOutputStream that does its compression in a separate thread


Is there an implemetation of GZIPOutputStream that would do the heavy lifting (compressing + writing to disk) in a separate thread?

We are continuously writing huge amounts of GZIP-compressed data. I am looking for a drop-in replacement that could be used instead of GZIPOutputStream.


Solution

  • You can write to a PipedOutputStream and have a thread which reads the PipedInputStream and copies it to any stream you like.

    This is a generic implementation. You give it an OutputStream to write to and it returns an OutputStream for you to write to.

    public static OutputStream asyncOutputStream(final OutputStream out) throws IOException {
        PipedOutputStream pos = new PipedOutputStream();
        final PipedInputStream pis = new PipedInputStream(pos);
        new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    byte[] bytes = new byte[8192];
                    for(int len; (len = pis.read(bytes)) > 0;)
                        out.write(bytes, 0, len);
                } catch(IOException ioe) {
                    ioe.printStackTrace();
                } finally {
                    close(pis);
                    close(out);
                }
            }
        }, "async-output-stream").start();
        return pos;
    }
    
    static void close(Closeable closeable) {
        if (closeable != null) try {
            closeable.close();
        } catch (IOException ignored) {
        }
    }