javaoutputstreambufferedwritergzipoutputstream

Count the bytes written to file via BufferedWriter formed by GZIPOutputStream

I have a BufferedWriter as shown below:

BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
        new GZIPOutputStream( hdfs.create(filepath, true ))));

String line = "text";
writer.write(line);

I want to find out the bytes written to the file with out querying file like 

hdfs = FileSystem.get( new URI( "hdfs://localhost:8020" ), configuration );

filepath = new Path("path");
hdfs.getFileStatus(filepath).getLen();

as it will add overhead and I don't want that.

Also I cant do this:

line.getBytes().length;

As it give size before compression.

Solution

  • You can use the CountingOutputStream from Apache commons IO library.

    Place it between the GZIPOutputStream and the file Outputstream (hdfs.create(..)).

    After writing the content to the file you can read the number of written bytes from the CountingOutputStream instance.