I am trying to write a series of Long
's to a GZIPOutputStream
, hoping to decompress the numbers later.
The following program works fine when I try it with a small number of Long
's, but it will throw an exception with many Long
's, like (10240).
Please see and run this code:
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class GzipTest {
private static final String filename = "data.gz";
public static void main(String... args) throws Exception {
test(32); // OK
test(10240); // Why won't this work? (unexpected read: 6)
}
public static void test(int max) throws Exception {
System.out.println("testing with " + max);
FileOutputStream fos = new FileOutputStream(filename);
GZIPOutputStream gos = new GZIPOutputStream(fos, 4096);
long num = 10241000L;
for (int i = 0; i < max; ++i) {
gos.write(long2bytes(num + i));
}
gos.close();
fos.close();
GZIPInputStream gis = new GZIPInputStream(new FileInputStream(filename), 4096);
byte[] buf = new byte[Long.BYTES];
for (int i = 0; i < max; ++i) {
int nread = gis.read(buf);
if (nread != Long.BYTES) {
throw new RuntimeException("unexpected read: " + nread);
}
long value = bytes2long(buf);
if (value != num + i) {
throw new RuntimeException("unexpected value: " + value);
}
}
gis.close();
}
static byte[] long2bytes(long num) {
ByteBuffer buf = ByteBuffer.allocate(Long.BYTES);
buf.putLong(num);
return buf.array();
}
static long bytes2long(byte[] bytes) {
return ByteBuffer.wrap(bytes).getLong();
}
}
The outputs:
testing with 32
testing with 10240
Exception in thread "main" java.lang.RuntimeException: unexpected read: 6
at GzipTest.test(GzipTest.java:31)
at GzipTest.main(GzipTest.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Th code works without any problems if you make the changes proposed by Peter Lawrey:
try (DataInputStream in = new DataInputStream(new GZIPInputStream(new FileInputStream(filename), 4096))) {
byte[] buf = new byte[Long.BYTES];
for (int i = 0; i < max; ++i) {
in.readFully(buf);
long value = bytes2long(buf);
if (value != num + i) {
throw new RuntimeException("unexpected value: " + value);
}
}
}
Or you can simplify everything to:
try (DataInputStream in = new DataInputStream(new GZIPInputStream(new FileInputStream(filename), 4096))) {
for (int i = 0; i < max; ++i) {
long value = in.readLong();
if (value != num + i) {
throw new RuntimeException("unexpected value: " + value);
}
}
}