I try to create a gzip compress tar file. The problem, the output is not a valid gzip compress file.
When i do a file logfiles.tgz
it just say: logfiles.tgz: data
and it shows no files/nothing when i open it.
const { exec } = require("child_process");
const fs = require("fs");
const outputStream = fs.createWriteStream("../logfiles.tar.gz");
const child = exec(`tar -cz *`, {
cwd: "./logs"
}, (err) => {
if (err) {
console.error(`Could not create archive: ${err}`);
} else {
console.log(`Archive created`);
}
});
child.stdout.pipe(outputStream);
outputStream.on("error", (err) => {
console.error(`Could not write: ${err}`);
});
outputStream.on("finish", () => {
console.log("Write Done");
});
When i let tar write directly to the filesystem it works just fine:
exec("tar -czf ../logfiles.tar.gz *", {
cwd: "./logs"
}, (err) => {
if (err) {
console.error(`Could not create archive: ${err}`);
} else {
console.log(`Archive created`);
}
});
As soon as i want the output via stdout as stream, i get a corrupt/invalid file. I want to send the tar output over http without the need to first write to the filesystem.
When i do a gunzip: logfiles.tar.gz
it says gzip: logfiles.tar.gz: not in gzip format
, which is not really a surprise when file ...
says "data" instead of "gzip".
Why is this not a valid gzip compressed file when i use streams/stdout?
When i compare both files, the stream file has (nearly) twice the size:
-rw-rw-r-- 1 marc marc 117936 Apr 26 11:58 logfiles-direct.tar.gz
-rw-rw-r-- 1 marc marc 213813 Apr 26 11:58 logfiles-stream.tar.gz
With the help of Jochaim Sauer's comment i got it working.
The problem was the encoding
option for "exec". Setting it to "buffer" resolved the problem.
const child = exec(`tar -cz *`, {
cwd: "./logs",
encoding: "buffer"
});
Works like expected.