node.jssqliteadm-zip

Storing SQLite in Memory Database with ADM-ZIP (Node.js)


I'm trying to use the sqlite3 module to create an in memory SQLite database and the adm-zip module to save it into a zip file. So far I have been able to create an in memory database and add data to it, but I have been unsuccessful in finding a way to store it in a zip made through adm-zip as it requires either a file, buffer or string.

My question is this: does the sqlite3 module even support storing or saving as a buffer? If it doesn't then what would be an advisable solution for storing temporary files in Node.js when the script is used as both a requirable module and a command line script?

I've included the code I've been using to test with below and a cloneable gist.

main.js

var fs = require('fs'),
    admzip = require('adm-zip'),
    sqlite3 = require('sqlite3').verbose(),
    zip = new admzip(),
    db = new sqlite3.Database('test.sqlite');
    // db = new sqlite3.Database(':memory:');

db.serialize(function() {

    db.run('CREATE TABLE IF NOT EXISTS test(id INTEGER PRIMARY KEY, name TEXT);');
    db.run('INSERT OR IGNORE INTO test(name) VALUES ("neogeek");');

});

zip.addFile('README.md', '#SQLite3 + ADM-ZIP Test');

// zip.addFile('db.sqlite', db);

db.close();

fs.writeFileSync('test.zip', zip.toBuffer(), 'utf8');

package.json

{
    "private": true,
    "dependencies": {
        "sqlite3": "3.0.2",
        "adm-zip": "0.4.4"
    }
}

https://gist.github.com/neogeek/70c80c7ddaf998bee4bd


Solution

  • While continuing to pursue an answer for this question I stumbled upon the npm module temp and was able to put together a workable solution as detailed below.

    var fs = require('fs'),
        temp = require('temp').track(),
        admzip = require('adm-zip'),
        sqlite3 = require('sqlite3').verbose(),
        zip = new admzip(),
        tempdb = temp.openSync('db.sqlite'),
        db = new sqlite3.Database(tempdb.path);
    
    db.serialize(function() {
    
        db.run('CREATE TABLE IF NOT EXISTS test(id INTEGER PRIMARY KEY, name TEXT);');
        db.run('INSERT OR IGNORE INTO test(name) VALUES ("neogeek");');
    
    });
    
    zip.addFile('README.md', '#SQLite3 + ADM-ZIP Test');
    
    db.close(function () {
    
        zip.addFile('test.sqlite', fs.readFileSync(tempdb.path));
    
        fs.writeFileSync('test.zip', zip.toBuffer(), 'utf8');
    
    });
    

    I included the temp module and made sure to activate automatic cleanup of any temporary files created with.track().

    temp = require('temp').track(),
    

    I then created a new file to store the sqlite database in.

    tempdb = temp.openSync('db.sqlite'),
    

    Finally I moved the writing of both the sqlite file to the in memory zip and the final output zip file into the sqlite close method callback.

    db.close(function () {
    
        zip.addFile('test.sqlite', fs.readFileSync(tempdb.path));
    
        fs.writeFileSync('test.zip', zip.toBuffer(), 'utf8');
    
    });
    

    https://gist.github.com/neogeek/70c80c7ddaf998bee4bd