node.jsmemory-managementasync-awaitdb2ibm-midrange

Querying DB2 every 15 seconds causing memory leak in NodeJS


I have an application which checks for new entries in DB2 every 15 seconds on the iSeries using IBM's idb-connector. I have async functions which return the result of the query to socket.io which emits an event with the data included to the front end. I've narrowed down the memory leak to the async functions. I've read multiple articles on common memory leak causes and how to diagnose them.

MDN: memory management

Rising Stack: garbage collection explained

Marmelab: Finding And Fixing Node.js Memory Leaks: A Practical Guide

But I'm still not seeing where the problem is. Also, I'm unable to get permission to install node-gyp on the system which means most memory management tools are off limits as memwatch, heapdump and the like need node-gyp to install. Here's an example of what the functions basic structure is.

    const { dbconn, dbstmt } = require('idb-connector');// require idb-connector

    async function queryDB() {
        const sSql = `SELECT * FROM LIBNAME.TABLE LIMIT 500`;

        // create new promise
        let promise = new Promise ( function(resolve, reject) {
            // create new connection 
            const connection = new dbconn();
            connection.conn("*LOCAL");
            const statement = new dbstmt(connection);
            statement.exec(sSql, (rows, err) => {
              if (err) {
                throw err;
              }
              let ticks = rows;
              statement.close();
              connection.disconn();
              connection.close();

              resolve(ticks.length);// resolve promise with varying data
            })
        });

        let result = await promise;// await promise
        return result;
    };

    async function getNewData() {
        const data = await queryDB();// get new data
        io.emit('newData', data)// push to front end
        setTimeout(getNewData, 2000);// check again in 2 seconds
    };

Any ideas on where the leak is? Am i using async/await incorrectly? Or else am i creating/destroying DB connections improperly? Any help on figuring out why this code is leaky would be much appreciated!!

Edit: Forgot to mention that i have limited control on the backend processes as they are handled by another team. I'm only retrieving the data they populate the DB with and adding it to a web page.

Edit 2: I think I've narrowed it down to the DB connections not being cleaned up properly. But, as far as i can tell I've followed the instructions suggested on their github repo.


Solution

  • This was confirmed to be a memory leak in the idb-connector library that i was using. Link to github issue Here. Basically there was a variable that never had it's memory deallocated in the core code of the library. A new version was added and the commit can viewed Here.