[SOLVED] How to debug "Jest has detected the following ... open handle potentially keeping Jest from exiting"

I am having further problems with async code in Jest. My previous question (for the same project) was related to running async code in a Jest bootstrap. My new issue relates to running async database calls within the tests. My aim is to connect to database services and to make calls to make sure they read and write to the database correctly. I have the tests running in one Docker container, connecting to a MySQL instance in another container.

I am using the mysql2/promise Node library, which as the same suggests, wraps callback-based database operations in a Promise. Most of the operations are async, except for connection closing (and a few others). Indeed, I wonder if this is relevant.

I should start with some code. Here is my test:

import TestDatabase from '../TestDatabase';
var config = require('../../config/config.json');
import FetchDonations from "../../src/services/FetchDonations";
const envName = 'test';

let database = new TestDatabase(config);

// Connect before all tests
beforeAll(() => {
  console.log('Connect Jest database');
  return database.connect(envName);
});

// Disconnect after all tests
afterAll(async done => {
  console.log('Disconnect Jest database');
  database.close();
  done();
});

describe('Database tests', () => {

  // Before every test
  beforeEach(() => database.beforeEachTest(envName));

  test('Describe this demo test', () => {
    console.log('Test #1');
    expect(true).toEqual(true);
  });

  test('Describe this demo test 2', () => {
    console.log('Test #2');
    expect(true).toEqual(true);
  });

});

This just runs a couple of dummy tests. They don't do anything, I'm just trying to get the before/after hooks working. These are what they should do:

beforeAll - connect to the database once (asynchronous op)
afterAll - disconnect from the database once (synchronous op in mysql2)
beforeEach - run database.beforeEachTest() before every test, this truncates the tables in the database (asynchronous ops)

Here is what TestDatabase looks like - these are utility methods I've written to help with database testing:

const mysql = require('mysql2/promise');

export default class TestDatabase {

  constructor(config) {
    this.config = config;
  }

  beforeEachTest(environmentName) {
    console.log('Before a test');

    return this.setForeignKeyChecks(false).then(() => {
      return this.truncateTables();
    }).then(() => {
      return this.setForeignKeyChecks(true);
    }).catch((error) => {
      console.log('Failed to clear down database: ' + error);
    });
  }

  connect(environmentName) {
    const config = this.getEnvConfig(environmentName);

    return mysql.createConnection({
      host: config.host, user: config.username,
      password: config.password
    }).then((connection) => {
      this.connection = connection;
      return this.useDatabase(environmentName);
    }).catch((error) => {
      console.log('Failed to connect to the db');
    });
  }

  getConnection() {
    if (!this.connection) {
      throw 'Database not connected';
    }

    return this.connection;
  }

  dropDatabase(environmentName) {
    const config = this.getEnvConfig(environmentName);

    return this.getConnection().query(
      `DROP DATABASE IF EXISTS ${config.database}`
    );
  }


  createDatabase(environmentName) {
    const config = this.getEnvConfig(environmentName);

    return this.getConnection().query(
      `CREATE DATABASE IF NOT EXISTS ${config.database}`
    );
  }

  useDatabase(environmentName) {
    const config = this.getEnvConfig(environmentName);

    return this.getConnection().query(
      `USE ${config.database}`
    );
  }

  setForeignKeyChecks(value) {
    // Make injected value safe
    var boolStr = value ? '1' : '0';

    return this.getConnection().query(
      `SET FOREIGN_KEY_CHECKS = ${boolStr}`
    );
  }

  getTables() {
    return ['contribution', 'donation', 'expenditure',
      'tag', 'expenditure_tag'];
  }

  truncateTables() {
    return Promise.all(
      this.getTables().map(table => this.truncateTable(table))
    );
  }

  truncateTable(table) {
    return this.getConnection().query(
      `TRUNCATE TABLE ${table}`
    );
  }

  /**
   * Close is synchronous so there is no returned promise
   */
  close() {
    this.getConnection().close();
  }

  getEnvConfig(environmentName) {
    if (!environmentName) {
      throw 'Please supply an environment name'
    }
    if (!this.config[environmentName]) {
      throw 'Cannot find database environment data'
    }

    return this.config[environmentName];
  }
}

Now, if I run the tests, they pass and finish, but there are two oddities. Firstly, the some of the async console.log output is being output after the test summary, so I think I am not handling async in the way Jest wants it. In other words, I think the summary should be rendered after all of this:

/project/node_modules/.bin/jest tests
  console.log
    Connect Jest database

      at Object.<anonymous> (tests/database/TestDemo.test.js:29:11)

  console.log
    Before a test

      at TestDatabase.beforeEachTest (tests/TestDatabase.js:10:13)

 PASS  tests/database/TestDemo.test.js
  Database tests
    ✓ Describe this demo test (72ms)
    ✓ Describe this demo test 2 (58ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        2.118s, estimated 3s
Ran all test suites matching /tests/i.
  console.log
    Test #1

      at Object.<anonymous> (tests/database/TestDemo.test.js:46:13)

  console.log
    Before a test

      at TestDatabase.beforeEachTest (tests/TestDatabase.js:10:13)

  console.log
    Test #2

      at Object.<anonymous> (tests/database/TestDemo.test.js:51:13)

  console.log
    Disconnect Jest database

      at _callee$ (tests/database/TestDemo.test.js:35:11)

As you can see, output from both Tests appears after the summary, but output from the beforeEach for the first test appears before the test summary.

Moreover, if I add real tests that use the database, I get errors saying that I have unhandled promises, and that I should try Jest's unhandled promise detector (--detectOpenHandles). Moreover, in that situation, Jest stops in a loop and needs ^C to give the console prompt back.

So, I am trying --detectOpenHandles with the current code, and although I don't get a Jest freeze, I get the following.

Jest has detected the following 1 open handle potentially keeping Jest from exiting:

  ●  TCPWRAP

      22 |     const config = this.getEnvConfig(environmentName);
      23 | 
    > 24 |     return mysql.createConnection({
         |                  ^
      25 |       host: config.host, user: config.username,
      26 |       password: config.password
      27 |     }).then((connection) => {

      at new Connection (node_modules/mysql2/lib/connection.js:35:27)
      at Object.<anonymous>.exports.createConnection (node_modules/mysql2/index.js:10:10)
      at Object.createConnection (node_modules/mysql2/promise.js:230:31)
      at TestDatabase.connect (tests/TestDatabase.js:24:18)
      at Object.<anonymous> (tests/database/TestDemo.test.js:30:19)

My view is that this is directly connected to the freeze I get with more tests, and that I should fix that before attempting to add more tests.

I have been through several investigation loops to determine what might cause this, and the code has been tweaked several times:

afterAll and beforeEach are async ops, so they need to be returned to Jest, so Jest knows to wait for them to resolve.
afterAll does a db close, but this is not async, so I am using Jest's done() here, though it also did not work if it is done without done().
TestDatabase contains two main methods, beforeEachTest and connect, and I have been very careful to ensure they return Promises.
I have tended towards chained thenables, rather than async-await, as it feels clearer to me. However I have tried async-await in several areas, and it did not help.
Utility code like dropDatabase, createDatabase, setForeignKeyChecks, truncateTables, truncateTable all return Promises.
I've read the Jest async docs and there are a lot of approaches. The main takeaway is that if you are testing something async, the promise should be returned to Jest, so the appropriate waiting is done. My real tests are synchronous, it's just my before hooks that are async. Thinking about it, I wonder if that is the problem?

I am fairly new to Jest, and not too experienced in JS async either. Every time I think I have an improved understanding of async, I get a fresh curveball. However, I wonder if this is more Jest oddities, rather than a difficulty understanding raw async.