Setup

I'm building a serverless application that uses a website frontend. The website sends queries to the API, which connects to the database. The infrastructure is hosted on the AWS, but the problem I have ran into only affected the local development. A key component of this setup is the serverless API, which uses Lambdas for each API route. To ensure that connections to the database are present, I connect to the database and pass that connection to the ORM prior to the execution of the handler, and to ensure that no idle connections remain after completing the query, I destroy them once the handler completes (be it with error or gracefully). The logic behind these is sound, as far as implementation goes.

Stack

I am running Serverless with Serverless Offline plugin to develop a Node 14 application locally. The database is a Postgres instance, which I connect to using Objection.JS as ORM. Under the hood, Objection uses Knex to manage the connection, handle query building etc.

Versions

The versions of software used.

• Node 14
• knex (2.1.0)
• objection (3.0.1)
• serverless (2.55.0)
• serverless-offline (8.7.0)

Problem

The problem I have ran into is connections not being available, when running the setup locally using serverless-offline. Despite connecting to the database at the start of the handler, sometimes, the API queries throw an error stating that connection could not be acquired, or that no database is available to a query. This happens intermittently, and sometimes the same API calls execute without any problems.

I have realised that this happens only when multiple API queries are being executed simultaneously, although once again, not always.

Pseudocode

The lambda handler in serverless-offline looks something like this:

module.exports.handler = async function() {
  const connection = await connectToDatabase();
  await ORM.databaseConnection.setup(connection);

  // ...

  const results = ORM.executeQuery();

  // ...

  await ORM.databaseConnection.destroy();
  return results;
}

It took me an embarasingly long amount of time to finally find the source of this problem, mainly because it would only occur sporadically (when multiple handlers are being ran locally).

I have realised that the reason why these database connections result in errors (not being able to acquire connection, not having available connection to execute query), is because serverless-offline executes the lambda handlers in a shared process, meaning that:

Connection stored by the Objection.JS ORM, acquired by one lambda function, would persist to the other lambda function executing locally at the same time (which is fine), however the moment one of those lambdas destroys its connection, the second lambda would lose its connection, meaning it would have no connection to execute queries against anymore.

Solution

The solution to this problem, when running serverless-offline, is to run the lambda handlers as separate processes. This will result in each handler execution having autonomy over its own connection without affecting other executions. This can be achieved by adding the following flag to the serverless-offline command: --useChildProcesses, like so:

serverless-offline --useChildProcesses