[SOLVED] CDK API Gateway Lambda still getting timed out after increasing timeout duration

I have increased the timeout duration for both the API Gateway and the Lambda function to 60 seconds after requesting quota increase from Service Quotas. Ref: https://aws.amazon.com/about-aws/whats-new/2024/06/amazon-api-gateway-integration-timeout-limit-29-seconds/

But I am still getting 504 error now.

I have also checked the logs for lambda and the function completes in 40 seconds.

How can I diagnose what the issue might be?

The function should be working without any error.

EDIT:

This is the stack file:

import { CfnOutput, Duration, Stack, StackProps } from "aws-cdk-lib";
import * as apigateway from "aws-cdk-lib/aws-apigateway";
import * as lambda from "aws-cdk-lib/aws-lambda";
import { Construct } from "constructs";

export class HelloAwsCdkStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Defines AWS API Gateway
    const helloGateway = new apigateway.RestApi(this, "HelloGateway");

    // Defines AWS Lambda
    const helloLambda = new lambda.Function(this, "HelloHandler", {
      runtime: lambda.Runtime.NODEJS_16_X, // execution environment
      code: lambda.Code.fromAsset("lambda"), // code loaded from "lambda" directory.
      handler: "hello.handler", // file is "hello", function is "handler"
      timeout: Duration.seconds(90),
    });

    const helloLambdaIntegration = new apigateway.LambdaIntegration(
      helloLambda,
      {
        timeout: Duration.seconds(90),
      }
    );

    const helloResource = helloGateway.root.addResource("graphql");

    ["GET", "POST"].forEach((method) => {
      helloResource.addMethod(method, helloLambdaIntegration, {
        authorizationType: apigateway.AuthorizationType.NONE,
      });
    });

    // Defines the function url for the AWS Lambda
    const helloLambdaUrl = helloLambda.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    // print the function url after deployment
    new CfnOutput(this, "FunctionUrl", { value: helloLambdaUrl.url });
  }
}

This is my lambda function file:

exports.handler = async (event) => {
  console.log("DEBUG BEFORE SLEEP");
  await new Promise((r) => setTimeout(r, 60000));
  console.log("DEBUG AFTER SLEEP");

  return {
    statusCode: 200,
    body: JSON.stringify({ message: "Hello world" }),
  };
};

As you can see, the Gateway and Lambda timeout is 90 seconds and there is a wait of 60 seconds in the function, but I am still getting this error in postman after 60 seconds:

Postman Error

These are the CloudWatch logs:

CloudWatch logs

NOTE:

One thing to note, I have an express server in the lambda function in my production environment, but I tried using https://www.npmjs.com/package/connect-timeout for increasing the timeout, but it still gets out after 30 seconds.

UPDATE:

I have updated the stack file as follows because I have Apollo GraphQL Server in the function file.

import { CfnOutput, Duration, Stack, StackProps } from "aws-cdk-lib";
import * as apigateway from "aws-cdk-lib/aws-apigateway";
import * as lambda from "aws-cdk-lib/aws-lambda";
import { Construct } from "constructs";
import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";

export class HelloAwsCdkStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Defines AWS API Gateway
    const helloGateway = new apigateway.RestApi(this, "HelloGateway");

    // Defines AWS Lambda Layer
    const depsLayer = new lambda.LayerVersion(this, "NodeDependencies", {
      code: lambda.Code.fromAsset("layer"),
      compatibleRuntimes: [lambda.Runtime.NODEJS_16_X],
    });

    // Defines AWS Lambda
    const helloLambda = new NodejsFunction(this, "HelloHandler", {
      runtime: lambda.Runtime.NODEJS_16_X, // execution environment
      code: lambda.Code.fromAsset("lambda"), // code loaded from "lambda" directory.
      handler: "hello.handler", // file is "hello", function is "handler"
      timeout: Duration.seconds(90),
      layers: [depsLayer],
      bundling: {
        externalModules: [
          "apollo-server-core",
          "apollo-server-lambda",
          "connect-timeout",
          "express",
          "graphql",
        ],
      },
    });

    const helloLambdaIntegration = new apigateway.LambdaIntegration(
      helloLambda,
      {
        timeout: Duration.seconds(90),
      }
    );

    const helloResource = helloGateway.root.addResource("graphql");

    ["GET", "POST"].forEach((method) => {
      helloResource.addMethod(method, helloLambdaIntegration, {
        authorizationType: apigateway.AuthorizationType.NONE,
      });
    });

    // Defines the function url for the AWS Lambda
    const helloLambdaUrl = helloLambda.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    // print the function url after deployment
    new CfnOutput(this, "FunctionUrl", { value: helloLambdaUrl.url });
  }
}

This is the updated lambda function file:

const express = require("express");
const timeout = require("connect-timeout");
const { ApolloServer, gql } = require("apollo-server-lambda");
const {
  ApolloServerPluginLandingPageLocalDefault,
} = require("apollo-server-core");

// Construct a schema, using GraphQL schema language
const typeDefs = gql`
  type Query {
    hello: String
  }
`;

// Provide resolver functions for your schema fields
const resolvers = {
  Query: {
    hello: async () => {
      await new Promise(r => setTimeout(r, 40000));
      return "Hello world!";
    },
  },
};

const server = new ApolloServer({
  typeDefs,
  resolvers,
  csrfPrevention: true,
  cache: "bounded",
  context: ({ event, context, express }) => ({
    headers: event.headers,
    functionName: context.functionName,
    event,
    context,
    expressRequest: express.req,
  }),
  plugins: [ApolloServerPluginLandingPageLocalDefault({ embed: true })],
});

function haltOnTimedout(req, res, next) {
  if (!req.timedout) next();
}

exports.handler = server.createHandler({
  expressAppFromMiddleware: (middleware) => {
    const app = express();
    app.use(timeout("90s"));
    app.use(middleware);
    app.use(haltOnTimedout);
    return app;
  },
});

As you can see I have added timeout middleware of 90 seconds same as the lambda and the API gateway but the hello query only has 40 seconds timeout.

I tested with both the lambda function URL and the API gateway endpoint. Only the former worked, while the latter throws the following error after 30 seconds:

"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<HTML><HEAD><META HTTP-EQUIV=\"Content-Type\" CONTENT=\"text/html; charset=iso-8859-1\">\n<TITLE>ERROR: The request could not be satisfied</TITLE>\n</HEAD><BODY>\n<H1>504 ERROR</H1>\n<H2>The request could not be satisfied.</H2>\n<HR noshade size=\"1px\">\nCloudFront attempted to establish a connection with the origin, but either the attempt failed or the origin closed the connection.\nWe can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.\n<BR clear=\"all\">\nIf you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.\n<BR clear=\"all\">\n<HR noshade size=\"1px\">\n<PRE>\nGenerated by cloudfront (CloudFront)\nRequest ID: IgaxrsYnf_duw7CJYtivWNAOeV4YyMLR7YOridwIng9H0vnjXtir1w==\n</PRE>\n<ADDRESS>\n</ADDRESS>\n</BODY></HTML>"

I have to use the API gateway as my production environment is a huge existing codebase and I don't want to update that.