gremlinamazon-neptune

What is the proper way to establish a connection to AWS Neptune from AWS ECS?

Background / Setup

I'm ...

Here is a diagram to help explain the context: Context Diagram

In case it matters, here is where I'm getting the {{ writer-endpoint }} value: Neptune Console

I saw this discussion, but I'm not sure if it applies.

Does anyone have any insights or suggestions here?

I've tried Claude and ChatGPT to attack this, but the lack of detail as to what could be wrong is making this very difficult to troubleshoot.

I've intentionally made the IAM policy open (e.g., "*" for resource access) so as to troubleshoot, and I think I've granted the appropriate IAM permissions.

Update: based on the below answer from Taylor, I was able to resolve the issue. I created this TypeScript version of his answer, if it helps anyone in the future.

Solution

  • As to the issue of the 403 error, that is likely due to IAM auth. How are you signing the connection request to Neptune?

    Here's an example of connecting via Gremlin-Javascript library with IAM signing. This fetches credentials using the Default Credential Provider Chain. In this case, that would be fetched from whatever is provided by the ECS task execution role.

    const gremlin = require('gremlin');
const { fromNodeProviderChain } = require('@aws-sdk/credential-providers');
const {getUrlAndHeaders} = require('gremlin-aws-sigv4/lib/utils');
const __ = gremlin.process.statics;
const DriverRemoteConnection = gremlin.driver.DriverRemoteConnection;
const Graph = gremlin.structure.Graph;

const region = 'us-west-2';
const getCredentials = async () => {
    let credentials;
    try {
      credentials = fromNodeProviderChain({ clientConfig: { region }});
    } catch (e) {
      console.error("No credentials found", e);
    }
    return credentials;
  };

(main = async () => {
    console.log('starting');
    let provider = await getCredentials();
    let creds = await provider();
    creds['region'] = region;
    conninfo = getUrlAndHeaders(
        'neptune-cluster-endpoint.us-west-2.neptune.amazonaws.com',
        '8182',
        creds,
        '/gremlin',
        'wss'); 

    console.log(conninfo);

    dc = new DriverRemoteConnection(conninfo['url'], { headers: conninfo['headers'] });

    const graph = new Graph();
    const g = graph.traversal().withRemote(dc);

    query = g.V('some-id').valueMap(true);

    res = await query.next();

    console.log(res);

    dc.close();
})

main()

    This is similar to what is documented as the example Node Lambda function in our docs (https://docs.aws.amazon.com/neptune/latest/userguide/lambda-functions-examples.html#lambda-functions-examples-javascript). That example is more comprehensive as it also provides means for handling connection errors and reconnects.