aws-lambdaamazon-dynamodbserverlessdynamodb-queriesamazon-dynamodb-index

DynamoDB asks for Partition Key of main table when querying GSI


So, the thing is, I'm trying to query a GSI table on DynamoDB and get a really weird behaviour. The main table schema is as follows

- Partition key:    test_id (string)
- Sort key:         version (string)
- Other attributes  (createdAt, tags, etc)

I want to obtain every entry that has a sort_key equal to v0_test WITHOUT filtering by partition key. To do this and avoid a full scan I created a GSI (Global Secondary Index) as follows:

- Partition key:    version (string)
- Sort key:         createdAt (number)
- Other attributes  (test_id, tags, etc)

When querying this from AWS console I can query for every partition key equal to v0_test and I get the expected results, but when I query from inside a lambda function (runtime: nodejs16.x) I get an error. The code for the query is as follows:

const dynamoDb = new AWS.DynamoDB.DocumentClient();
let params = {
    TableName: dynamoTable,
    IndexName: dynamoTableIndex ,
    KeyConditionExpression: 'version = :v0 AND createdAt BETWEEN :tLower AND :tUpper',
    ExpressionAttributeValues: {
        ':v0': 'v0_test',
        ':tUpper': Math.floor(Date.now() / 1000).toString(),
        ':tLower': '0'
    }
};
let result = await dynamoDb.query(params).promise();
console.log("Success", result);

And the error I get is

ERROR ValidationException: Query condition missed key schema element: test_id

As you can see it's asking for the partition key for the main table.

Things I've tried:

I wasn't expecting that at all. It's my first time using DynamoDB but as I understand it the idea behind GSI's (or one of them) is to be able to query a DynamoDB table by other attribute than it's main partition key without having to do a full scan.

Any help appreciated and if you need more details just ask! It's my first time asking on StackOverflow too so I'm sure I'll miss something.

EDIT

Tested the suggested solution and got an error suggesting that the schema was wrong, which was a new error and got me thinking, so I tried specifying the ExpressionAttributeNames and it WORKED!. I created the request as follows using DocumentClient:

const dynamoDb = new AWS.DynamoDB.DocumentClient();
let params = {
    TableName: 'CLASSIFIER_TESTS_DEV_us-east-1',
    IndexName: 'version-createdAt-index',
    KeyConditionExpression: '#versionAttr = :version AND #ca BETWEEN :tLower AND :tUpper',
    ExpressionAttributeNames: {
        "#versionAttr": "version",
        "#ca": "createdAt"
    },
    ExpressionAttributeValues: {
        ":version": "v0_test",
        ":tUpper": Date.now(),
        ":tLower": 0
    }
};
let result = await dynamoDb.query(params).promise();

Thanks everyone! I still think that it should have worked the way I did it the first time as that's the way everyone does it on tutorials/examples/documentation, but oh well, got it working and that's what matters for now.


Solution

  • This should not be an issue. I would suggest the following:

    1. Test locally, have your DynamoDB API call run from your local machine, this will ensure that no issues are being caused by Lambda invocations
    2. Hard code your TableName and IndexName as strings in your parameters.
    3. If the above is still causing an issue, try the same logic but this time a Query from the base table.

    Let me know if you still face issue after that, happy to help.