amazon-web-servicesamazon-dynamodbaws-cdkamazon-dynamodb-index

DynamoDB not indexing items in GSI as expected


hope to find you well :)

I am trying to create a DynamoDB table with a GSI using the aws CDK. Here is my code:

export class LeaderboardsTableStack extends cdk.Stack {

    public leaderboardsTable

    constructor(scope: Construct, id: string, props: LeaderboardsTableStackProps) {
        super(scope, id, props)

        this.leaderboardsTable = new dynamodb.Table(this, `${props.STAGE}leaderboardsTable`, {
            tableName: `${props.STAGE}Leaderboards`,
            partitionKey: { name: 'PK', type: dynamodb.AttributeType.STRING },
            sortKey: { name: 'SK', type: dynamodb.AttributeType.STRING },
            removalPolicy: cdk.RemovalPolicy.DESTROY,
            billingMode: dynamodb.BillingMode.PROVISIONED,
            stream: dynamodb.StreamViewType.NEW_AND_OLD_IMAGES
        })

        this.leaderboardsTable.addGlobalSecondaryIndex({
            indexName: 'GSI1',
            partitionKey: { name: 'SK', type: dynamodb.AttributeType.STRING },
            sortKey: { name: 'GSI1SK', type: dynamodb.AttributeType.STRING },
            projectionType: dynamodb.ProjectionType.INCLUDE,
            nonKeyAttributes: ['participant_display_name']
        })

    }
}

As you can see, I am trying to specify that the GSI should have as the partition key the SK from the main table, and as a sort key the attribute GSI1SK that only certain items have.

However, for some reason, in the GSI table the items simply get replicated with the same PK and SK, but only those that have the GSI1SK attribute (at least this is going according to plan lol).

What am I doing wrong here? I am quite puzzled honestly :/

Even from the documentation, this should work as I intended right?

If have any tip or idea, please feel free to share! Thank you in advance!


Solution

  • Indexes in DynamoDB are sparse, meaning items will only be replicated to the index when they have the partition and sort key of the GSI as attributes in the item. If you don't have either the GSI PK or GSI SK as attributes, the item is not projected into the index.

    https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-indexes-general-sparse-indexes.html

    As for the attributes which exist in your items that are projected, you defined the following :

    projectionType: dynamodb.ProjectionType.INCLUDE,
    nonKeyAttributes: ['participant_display_name']
    

    This means that for the items projected they will have the GSI key attributes, SK, GSI1SK and the base table keys, which means also PK and finally what you decide to project: participant_display_name

    https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html#GSI.Projections

    Table Definition:

    {
        "Table": {
            "AttributeDefinitions": [
                {
                    "AttributeName": "pk",
                    "AttributeType": "S"
                },
                {
                    "AttributeName": "sk",
                    "AttributeType": "S"
                },
                {
                    "AttributeName": "timestamp",
                    "AttributeType": "S"
                }
            ],
            "TableName": "LocalTable",
            "KeySchema": [
                {
                    "AttributeName": "pk",
                    "KeyType": "HASH"
                },
                {
                    "AttributeName": "sk",
                    "KeyType": "RANGE"
                }
            ],
            "TableStatus": "ACTIVE",
            "CreationDateTime": 1690881682.108,
            "ProvisionedThroughput": {
                "LastIncreaseDateTime": 0.0,
                "LastDecreaseDateTime": 0.0,
                "NumberOfDecreasesToday": 0,
                "ReadCapacityUnits": 1,
                "WriteCapacityUnits": 1
            },
            "TableSizeBytes": 29,
            "ItemCount": 1,
            "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/LocalTable",
            "GlobalSecondaryIndexes": [
                {
                    "IndexName": "GSI1",
                    "KeySchema": [
                        {
                            "AttributeName": "sk",
                            "KeyType": "HASH"
                        },
                        {
                            "AttributeName": "timestamp",
                            "KeyType": "RANGE"
                        }
                    ],
                    "Projection": {
                        "ProjectionType": "ALL"
                    },
                    "IndexStatus": "ACTIVE",
                    "ProvisionedThroughput": {
                        "ReadCapacityUnits": 1,
                        "WriteCapacityUnits": 1
                    },
                    "IndexSizeBytes": 29,
                    "ItemCount": 1,
                    "IndexArn": "arn:aws:dynamodb:ddblocal:000000000000:table/LocalTable/index/GSI1"
                }
            ]
        }
    }
    

    Now you see that I have created a GSI called GSI1 which has a partition key of SK, here I make a Query using NoSQL Workbench on that index:

    NoSQL WB

    Now I am able to efficiently query using the base table SK attribute as it is the partition key in my GSI1.