I use boto3
client for managing DynamoDB items. Let's assume that the item structure is the following:
{
"id": "item_id",
"name": "item_name",
"age": 23,
"body": {
{"nested": "nested_data"}
}
}
There was a bug at some point, and instead of nested
key, nested_error
key was inserted for some items:
{
"id": "item_id",
"name": "item_name",
"age": 23,
"body": {
{"nested_error": "nested_data"}
}
}
The bug has been fixed and I would like to implement a "migration" that will rename nested_error
key to nested
one for all items affected. The action should be skipped if the name of the attribute is already nested
.
What is the correct way to do such a migration? Should I implement a script with boto3
or an awscli
command would be enough? Or maybe it can be done via the AWS Console? What command should I use?
The best approach depends on how much data you have. I would personally opt for the CLI, as it auto paginates. One thing to be careful of is throttling exceptions should it be a large data set. Might be easily fixed implementing some sleep in the code:
#!/bin/bash
TABLE_NAME="YourTableName"
SCAN_RESULTS=$(aws dynamodb scan \
--table-name $TABLE_NAME \
--filter-expression "attribute_exists(#body.#nested_error)" \
--expression-attribute-names '{"#body": "body", "#nested_error": "nested_error"}')
ITEMS=$(echo $SCAN_RESULTS | jq -c '.Items[]')
echo "$ITEMS" | while IFS= read -r ITEM; do
ITEM_ID=$(echo $ITEM | jq -r '.id.S')
NESTED_ERROR=$(echo $ITEM | jq -r '.body.M.nested_error.S')
aws dynamodb update-item \
--table-name $TABLE_NAME \
--key '{"id": {"S": "'"$ITEM_ID"'"}}' \
--update-expression "SET #body.#nested = :nested REMOVE #body.#nested_error" \
--expression-attribute-names '{"#body": "body", "#nested": "nested", "#nested_error": "nested_error"}' \
--expression-attribute-values '{":nested": {"S": "'"$NESTED_ERROR"'"}}'
done