[SOLVED] Delete all items in a DynamoDB table using bash with both partition and sort keys

Delete all items in a DynamoDB table using bash with both partition and sort keys

I'm trying to delete all items in a DynamoDB table that has both partition and sort keys using AWS CLI in bash. The best thing I've found so far is:

aws dynamodb scan --table-name $TABLE_NAME --attributes-to-get "$KEY" \
--query "Items[].$KEY.S" --output text | \
tr "\t" "\n" | \
xargs -t -I keyvalue aws dynamodb delete-item --table-name $TABLE_NAME \
--key "{\"$KEY\": {\"S\": \"keyvalue\"}}"

But this does not work with a table that has both the partition key and the sort key, and I have not yet been able to make it work with such a table. Any idea how to modify the script to make it work for a table with composite keys?

Solution

Depending on the size of your table this can be too expensive and result in downtime. Remember that deletes cost you the same as a write, so you'll get throttled by your provisioned WCU. It would be much simpler and faster to just delete and recreate the table.

# this uses jq but basically we're just removing 
# some of the json fields that describe an existing 
# ddb table and are not actually part of the table schema/defintion
aws dynamodb describe-table --table-name $table_name | jq '.Table | del(.TableId, .TableArn, .ItemCount, .TableSizeBytes, .CreationDateTime, .TableStatus, .ProvisionedThroughput.NumberOfDecreasesToday)' > schema.json
# delete the table
aws dynamodb delete-table --table-name $table_name
# create table with same schema (including name and provisioned capacity)
aws dynamodb create-table --cli-input-json file://schema.json

If you really want to you can delete each item individually and you're on the right track you just need to specify both the hash and range keys in your scan projection and delete command.

aws dynamodb scan \
  --attributes-to-get $HASH_KEY $RANGE_KEY \
  --table-name $TABLE_NAME --query "Items[*]" \
  # use jq to get each item on its own line
  | jq --compact-output '.[]' \
  # replace newlines with null terminated so 
  # we can tell xargs to ignore special characters 
  | tr '\n' '\0' \
  | xargs -0 -t -I keyItem \
  # use the whole item as the key to delete (dynamo keys *are* dynamo items)
aws dynamodb delete-item --table-name $TABLE_NAME --key=keyItem

If you want to get super fancy you can use the describe-table call to fetch the hash and range key to populate $HASH_KEY and $RANGE_KEY but i'll leave that as an exercise for you.