databaseamazon-web-servicesamazon-dynamodbrestoreaws-backup

Restore DynamoDB table to new table deployed with AWS CDK


I am using AWS CDK to deploy some DynamoDB tables in my stack. Once they are created, I want to be able to create a backup of the table, then delete all the stacks in my app, redeploy all and copy the data from the backup to the newest deployed table.

The problem is that restoring a backup with DynamoDB creates a new table. It does not update my existing table deployed with AWS CDK. This is an issue since I am using the table reference in AWS CDK to pass to other resources, so I need to be able to restore it to that table.

I have tried to create a Lambda that uses the backupArn to restore it to the new table:

async function restoreTableFromBackup(
  backupArn: string,
  targetTableName: string
): Promise<void> {
  try {
    const restoreParams = {
      BackupArn: backupArn,
      TargetTableName: targetTableName,
    };

    console.log(`Restoring from backup ${backupArn} to table ${targetTableName}`);
    await dynamoDBClient.send(new RestoreTableFromBackupCommand(restoreParams));
    console.log(`Successfully restored table ${targetTableName} from backup ${backupArn}`);
  } catch (error) {
    console.error(`Error restoring table ${targetTableName} from backup ${backupArn}:`, error);
    throw error;
  }
}

However, I am getting an error since the targetTableName table already exists:

ERROR   Error restoring table CommonStack-MovementsTableCommonxxx from backup arn:xxx:table/CommonStack-MovementsTableCommonxxx/backup/xxx: TableAlreadyExistsException: Table already exists: CommonStack-MovementsTableCommonxxx

I read here that there might be a not ideal solution but I want to avoid this:

Make a backup of Table_A, restore to Table_B. 
Take a backup of Table_B. Drop Table_A. 
Restore Table_B as Table_A. 
And drop Table_B.

I also tried this, but my stack detects that the tables were deleted and the new backup tables don't work, even though share the same name as the previous ones.

Finally, I also read here this possible solution:

1. Delete existing tables directly from dynamodb (do not delete from cloudformation)
2. Restore backup to new table, using the name of the deleted table
3. In cloudformation, detect drift, manually fix any drift errors in dynamodb, and then detect drift again

It did not work since CloudFormation detected the tables as deleted even though I created new ones with the same name.

All I want to do is have a simple way of restoring all of my data safely in case I need to destroy my stacks.

Thanks!


Solution

    1. You should not delete the table. If you leave it there with on-demand mode then you pay nothing except a small amount for storage (assuming you're not storing TBs).

    2. CFN will only detect drift of something changed, the name of the table and the table ARN will be the same. What could be causing drift is the stream ARN which has a time stamp. If that's the case when restoring the table don't enable a stream until after you connect it back to CFN, then allow CFN to enable a new stream via update. There may be other setting that CFN monitors that you'll have to make sure matches, as backups do not save all table settings.