amazon-dynamodbaws-step-functions

dynamodb ConditionExpression in Step functions


With the use of aws-step-functions I'm trying to putItem into dynamodb.

Here is an extract of the .yml, which works as intended.

...

AddingURLs:
  Type: Map
  ItemsPath: "$.vidPosts"
  Parameters:
    vidPostsIndex.$: "$$.Map.Item.Index" 
      vidPosts.$: "$$.Map.Item.Value"

  Iterator:
    StartAt: AddingURLIntoDynamoDB
    Comment: Looping over urls and adding them into the db

    States:
      AddingURLIntoDynamoDB:
        Type: Task
        Resource: arn:aws:states:::dynamodb:putItem
        ResultPath: null
        Parameters:
          TableName: my-table-table-name
          Item:
            videoID:
              S.$: "$.vidPosts.[0]"
            urls:
              S.$: "$.vidPosts.[1]"
          ConditionExpression: "attribute_not_exists(videoID)"    

        End: true

  Next: EndStepFunctions

EndStepFunctions:
  Comment: This marks the end of a step functions
  Type: Succeed

But the next time around, when this state is called, if it encounters an entry that already exists, because of, ConditionExpression: "attribute_not_exists(videoID)", as it should, it throws,

{
  executionArn: 'arn:aws:states:us-east-2:aaabbbccc:execution:xxxStateMachine:XXX-YYY-ZZZ',
  stateMachineArn: 'arn:aws:states:us-east-2:aaabbbccc:stateMachine:xxxStateMachine',
  name: '9814ce93-bc42-4f31-a742-089ea7db9ad6',
  status: 'FAILED',
  startDate: 2021-03-18T15:30:45.912Z,
  stopDate: 2021-03-18T15:30:52.386Z,
  input: '{}',
  inputDetails: { included: true },
  error: 'DynamoDB.ConditionalCheckFailedException',
  cause: 'The conditional request failed (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ConditionalCheckFailedException; Request ID: bfd76f51-567b-439c-9a39-5a1643e75a69; Proxy: null)'
}

My question is within aws-step-functions, how do I make it ignore existing entries but continue with the iterator?

Appreciate any help and thank you in advance.


Solution

  • We just need to catch DynamoDB.ConditionalCheckFailedException exception and move forward, without throwing any error.

    What we need is

      "Catch": [
        {
          "ErrorEquals": [
            "DynamoDB.ConditionalCheckFailedException"
          ],
          "Next": "FinalStep"
        }
      ]
    

    Here is full step function.

    {
      "StartAt": "put-item",
      "States": {
        "put-item": {
          "Next": "FinalStep",
          "Catch": [
            {
              "ErrorEquals": [
                "DynamoDB.ConditionalCheckFailedException"
              ],
              "Next": "FinalStep"
            }
          ],
          "Type": "Task",
          "Resource": "arn:aws:states:::dynamodb:putItem",
          "Parameters": {
            "Item": {
              "pk": {
                "S": "1"
              },
              "test": {
                "S": "value"
              }
            },
            "TableName": "test",
            "ConditionExpression": "attribute_not_exists(#videoID)",
            "ExpressionAttributeNames": {
              "#videoID": "videoID"
            }
          }
        },
        "FinalStep": {
          "Type": "Pass",
          "End": true
        }
      }
    }
    

    Put-item failed with error and still moved to next task.

    {
      "Error": "DynamoDB.ConditionalCheckFailedException",
      "Cause": "The conditional request failed (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ConditionalCheckFailedException; Request ID: xxxx-xxx-xxx-xxx-xxxxx; Proxy: null)"
    }
    

    enter image description here