google-cloud-platformgoogle-cloud-firestoregoogle-workflows

Running Google Workflows in Parallel


I have a collection of documents in cloud firestore, every night I would like to run Workflows on each document to perform a series of transformations. The transformation of one document has no effect on another. I currently have a for loop in my YAML file that does the transformation of each document one at a time. I was wondering if I can perform the transformation to all the documents in parallel instead of one at a time.


Solution

  • Workflow steps can be done in parallel:

    https://cloud.google.com/workflows/docs/reference/syntax/parallel-steps#example_of_parallel_branches

    Code snippet:

    main:
      params: [args]
      steps:
        - init:
            assign:
              - user: {}
              - notification: {}
        - parallelStep:
            parallel:
              shared: [user, notification]
              branches:
                - getUser:
                    steps:
                      - getUserCall:
                          call: http.get
                          args:
                            url: ${"https://example.com/users/" + args.userId}
                          result: user
                - getNotification:
                    steps:
                      - getNotificationCall:
                          call: http.get
                          args:
                            url: ${"https://example.com/notification/" + args.notificationId}
                          result: notification