amazon-web-servicessharepointaws-cloudformationamazon-appflow

How to use Cloudformation to create an Appflow flow using Microsoft Sharepoint Connector?


I am trying to deploy a flow that will retrieve files from a sharepoint and move them to AWS S3.

I managed to do it using AWS website but need to industrialize and automate the process using cloudformation. The "Microsoft Sharepoint Online" connector is not an official one, hence I need to use the CustomConnector (See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-appflow-flow-customconnectorsourceproperties.html)

On the Sharepoint connector info on AWS we can see that the online Runtime Settings to set is called subEntities and is described as follows: JSON Array of folderIds, driveIds that need to be synced (Required, type String)

I tried several solutions but I do not know how I am supposed to format this argument and where to find the driveId/folderId.

I found several values that seems plausibles and tried all the combinations.

At first I had the following error in Cloudformation:

Error while communicating to connector: The request failed because the service Microsoft SharePoint Online returned the following error: Details: invalid Json array of driveId/folder Id, ErrorCode: InvalidArgument. (Service: Appflow, Status Code: 400, Request ID: xxx)

But now I am using the following template:

SourceFlowConfig:
  ApiVersion: v1.0
  ConnectorProfileName: 'myPreviouslyCreatedConnector'
  ConnectorType: CustomConnector
  SourceConnectorProperties:
    CustomConnector:
       CustomProperties:
         subEntities: '["[What I think is driveId]/[What I think is folderId]"]'

       EntityName: Microsoft SharePoint Online

But I am getting:

Error while communicating to connector: The request failed because the service Microsoft SharePoint Online returned the following error: Details: Invalid drive Id/folder Id, ErrorCode: InvalidArgument. (Service: Appflow, Status Code: 400, Request ID: xxx)

I guess I am on a good path as the array seems to be correctly parsed, but cannot find where to go from here.

I found the IDs using the graph api endpoint: GET /drive/items/[ITEM ID] that retrieved both driveId and id.

Has someone tried to use cloudformation to create these kind of connector?


Solution

  • The correct format is :

    CustomProperties:
      subEntities: '["drives/[driveId]/items/[folderId]"]'
    

    You can see it using the console network inspector of the web browser while creating the flow using AWS Web Console.

    To get those IDs:

    1. Call GET https://[ORGANIZATION].sharepoint.com/sites/[SITENAME]/_api/v2.0/drive/list/items/ and search for you object. In the eTag you will have something like "\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx,[NUMBER]\"". You have to use the xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx in the next step
    2. Call GET https://[ORGANIZATION].sharepoint.com/sites/[SITENAME]/_api/v2.0/drive/items/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx
    3. Use driveId (from ParentReference) and the value of the id field as your folderId (NOT THE xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx FROM STEP 1)

    Hoping it will save time to someone.