aws-lambdaeventtriggeraws-event-bridgeamazon-cloudtrailamazon-fsx

Trigger AWS Lambda function whenever a file is added to AWS FSX


I would like to trigger an AWS Lambda Function whenever a new file is added to AWS FSX. This is in order to perform an action on the file using to the Lambda function that gets notified.

While considering AWS cloudtrail, Eventbridge and Cloudwatch to trigger the Lambda function; I was unable to find AWS FSX in the data source options for this monitoring resource in AWS. Any suggestion on what tool can be used?


Solution

  • You didn't specify which FSx file system you are referring to. If you are referring to FSx for ONTAP then I have a potential solution for you. You can use this ONTAP API (https://docs.netapp.com/us-en/ontap-restapi/ontap/get-storage-volumes-files-.html) to list the contents of any directory within the file system. You could use that API to detect when a new file is created. Then, just "invoke" the other Lambda function that processes the file. So, basically, I would see two lambda functions. One that is triggered from an EventBridge schedule, that scans directories looking for new files and if it finds one, it invokes the other Lambda function to process the file.

    The first Lambda function would probably want to save its "state" somewhere, so it would know which files it has already reported on so it doesn't report on the same file more than once. Maybe just the time of the last run, so it can just report on any files have been created after that. You could easily just use an S3 bucket for that, or DynamoDB.

    Note that the Lambda function that uses the ONTAP APIs must run within a VPC that has access to the FSx for ONTAP management endpoint (typically the same VPC and Subnet that the FSxN resides in). Because of that, it will most likely need EC2 "endpoints" so it can access the AWS services it might need (e.g. S3 to store state, or Lambda to invoke the other Lambda function).