I have an ECS Service with Tasks using an EFS Volume. I also have a Datasync task to update the files on the EFS Filesystem(destination) from an S3 Bucket(source)
The container task is serving files stored on the EFS.
Whenever I run the Datasync task and update EFS the data is not updated on the container. In order to have the latest data I have to set the desired tasks on the Service to 0 and then to 1 again so it creates new instances of the task.
I want to avoid having downtime for my service. Is there a way to have the latest data without doing what I previously explained?
Update:
Looks like my main issue is that the container is indexing files to memory so a restart(redeployment is needed). But I still have an availability issue, I need a way to automate this process
Ok so after a lot of research I found out that there's an AWS-CLI command to force a new deployment for the ECS Service, with that new tasks are created and the old ones are kept until the newest ones are in a stable state, that removes my availability issue.
The command is:
aws ecs update-service --force-new-deployment --service my-service-name --cluster my-cluster-name
Now I needed a way to automatically update my task whenever the Datasync task is executed, I thought about creating a pipeline for it to execute the command from a Codebuild container. The issue is that CodePipeline does not have a Datasync Task Execution as a trigger for the Source Stage...
My solution was to create an AWS EventBridge Rule with a Pattern to detect the event of Datasync Task State Change for my task and then I added as a Target a CodeBuild Project where I defined the previously stated command in the buildspec
The pattern looks like this in case someone needs it (wasn't able to find any example about it online)
{
"detail-type": ["DataSync Task State Change"],
"resources": ["arn:aws:datasync:us-east-1:1234567890:task/task-1234567890"],
"source": ["aws.datasync"],
"detail": {
"State": ["AVAILABLE"]
}
}
With that my Datasync Task Completes its execution the EventBridge rule will automatically trigger my CodeBuild project, therefore automatically updating my ECS Servcice with 0 downtime!