I can create Amazon FSx Lustre SCRATCH2 file system with a data repository association (DRA) with an S3 bucket in one action if I do this via AWS Management Console. It also works if I first create a SCRATCH2 and then add an S3 DRA to it. Via Terraform AWS provider I try to create an FSx Lustre Scratch 2 file system and an S3 bucket and link them via creation of a DRA. I get an error message:
Error: creating FSx for Lustre Data Repository Association: UnsupportedOperation: This file system does not support data repository associations.
In Amazon FSx Lustre documentation I cannot find anything about inability to add a DRA later. There's only a notice about inability to use DRAs with Scratch 1. See for example: https://docs.aws.amazon.com/fsx/latest/LustreGuide/overview-dra-data-repo.html https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html
Looks like Amazon FSx Lustre documentation is incomplete on DRAs and Scratch 2. Or maybe I simple have not found the right place.
Terraform AWS provider documentation says it is only PERSISTENT_2 that DRA resource (aws_fsx_data_repository_association) is compatible with. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fsx_data_repository_association
Which seems strange as it's possible to create Scratch 2 with S3 DRA via AWS Management Console and the DRA import/export works fine with such Scratch 2 file systems.
Seems that at least Terraform AWS provider documentation doesn't contradict what the provider can do.
I use Terraform 1.1.5 and AWS Provider 5.26.0 (checked on 4.65.0 as well) but I could upgrade Terraform to the latest if it might help.
My Terraform resources I have already tried are:
resource "aws_fsx_lustre_file_system" "smpl_lstr_scr2" {
deployment_type = "SCRATCH_2"
storage_capacity = 1200
data_compression_type = null
subnet_ids = [ var.mysubnet ]
}
resource "aws_s3_bucket" "a_buck" {
bucket = "my_buck"
}
resource "aws_fsx_data_repository_association" "scr2_s3_dra" {
file_system_id = aws_fsx_lustre_file_system.a_scr2.id
data_repository_path = s3://${aws_s3_bucket.a_buck.id}"
file_system_path = "/"
s3 {
auto_export_policy {
events = ["NEW", "CHANGED", "DELETED"]
}
auto_import_policy {
events = ["NEW", "CHANGED", "DELETED"]
}
}
}
Is there a way to mimic AWS Management Console Scratch 2 file system with a DRA with an S3 bucket behaviour in Terraform to create an FSx Lustre Scratch 2 file system and its association to an S3 bucket? Or any other Terraform way to create Scratch 2 to S3 DRAs?
I have found that Terraform AWS Provider 5.26.0 creates SCRATCH 2 FSx for Lustre file systems with Lustre version 2.10 by default. It explains why I get the error message when using Terraform. Amazon FSx for Lustre documentation explicitly tells it's impossible to use DRAs with 2.10. https://docs.aws.amazon.com/fsx/latest/LustreGuide/overview-dra-data-repo.html https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html
Correct SCRATCH 2 FSx for Lustre resource to use DRA is:
resource "aws_fsx_lustre_file_system" "smpl_lstr_scr2" {
file_system_type_version = "2.15"
deployment_type = "SCRATCH_2"
storage_capacity = 1200
data_compression_type = null
subnet_ids = [ var.mysubnet ]
}