AWS ASG Error State transition reason Server.InternalError
Error:- State transition reason: Server.InternalError Client.InternalError: Client error on launch
When using the launch template to launch the instance it is working fine but if i use ASG with launch template getting above error on ec2 instance.
Kms Key policy used for the ebs
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxx:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow administration of the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::xxxxxxx:role/core-CloudformationStackAdmin",
"arn:aws:iam::xxxxxxx:root",
"arn:aws:iam::xxxxxxx:role/core-ServiceCatalogLaunchAdmin",
"arn:aws:iam::xxxxxxx:role/core-AccountAdmin=fGLB@000"
]
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::xxxxxxx:role/core-CloudformationStackAdmin",
"arn:aws:iam::xxxxxxx:root",
"arn:aws:iam::xxxxxxx:role/core-ServiceCatalogLaunchAdmin",
"arn:aws:iam::xxxxxxx:role/core-AccountAdmin=fGLB@I+000"
]
},
"Action": [
"kms:DescribeKey",
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey",
"kms:GenerateDataKeyWithoutPlaintext"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::xxxxxxx:role/core-CloudformationStackAdmin",
"arn:aws:iam::xxxxxxx:root",
"arn:aws:iam::xxxxxxx:role/core-ServiceCatalogLaunchAdmin",
"arn:aws:iam::xxxxxxx:role/core-AccountAdmin=fGLB@000"
]
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
},
{
"Sid": "Allow use of the key for Cloudwatch Log Groups Encryption",
"Effect": "Allow",
"Principal": {
"Service": "logs.eu-west-1.amazonaws.com"
},
"Action": [
"kms:Encrypt*",
"kms:Decrypt*",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*"
],
"Resource": "*",
"Condition": {
"ArnEquals": {
"kms:EncryptionContext:aws:logs:arn": "arn:aws:logs:eu-west-1:xxxxxxxx:log-group:*"
}
}
}
]
}
output of the instance :-
{
"Reservations": [
{
"Instances": [
{
"Monitoring": {
"State": "pending"
},
"PublicDnsName": "",
"StateReason": {
"Message": "Client.InternalError: Client error on launch",
"Code": "Client.InternalError"
},
"State": {
"Code": 48,
"Name": "terminated"
},
"EbsOptimized": true,
"LaunchTime": "2021-11-08T11:38:29.000Z",
"ProductCodes": [],
"CpuOptions": {
"CoreCount": 8,
"ThreadsPerCore": 2
},
"StateTransitionReason": "Server.InternalError",
"InstanceId": "i-0a266c694eb414f70",
"EnaSupport": true,
"ImageId": "ami-0ed588d6f749dcf28",
"PrivateDnsName": "",
"SecurityGroups": [],
"ClientToken": "2c45f3fd-5ea9-acd3-d703-c39a55955c94",
"InstanceType": "m5.4xlarge",
"CapacityReservationSpecification": {
"CapacityReservationPreference": "open"
},
"NetworkInterfaces": [],
"Placement": {
"Tenancy": "default",
"GroupName": "",
"AvailabilityZone": "eu-west-1a"
},
"Hypervisor": "xen",
"BlockDeviceMappings": [],
"Architecture": "x86_64",
"RootDeviceType": "ebs",
"RootDeviceName": "/dev/sda1",
"VirtualizationType": "hvm",
"Tags": [
{
"Value": "False",
"Key": "AutoShutdown"
},
{
"Value": "False",
"Key": "AutoStart"
},
{
"Value": "AutoScalingGroup",
"Key": "aws:cloudformation:logical-id"
},
{
"Value": "lt-0014c04827c2647b7",
"Key": "aws:ec2launchtemplate:id"
},
{
"Value": "True",
"Key": "RunAtWeekends"
},
{
"Value": "arn:aws:cloudformation:eu-west-1:9887878787:stack/ppe-devops-ecs-06/d5763820-4087-11ec-933b-02d79c55316d",
"Key": "aws:cloudformation:stack-id"
},
{
"Value": "1",
"Key": "aws:ec2launchtemplate:version"
},
{
"Value": "True",
"Key": "KeepMe"
},
{
"Value": "True",
"Key": "SaveIfOrphaned"
},
{
"Value": "ppe-devops-ecs-06-asg",
"Key": "aws:autoscaling:groupName"
},
{
"Value": "ppe-devops-ecs-06",
"Key": "aws:cloudformation:stack-name"
}
],
"HibernationOptions": {
"Configured": false
},
"AmiLaunchIndex": 0
}
],
"ReservationId": "r-014c8f943e5ca3655",
"RequesterId": "178953610797",
"Groups": [],
"OwnerId": "9887878787"
}
]
}
Solution
KMS key access was the issue. Granting the permission has resolved the issue. When there are extra encrypted volume is attached then this can come as error
- AmazonS3Exception: The authorization header is malformed; the region is wrong
- Sending an SNS Notification when Alarm state changes to OK
- Unable to run a simple python application in AWS App Runner
- Unnesting a comma seperated string in AWS cloudwatch insights query
- Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect)
- Difference between KMS encryption and S3 SSE
- How to Manage Terraform State Across Multiple Workspaces and Environments Without Causing Downtime in AWS?
- Get Cognito verification code in lambda to send SMS using local provider
- AWS ECR basic scanning - different outcomes from different accounts
- What do the SSM PingStatus values mean?
- Adding On-Premise node to EKS - Kubernetes
- Use macro inside CloudFormation Conditions
- "sls dynamodb start" throws spawn java ENOENT
- Accessing AWS ElastiCache (Redis CLUSTER mode) from different AWS accounts via AWS PrivateLink
- Cannot start an AWS ssm session on EC2 Amazon linux instance
- elasticache -ERR unknown command 'PSYNC
- AWS Cloudformation to create EventBridge Scheduler to trigger Instance Refresh
- `aws secretsmanager list-secrets` command to return secrets and filter them by tag
- nginx ingress controller type nlb with static ip giving error "AllocationIdNotFound"
- ModuleNotFoundError: No module named 'imp'
- cdk api gateway route53 lambda custom domain name not working
- couldn't get current server API group list: the server has asked for the client to provide credentials error: You must be logged in to the server
- Bash with AWS CLI - unable to locate credentials
- InvalidClientTokenId: The security token included in the request is invalid. status code: 403
- s3 - how to get fast line count of file? wc -l is too slow
- AWS PostgreSQL `Cannot find version X for aurora`
- Cannot connect to Amazon Keyspaces with cqlsh
- Can't connect to phpmyadmin in Bitnami instance hosted by AWS
- AWS Amplify - 404 error when deploying Nextjs App
- How can I connect to my aurora serverless v2 database instance?