I tried looking for an answer to this in Packer community-forum without any joy. so, I am wondering if anyone can give me a hand here to understand why it's being failed to push the image to ECR.
Below is my very simple packer HCL file, with two stepes under post-processors block to tag and push:
packer {
required_plugins {
docker = {
source = "github.com/hashicorp/docker"
version = "~> 1"
}
}
}
#
locals {
timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}
#
variable "img_name" {
type = string
}
variable "repo" {
type = string
}
#
source "docker" "al2023" {
image = "public.ecr.aws/amazonlinux/amazonlinux:2023"
commit = true
}
#
build {
name = var.img_name
sources = ["source.docker.al2023"]
post-processors {
post-processor "docker-tag" {
repository = var.repo
tags = ["latest", local.timestamp]
}
post-processor "docker-push" {
ecr_login = true
keep_input_artifact = false
login_server = var.repo
}
}
}
then I run packer as null_resource through terraform:
resource "aws_ecr_repository" "this" {
name = "${var.app_name}build"
image_tag_mutability = "IMMUTABLE"
encryption_configuration {
encryption_type = "KMS"
kms_key = var.kms_key_arn
}
image_scanning_configuration {
scan_on_push = true
}
tags = merge(
var.extra_tags,
{ Name = "${var.app_name}build" }
)
}
#
resource "null_resource" "packer" {
depends_on = [aws_ecr_repository_policy.this]
triggers = {
img_name = local.img_name
}
provisioner "local-exec" {
working_dir = "${path.module}/packer_build"
command = <<EOF
packer init ${path.module}/my_build.pkr.hcl && \
PACKER_LOG=1 packer build \
-var img_name=${local.img_name} \
-var repo=${aws_ecr_repository.this.repository_url} \
${path.module}/my_build.pkr.hcl
if [ $? -eq 0 ]; then
printf "\n[+] Packer build SUCCEEDED!!\n"
else
printf "\n[-] Packer build FAILED!!!\n" >&2
exit 1
fi
EOF
}
}
which fails with:
│ ': exit status 1. Output: r-push): 79a6254fc7b1: Retrying in 7 seconds
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 76ec60dbc4a7: Retrying in 6 seconds
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 79a6254fc7b1: Retrying in 6 seconds
| ....
| ....
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 79a6254fc7b1: Retrying in 1 second
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ EOF
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Logging out...
│ 2023/10/09 07:03:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin:
│ 2023/10/09 07:03:36 Executing: /usr/bin/docker [--config
│ /tmp/packer1870521035 logout
│ 99xxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com/znpexc-ibc-autbuild]
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Removing login credentials for
│ 99xxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com
│ znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Removing temporary Docker configuration directory
│ 2023/10/09 07:03:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin:
│ 2023/10/09 07:03:36 error: Bad exit status: 1
│ 2023/10/09 07:03:36 [INFO] (telemetry) ending docker-push
│
│ * Post-processor failed: Bad exit status: 1
│ ==> Wait completed after 1 minute 951 milliseconds
│ 2023/10/09 07:03:36 machine readable: error-count []string{"1"}
│ ==> Some builds didn't complete successfully and had errors:
│ 2023/10/09 07:03:36 machine readable:
│ znpexc-ibc-aut-20231009070233.docker.al2023,error []string{"1 error(s)
│ occurred:\n\n* Post-processor failed: Bad exit status: 1"}
│
│ * Post-processor failed: Bad exit status: 1
│ ==> Builds finished but no artifacts were created.
│ Build 'znpexc-ibc-aut-20231009070233.docker.al2023' errored after 1
│ minute 951 milliseconds: 1 error(s) occurred:
│
│ * Post-processor failed: Bad exit status: 1
│
│ ==> Wait completed after 1 minute 951 milliseconds
│
│ ==> Some builds didn't complete successfully and had errors:
│ --> znpexc-ibc-aut-20231009070233.docker.al2023: 1 error(s) occurred:
│
│ * Post-processor failed: Bad exit status: 1
│
│ ==> Builds finished but no artifacts were created.
│ 2023/10/09 07:03:36 [INFO] (telemetry) Finalizing.
│ 2023/10/09 07:03:36 waiting for all plugin processes to complete...
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│
│ [-] Packer build FAILED!!!
Any idea what am I doing wrong here?
It works perfectly okay with source type amazon-ebs, using assume_role{} configuration in place but I don't see anything simialr for docker build-type.
I'm trying to find out if there is anything missing in the packer HCL or issue with my ECR repo. Below is the policy that attached to the repo, if that helps...
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowPullPush",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::99xxxxxxxxxx:root"
]
},
"Action": [
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:CompleteLayerUpload",
"ecr:GetDownloadUrlForLayer",
"ecr:InitiateLayerUpload",
"ecr:ListImages",
"ecr:PutImage",
"ecr:UploadLayerPart"
]
}
]
}
as requiest by @Matthew Schuchard, this is what I see when packer runs by itself:
==> znpexc-ibc-aut-20231009203420.docker.al2023: Creating a temporary directory for sharing data...
==> znpexc-ibc-aut-20231009203420.docker.al2023: Pulling Docker image: public.ecr.aws/amazonlinux/amazonlinux:2023
2023/10/10 14:56:35 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:35 Set Packer temp dir to /home/santanu/.config/packer/tmp3020070013
2023/10/10 14:56:35 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:35 Executing: /usr/bin/docker [pull public.ecr.aws/amazonlinux/amazonlinux:2023]
znpexc-ibc-aut-20231009203420.docker.al2023: 2023: Pulling from amazonlinux/amazonlinux
znpexc-ibc-aut-20231009203420.docker.al2023: Digest: sha256:f11c9d2a038d1b658dac60a51222bfb7a5d3e04f919093528581f6ce41906b37
znpexc-ibc-aut-20231009203420.docker.al2023: Status: Image is up to date for public.ecr.aws/amazonlinux/amazonlinux:2023
znpexc-ibc-aut-20231009203420.docker.al2023: public.ecr.aws/amazonlinux/amazonlinux:2023
==> znpexc-ibc-aut-20231009203420.docker.al2023: Starting docker container...
znpexc-ibc-aut-20231009203420.docker.al2023: Run command: docker run -v /home/santanu/.config/packer/tmp3020070013:/packer-files -d -i -t --entrypoint=/bin/sh -- public.ecr.aws/amazonlinux/amazonlinux:2023
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Starting container with args: [run -v /home/santanu/.config/packer/tmp3020070013:/packer-files -d -i -t --entrypoint=/bin/sh -- public.ecr.aws/amazonlinux/amazonlinux:2023]
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Waiting for container to finish starting
znpexc-ibc-aut-20231009203420.docker.al2023: Container ID: 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff
==> znpexc-ibc-aut-20231009203420.docker.al2023: Using docker communicator to connect: 172.17.0.2
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Running the provision hook
==> znpexc-ibc-aut-20231009203420.docker.al2023: Committing the container
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Committing container with args: [commit 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff]
znpexc-ibc-aut-20231009203420.docker.al2023: Image ID: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
==> znpexc-ibc-aut-20231009203420.docker.al2023: Killing the container: 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff
==> znpexc-ibc-aut-20231009203420.docker.al2023: Running post-processor: (type docker-tag)
2023/10/10 14:56:37 [INFO] (telemetry) ending docker.al2023
2023/10/10 14:56:37 [INFO] (telemetry) Starting post-processor docker-tag
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Tagging image: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Repository: <sensitive>:latest
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Tagging image: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Repository: <sensitive>:20231010145634
2023/10/10 14:56:37 [INFO] (telemetry) ending docker-tag
2023/10/10 14:56:37 Flagging to keep original artifact from post-processor 'docker-tag'
2023/10/10 14:56:37 [INFO] (telemetry) Starting post-processor docker-push
==> znpexc-ibc-aut-20231009203420.docker.al2023: Running post-processor: (type docker-push)
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Creating temporary Docker configuration directory
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Fetching ECR credentials...
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Getting ECR token for account: 998380306071 in eu-west-2..
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Found region eu-west-2
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 [INFO] AWS authentication used: "SharedCredentialsProvider"
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Successfully got login for ECR: <sensitive>
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Logging in...
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Executing: /usr/bin/docker [--config /tmp/packer296049404 login -u AWS --password-stdin <sensitive>]
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Login Succeeded
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): WARNING! Your password will be stored unencrypted in /tmp/packer296049404/config.json.
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Configure a credential helper to remove this warning. See
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Pushing: <sensitive>:20231010145634
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Executing: /usr/bin/docker [--config /tmp/packer296049404 push <sensitive>:20231010145634]
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): The push refers to repository [<sensitive>]
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 5392f2132bbf: Preparing
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 79a6254fc7b1: Preparing
.... <goes for a min or so> ....
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 5392f2132bbf: Retrying in 1 second
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 79a6254fc7b1: Retrying in 1 second
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): EOF
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Logging out...
2023/10/10 14:57:28 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:57:28 Executing: /usr/bin/docker [--config /tmp/packer296049404 logout <sensitive>]
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Removing login credentials for 998380306071.dkr.ecr.eu-west-2.amazonaws.com
znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Removing temporary Docker configuration directory
2023/10/10 14:57:28 [INFO] (telemetry) ending docker-push
* Post-processor failed: Bad exit status: 1
Build 'znpexc-ibc-aut-20231009203420.docker.al2023' errored after 53 seconds 42 milliseconds: 1 error(s) occurred:
* Post-processor failed: Bad exit status: 1
==> Wait completed after 53 seconds 43 milliseconds
2023/10/10 14:57:28 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:57:28 error: Bad exit status: 1
==> Wait completed after 53 seconds 43 milliseconds
2023/10/10 14:57:28 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/10/10 14:57:28 machine readable: znpexc-ibc-aut-20231009203420.docker.al2023,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Bad exit status: 1"}
* Post-processor failed: Bad exit status: 1
==> Builds finished but no artifacts were created.
2023/10/10 14:57:28 [INFO] (telemetry) Finalizing.
==> Some builds didn't complete successfully and had errors:
--> znpexc-ibc-aut-20231009203420.docker.al2023: 1 error(s) occurred:
* Post-processor failed: Bad exit status: 1
==> Builds finished but no artifacts were created.
2023/10/10 14:57:28 waiting for all plugin processes to complete...
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited
-S
This post gave me the asnswer: How to Push to Account Using ASSUME_ROLE in AWS ECR
I had to add aws_profile to "docker-push", like this:
post-processor "docker-push" {
aws_profile = var.aws_profile
ecr_login = true
keep_input_artifact = false
login_server = var.img_repo
only = ["docker.ub2204"]
}
where var.aws_profile is the name of the profile in ~/.aws/config, which represents the role that terraform runs with. I can now push the image to ECR without any issue.
Thanks all for helping me out.