docker google-cloud-build google-container-registry

Google Cloud Build - Summary status shows FAILURE, all steps succeded

I've setup a Google Cloud Build pipeline that'll build a docker image from a Dockerfile, test the image and push the image into Google Container Registry.

Upon running the pipeline I noticed that all defined steps passed with SUCCESS status but the build summary itself returned with FAILURE status even though I can see the image being produced into Google Container Registry.

I used following command to build the image

gcloud builds submit --config cloudbuild.yml --gcs-log-dir 'gs://<bucket>' .

and below is the error message returned:

ERROR: (gcloud.builds.submit) build www-xxxx-yyyy-zzzz completed with status "FAILURE"
🚨 Error: The command exited with status 1

Is there any reason for the gcloud builds submit command to exit with code 1 as above if all the steps were marked as SUCCESS?

Below is some filtered log data taken from gcloud builds describe command for that specific build.

steps:
- args:
  - build
  - -t
  - <host>/<project/<image>:<tag>
  - .
  name: gcr.io/cloud-builders/docker
  status: SUCCESS
- args:
  - test
  - --image
  - <host>/<project/<image>:<tag>
  - --config
  - test_config.yml
  - -o
  - json
  name: gcr.io/gcp-runtimes/container-structure-test
  status: SUCCESS

Below is Google Cloud Build setup:

# cloudbuild.yml

steps:
# Build the image
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', '<host>/<project/<image>:<tag>', '.' ]
# Test the image
- name: 'gcr.io/gcp-runtimes/container-structure-test'
  args: [ 
    'test',
    '--image',
    '<host>/<project/<image>:<tag>',
    '--config',
    'test_config.yml',
    '-o',
    'json'
  ]

# Push the image
images: [ '<host>/<project/<image>:<tag>' ]

Solution

I've finally resolved this issue with the assistance of Google Cloud support team.

They found a 403 Permission Denied error as the Cloud Build container trying to access Google Cloud Storage to delete a certain log object stored in the bucket, this error message is generated at the back system of Cloud Build where users/clients have no access to. The 403 Permission Denied error is the result of the object retention policy applied to the bucket.

In my case, I've replaced retention policy with lifecycle policy to resolve this issue and it worked. We do this as we consider keeping Cloud Build log size under control is our primary objective and, to prevent any accidental deletion/modification to the log file, we ended up with setting read-only access to the resources in the log bucket except for the service account used by Cloud Build.