google-cloud-platformgoogle-cloud-logging

Cloud Batch: "Sent all pending logs" message with severity ERROR in GCP Logging


I am running a simple Cloud Batch job via the Cloud Workflow in the GCP. Here is the batch job code.

import json
import os
import logging
import google.cloud.logging

log_client = google.cloud.logging.Client()
log_client.setup_logging()

# total arguments
file_task_records = json.loads(os.environ['FILE_TASK_RECORDS'].replace("'", '"'))
logging.info(f'BATCH JOB: {file_task_records}')

The job runs as intended, but the Log Explorer shows three messages with severity ERROR after the job is complete:

Program shutting down, attempting to send 1 queued log entries to Cloud Logging...
Waiting up to 5 seconds. 
Sent all pending logs. 

enter image description here

enter image description here

Why is there an error? How can I fix it?


Solution

  • In your python code, you are using google.cloud.logging library. However, for Google Batch, you don't need to use the cloudLogging library. Just direct your log to stdout and stderr. You'll get the log sending to the cloudLogging. So as you can see,logging.info(f'BATCH JOB: {file_task_records}') will have the write logging because it use the logging package and will write the log to stdout.

    When the program exit, the cloudLogging client will print several logs to stderr code. That's why you are seeing those extra logs.

    A screenshot for better explanation