I have a few Google Cloud projects that accumulate a substantial amount of logs, around 200GB per month, and this is expected to nearly double in the next period. Initially, I assumed it would be straightforward to manage this by adjusting the retention settings: Edit the log bucket, set the retention period in days, and then update the bucket.
I configured this more than six months ago, setting it to one day for a non-production project. However, it seems that nothing gets deleted until the beginning of the next month. The logs continue to accumulate daily and only reset on the first day of the following month.
After reviewing the documentation and various Stack Overflow questions, I discovered that retention isn't the period for log deletion; it's actually the duration to preserve logs. To address this, we need to establish a lifecycle for the objects within the bucket. Unfortunately, the bucket in question (_Default logging bucket) is a system bucket, and I can't configure it.
Even when I delete logs using gcloud logging logs delete
, the storage remains unchanged and doesn't reset until the next month. Is there a way to set the logs' lifecycle to be less than a month?
Edit: Actually, retention should delete the logs, but I don't think this is happening. My logs for all projects.
retentionDays integer
Logs will be retained by default for this amount of time, after which
they will automatically be deleted. The minimum retention period is 1
day. If this value is set to zero at bucket creation time, the
default time of 30 days will be used.
https://cloud.google.com/logging/docs/reference/v2/rest/v2/locations.buckets
I believe I solved my own question; it is a misunderstanding of how Cloud Logging pricing works.
That's why it says ingestion, not storage.
Logging charges for the pre-indexed volume of logs data that is stored in
the _Default log bucket and in user-defined log buckets, when the total
volume exceeds the free monthly allotment. Every write of a log entry to
the _Default log bucket or to a user-defined log bucket counts toward your
storage allotment. For example, if you have sinks that route a log entry to
three log buckets, then that log entry is stored three times.
And the billable storage is 0 because retention was set to 1 day.
Logging charges retention costs when the logs are retained longer than the
default retention period. You can't configure the retention period for the
_Required log bucket. There are no retention costs when logs are stored
only for the default retention period of the log bucket.
https://cloud.google.com/stackdriver/pricing#storage-pricing