How to clear Stackdriver logs in Google Cloud Platform?

Question

I recently realized I was paying way too much for my logs:

As you can see in the image, logs are getting bigger each month

As you also can see I just today put a "limit" on the ingestion. Hopefully this will slow things down.

But as I understand it, my logs have gotten so big that I have to pay for their retention each month. I cannot figure out how to: a) delete logs of a certain period (or just all of them) b) make logs auto delete after x days

I also just today put a quota limit of 100 instead of 6000

Answer 1

The logs expire according to the retention policy:

Admin Activity        400 days
System Events         400 days
Data Access           30 days
Access Transparency   30 days
Other Logs            30 days

Note that you're not charged for Admin Activity or System Event logs.

Some solutions to control costs are exclusions and exports, but even if you use timestamp to specify the range of dates in the filter expressions to create an exclusion filter, since it's already loaded, it won't be excluded. The same applies to creating a log sink for exporting data, since it will export future matching logs.

You can use gcloud logging logs delete to delete all the logs for a given project or for a given resource, but you can't specify a range of time.

So, my suggestions are the next ones:

1.- Delete all the existing logs for resources you don't need logging.

2.- Create exclusions to keep only the logs you may need during 30 days.

3.- Create export sinks for all the logs you may need for more than 30 days.