logginggoogle-cloud-platformgoogle-cloud-pubsubproducer-consumeraudit-logging

GCP Log Router Sink Not Routing Logs to Topic?


Context: I am trying to store GCP Audit logs in a Pub/Sub topic via a Log Sink so I can pull those messages in a subscription (and have a 3rd party subscribe to that subscription).

Attempted solution :
Log Sink with inclusion filter of: logName:"cloudaudit.googleapis.com" to route logs to -> Pub/Sub Topic to store those messages -> Subscription (Pull Based) to view those messages.

Problem: Logs from the sink do not seem to be ending up in the topic. When I try to view the messages in the topic (by clicking pull in the GCP GUI), I get back no messages, even though I know for sure that audit logs exist (I ran a query against the inclusion filter above) and continue to be generated pretty frequently.

Question: Why aren't the logs being routed to the topic OR why isn't the topic showing the logs when I pull from the topic?


Solution

  • It's a permission issue. When you create a sink, this sink as an identity, you can get it like that

    gcloud logging sinks describe \
     --format='value(writerIdentity)' <SINK_NAME>
    

    Then, grant this identity the permission to publish to pubsub, for example

    gcloud pubsub topics add-iam-policy-binding <TOPIC_ID> \
      --member=<WRITER_IDENTITY> --role=roles/pubsub.publisher
    

    EDIT 1

    Though the console