nginxgoogle-cloud-platformgoogle-logging

Use log filenames as Log names on Google logging


How could I use log filenames as log names on google logging.

I have this on google-cloud-ops-agent/config.yaml

logging:
  receivers:
    nginx_access:
      type: nginx_access
    nginx_error:
      type: nginx_error
    nginx_access_files:
      type: files
      include_paths: [/var/log/nginx/*access*.log]
  service:
    pipelines:
      nginx:
        receivers:
          - nginx_access
          - nginx_error
          - nginx_access_files
metrics:
  receivers:
    nginx:
      type: nginx
      stub_status_url: http://127.0.0.1:80/status
  service:
    pipelines:
      nginx:
        receivers:
          - nginx

The problem is that all logs from folder /var/log/nginx/access.log shows up on google logging as one entity "nginx_access_files". The folder has hundreds of access logs with the site name on them. I would like to have that log filename as the log name on the logging console too.

This I have now on Google logging console: enter image description here

And I would need to set up it so that the filename shows upon logging console. Like the default, Nginx error does on OPS agent. Or is there another way to route all access logs to google logging with the filenames?


Solution

  • There is no need to configure the nginx_access_files receiver, if you are using the default path for the access.log file, then the agent automatically pulls the logs and sends them to Cloud Logging.

    The agent yaml file should be something like this:

    logging:
      receivers:
        nginx_access:
          type: nginx_access
        nginx_error:
          type: nginx_error
      service:
        pipelines:
          nginx:
            receivers:
              - nginx_access
              - nginx_error
    metrics:
      receivers:
        nginx:
          type: nginx
          stub_status_url: http://127.0.0.1:80/status
      service:
        pipelines:
          nginx:
            receivers:
              - nginx
    

    Under the nginx_access log name, you will see all the access logs generated and since they are generated with the instance name as a label, you can use it to filter them: enter image description here

    If your access.log file is not in the default path, then you can specify where it is, but there is no need to create a new receiver for that, this is an example:

    logging:
      receivers:
        nginx_access:
          type: nginx_access
          include_paths: [/your/log/path/access.log]
        nginx_error:
          type: nginx_error
      service:
        pipelines:
          nginx:
            receivers:
              - nginx_access
              - nginx_error
    metrics:
      receivers:
        nginx:
          type: nginx
          stub_status_url: http://127.0.0.1:80/status
      service:
        pipelines:
          nginx:
            receivers:
              - nginx
    

    Finally, I recommend you to take a look at the NGINX Cloud Logging integration tutorial.