nginxwebserverdevopsproduction-environmentreliability

How to make make nginx run a script whenever there's a 502 error


So here's what I'm trying to do. Whenever nginx returns a 502, we want it to send a message to our dashboard with the url path that returned a 502.

Then in our dashboard we will display in descending order the urls that returned 502 and their respective count

Example:

+--------------+-------+
|     url      | count |
+--------------+-------+
| /foo/bar     |    45 |
| /foo/bar/123 |    23 |
| /foo/bar/baz |    12 |
+--------------+-------+

I'm assuming this is possible with nginx directives, and just figuring how to do go about it.

I'm also concerned how to handle overloads let's say our servers get overloaded for legitimate reasons and this causes 502 errors to spike up, there will also be a burst of traffic to our dashboard.


Solution

  • Following comments, although it is still log monitoring solution but I think it might alleviate your concern with overhead. Still there could be another answer.

    Main idea - create a separate log where you would only log 502 errors and react on that.

    First, switch 502 error to its own location. Then create location block for this error (it must be valid). Finally, pipe it into its own log:

    error_page   502  /502.html;
    location = /502.html {
        root   /var/www/html;
        access_log  /var/log/nginx/502_error.log  main;
    }
    

    This way you would only get 502s in this special log file and therefore overhead will be low relative to full-blown log monitoring. You'd also need to play with log format to get correct representation of initial pages.

    Also, if using proxy (which I presume you do) make sure you have

    proxy_intercept_errors on;
    

    for this to work.

    Hope this helps, but maybe somebody has a better solution without using logs.