pythonprometheus

prometheus custom collector with "cached" metrics


i want to have a prometheus collector that can be queried via http on port 9100 which is delivering a metric. the curl should always just provide the latest metric i am creating with my function here:

https://github.com/unnuetz/custom-prometheus-collector/blob/master/check.py#L5-L11

my problem now is, that after each cycle of 100secondes i added here: https://github.com/unnuetz/custom-prometheus-collector/blob/master/main.py#L20-L23 the next metric i am creating is added, but i would like to have a "reset" after each call of checker(), and not that the individual metrics i created are always just added to my lets say registry. is there a way, to remove all metrics which were added before? the important thing ist, that between the 100seconds the start_http_server(9100, addr='0.0.0.0') is still availabe and delivering the latest metric data it have from the last checker() run. in my demo file i always want to have just one metric with an random generated label name.

enter image description here


Solution

  • Instead of using using builtin Gauge, you can use a custom collector.

    Note that it won't save you in Prometheus:

    1. it will create a new metrics each time prometheus scrapes your application
    2. the previous metric(s) will still be visible in Prometheus until it is marked as stale (~5 min).

    If using using a custom collector is not an option because you need to show some code using global metrics. You can inject a reset() function similar to the remove() function using a bound method.

    import CollectorRegistry from prometheus_client.registry
    
    def _resetCollector(self):
        with self._lock:
            # reset whatever name / value you want
            pass
    
    CollectorRegistry.reset = _resetCollector
    
    
    # wherever you want to reset it
    import REGISTRY from prometheus_client.registry
    REGISTRY.reset()
    

    Of course, it will break whenever the implementation of CollectorRegistry change.