gokubernetesprometheuskubernetes-operatorgolangci-lint

Deletion of metrics not working kubernetes operator controller

I have been working on my operator where I have some of the custom metric setting the values and that works fine (registration and displaying the metric value). The problem is that metric deletion isn't happening. I tried to declare a separate function that delete the metric.

My operator is spinning up statefulset and service but upon deletion of my CR the child resources gets removed but metric doesn't get any update/deletion.

func (r *cr) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
......
......
        if errors.IsNotFound(err) {
            l.Info("CR not found so maybe deleted")
            l.Info("Deleting CR metric Instance")
            DeleteMetric(instance, true)
            return ctrl.Result{}, nil
        }

func DeleteMetric(cr *CR, condition bool) {
    l := logf.Log.WithName("CR Metric deletion")
    l.Info("Deleting metric for CR", "instance", cr.Name, "from namespace", cr.Namespace)
    if condition {
        custom_metric_name.Delete(prometheus.Labels{
            "name":      cr.Name,
            "namespace": cr.Namespace,
        })
    }
}

I also tried to declare predicate with DeleteFunc but no luck and my metrics still can't be deleted.

Appreciate any help or pointers.

Solution

  • I am able to get this working, deletion of metric happens to be simply by calling the custom metric with the delete function based on the finished operation of resource.

    For reference calling delete on the custom metric would work and you can call the function when the work is done on your custom resource.

    https://pkg.go.dev/github.com/prometheus/client_golang/prometheus#MetricVec.DeleteLabelValues