dockerprometheusprometheus-alertmanagerblack-box-testingprometheus-blackbox-exporter

Prometheus x509: certificate signed by unknown authority

I was trying to monitor one of our webservices by using Prometheus but I faced some problem for some our endpoints.

my prometheus.yml

job_name: 'blackbox'
metrics_path: /probe
params:
  module: [http_2xx_example]
static_configs:
  - targets:
    # Target to probe with https.
    - https://test-services.xxxxx.com/ 
    - https://test-services1.ffe.yy.com/
    - https://test-services2.ffe.yy.com/ 
    - http://test-services3.xxxxx.com/ 
tls_config:
  insecure_skip_verify: true
relabel_configs:
  - source_labels: [__address__]
    target_label: __param_target
  - source_labels: [__param_target]
    target_label: instance
  - target_label: __address__
    replacement: xx:xx:xx:xx:9115  # The blackbox exporter's real hostname:port.

Although we don't have any problem with https://test-services1.ffe.yy.com/ and https://test-services2.ffe.yy.com/, but ı have problem with - https://test-services.xxxxx.com/ and http://test-services3.xxxxx.com/.

it’s showing some error like –

/prometheus: x509: certificate signed by unknown authority, Prometheus service is UP and also able to monitor some of the http requests.

How can I solve this problem.

Solution

  • Segment

    tls_config:
  insecure_skip_verify: true

    from config you provided in question is responsible for configuration of tls between prometheus and blackbox_exporter.

    Your error on the other hand occurs on connection between blackbox_exporter and target services. Check blackbox exporter's configuration: most likely list of CAs there out of date.