Prometheus - Blackbox Exporter

BlackBox Exporter 顧名思義就是在應用程式的外部對其進行探測，

支援 HTTP、HTTPS、DNS、TCP、ICMP等方式對目标進行檢測。

ICMP

cat prometheus.yml

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs: 
  - job_name: "ICMP"
    metrics_path: /probe
    params:
      module: [icmp]
    file_sd_configs:
    - refresh_interval: 10s
      files:
      - targets/blackbox-exporter-icmp.yml
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: blackbox-exporter-service:9115      

cat targets/blackbox-exporter-icmp.yml

- targets: 
  - 119.29.29.29
  - 223.5.5.5      

HTTP

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs: 
  - job_name: "HTTP"
    metrics_path: /probe
    params:
      module: [http_2xx]
    file_sd_configs:
    - refresh_interval: 10s
      files:
      - targets/blackbox-exporter-http.yml
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: blackbox-exporter-service:9115      

cat targets/blackbox-exporter-http.yml

- targets: 
  - https://www.baidu.com
  - https://www.google.com      

TCP

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs: 
  - job_name: "TCP"
    metrics_path: /probe
    params:
      module: [tcp]
    file_sd_configs:
    - refresh_interval: 10s
      files:
      - targets/blackbox-exporter-tcp.yml
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: blackbox-exporter-service:9115      

cat targets/blackbox-exporter-tcp.yml

- targets: 
  - aliyun.com:443
  - huaweicloud.com:443      

标簽重改& 監控原理

将 [__address__] 輸出到 __param_target，是為了 Prometheus 在BlackboxExporter采集時使用對應的域名

例如 Target 中設定的是

https://www.baidu.com

則 __address__ 為

Prometheus 采集時則使用 http://blackbox-exporter:9115/probe?module=http_2xx&target=https://www.baidu.com

由此可見，Prometheus 主動傳遞參數給 Blackbox 進行執行，并在 Blackbox 接口暴露出名額提供給 Prometheus 采集。

probe?module=http_2xx&target=https://www.google.com

http_2xx 為子產品名稱

https://www.google.com

為執行監控的目标

Grafana

Dashboard ID: 13659

Dashboard ID: 9965

需要額外安裝餅圖插件支援： grafana-cli plugins install grafana-piechart-panel

Trouble

報錯: 通過 IPv6 連接配接不到目标

解決方案: 修改對應子產品的IP協定為 IPv4

modules:
  http_2xx:
    prober: http
    http:
 preferred_ip_protocol: "ip4"
  icmp:
    prober: icmp
    icmp:
 preferred_ip_protocol: "ip4"
  tcp:
    prober: tcp
    tcp:
 preferred_ip_protocol: "ip4"      

Refer:

https://github.com/prometheus/blackbox_exporter/blob/master/CONFIGURATION.md

Debug

probe?target=119.29.29.29&module=icmp&debug=true

通過傳遞 debug=true 參數可以進行 Probe 過程中的調試