基于Prometheus+Grafana的應用監控系統建設（二）

Prometheus + Grafana 應用級監控方案(2)-安裝及初始化

概述

在監控系統建設過程中，建議能使用Docker部署的服務，盡量用Docker部署，包括監控系統本身(Prometheus/Grafana/InfluxDB)及衆多的采集器(export)。其帶來的好處：

• 便利性：無需複雜的編譯、安裝過程，即下即用，支援離線安裝
• 安全性：虛拟化後與主控端及其它容器隔離
• 移植性：不同的Linux發行版、Windows跨系統基本無影響
• 持續內建: 為後續的持續內建、自動化部署帶來較多的友善之處

注：本文以CentOS 7 為基礎環境安裝

Docker安裝參考

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum makecache fast
sudo yum install docker-ce -y
sudo systemctl start docker
sudo docker info           

Prometheus 安裝參考

docker pull prom/prometheus
mkdir -p /data/docker-lv/prometheus/etc
mkdir -p /data/docker-lv/prometheus/data
docker run --name tmp-prometheus --rm  prom/prometheus 
#開一個新視窗，提取檔案，其中配置與資料為主控端映射
docker cp tmp-prometheus:/etc/prometheus/prometheus.yml /data/docker-lv/prometheus/etc/.
#中止tmp-prometheus(ctrl+C)
docker run -d --name prometheus \
       -p 9090:9090 \
       -v /data/docker-lv/prometheus/etc:/etc/prometheus/ \
       -v /data/docker-lv/prometheus/data:/prometheus-data \
       prom/prometheus
#檢視啟動日志
docker logs prometheus     

#浏覽器打開
 http://host:9090           

Grafana 安裝參考

docker pull grafana/grafana
mkdir -p /data/docker-lv/grafana/data
mkdir -p /data/docker-lv/grafana/etc
#chmod 777 /data/docker-lv/grafana/data
chown 472:472 /data/docker-lv/grafana/*
docker run --name tmp-grafana --rm  grafana/grafana
#開一個新視窗，提取檔案，其中配置與資料為主控端映射
docker cp tmp-grafana:/etc/grafana/grafana.ini /data/docker-lv/grafana/etc/.
chown 472:472 -R /data/docker-lv/grafana/*
#中止tmp-prometheus(ctrl+C)
docker run -d --name=grafana \
           -p 3000:3000 \
           -v /data/docker-lv/grafana/etc/grafana.ini:/etc/grafana/grafana.ini \
           -v /data/docker-lv/grafana/data:/var/lib/grafana \
           grafana/grafana
#檢視啟動日志
docker logs grafana   
#浏覽器打開
 http://host:3000
 admin/admin           

Grafana 第一個Dashboard

• 建立資料源，類型為 Prometheus, 位址為 http://xxx.xxx.x.xx:9090
• 建立Dashboard，Prometheus已有資料，可用其制作圖表
• 以下為樣例Dashboard, 菜單 --> + --> Import --> JSON 粘貼以下JSON字元串

{"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":true,"gnetId":null,"graphTooltip":0,"id":1,"links":[],"panels":[{"cacheTimeout":null,"colorBackground":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","decimals":2,"format":"s","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":2,"w":6,"x":0,"y":0},"id":4,"interval":null,"links":[],"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"options":{},"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":false,"ymax":null,"ymin":null},"tableColumn":"","targets":[{"expr":"time() - process_start_time_seconds","refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"運作時間","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"cacheTimeout":null,"colorBackground":false,"colorPostfix":false,"colorValue":false,"colors":["#299c46","rgba(237, 129, 40, 0.89)","#d44a3a"],"datasource":"Prometheus","format":"decbytes","gauge":{"maxValue":100,"minValue":0,"show":false,"thresholdLabels":false,"thresholdMarkers":true},"gridPos":{"h":2,"w":5,"x":6,"y":0},"id":6,"interval":null,"links":[],"mappingType":1,"mappingTypes":[{"name":"value to text","value":1},{"name":"range to text","value":2}],"maxDataPoints":100,"nullPointMode":"connected","nullText":null,"options":{},"postfix":"","postfixFontSize":"50%","prefix":"","prefixFontSize":"50%","rangeMaps":[{"from":"null","text":"N/A","to":"null"}],"sparkline":{"fillColor":"rgba(31, 118, 189, 0.18)","full":false,"lineColor":"rgb(31, 120, 193)","show":true,"ymax":null,"ymin":null},"tableColumn":"","targets":[{"expr":"process_resident_memory_bytes","refId":"A"}],"thresholds":"","timeFrom":null,"timeShift":null,"title":"記憶體使用","type":"singlestat","valueFontSize":"80%","valueMaps":[{"op":"=","text":"N/A","value":"null"}],"valueName":"avg"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"Prometheus","decimals":0,"fill":1,"fillGradient":0,"gridPos":{"h":7,"w":24,"x":0,"y":2},"hiddenSeries":false,"id":2,"legend":{"alignAsTable":true,"avg":false,"current":true,"max":true,"min":false,"rightSide":true,"show":true,"total":false,"values":true},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"expr":"go_goroutines","legendFormat":"{{instance}}","refId":"A"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"協程數","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"schemaVersion":21,"style":"dark","tags":[],"templating":{"list":[]},"time":{"from":"now-6h","to":"now"},"timepicker":{},"timezone":"","title":"prometheus-view","uid":"HbfbwzBWk","version":5}           

• 顯示效果如下圖所示

  

小結

• Prometheus + Grafana 安裝&使用都比較友善，很快可以搭建一個看上去還不錯的運作界面
• 上手很快，無需編碼
• 下一節介紹一個典型的應用監控實作過程- Prometheus+Grafana+Redis監控