graphite安裝筆記
實驗環境： centos7.5 x64
部署python的程式推薦使用virtualenv這列的虛拟環境來進行，防止污染系統自帶的python包。
安裝一些必要的rpm包
yum install cairo -- 另外的2個包 cairo-devel pycairo 可以不用安裝
安裝virtualenv
yum install python-pip
pip install virtualenv
激活虛拟環境
virtualenv /opt/graphite
source /opt/graphite/bin/activate
安裝graphite全套元件
export pythonpath="/opt/graphite/lib/:/opt/graphite/webapp/"
pip install --no-binary=:all: https://github.com/graphite-project/whisper/tarball/master
pip install --no-binary=:all: https://github.com/graphite-project/carbon/tarball/master
pip install --no-binary=:all: https://github.com/graphite-project/graphite-web/tarball/master
如果這步提示某個依賴包版本不對，我們可以先人工執行下pip安裝依賴包。下面提供了一個正常運作的graphite上看到的pip包清單：
attrs==19.1.0
automat==0.7.0
cachetools==3.1.1
cairocffi==0.9.0
cffi==1.12.3
constantly==15.1.0
django==1.11.24
django-tagging==0.4.3
gunicorn==19.9.0
hyperlink==19.0.0
idna==2.8
incremental==17.5.0
pycparser==2.19
pyhamcrest==1.9.0
pyparsing==2.4.2
pytz==2019.2
scandir==1.10.0
six==1.12.0
twisted==19.7.0
txamqp==0.8.2
urllib3==1.25.3
whisper==1.2.0
zope.interface==4.6.0
初始化
cd /opt/graphite/webapp/graphite
cp local_settings.py.example local_settings.py
vim local_settings.py 修改如下3行代碼：
conf_dir = '/opt/graphite/conf'
storage_dir = '/opt/graphite/storage'
static_root = '/opt/graphite/webapp/content'
log_dir = '/opt/graphite/storage/log/webapp'
初始化下資料，不然後面啟動graphite後會提示no such user 這類報錯
cd /opt/graphite/webapp
pythonpath=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb ## 如果上面第一步時候沒有yum install cairo的話，這步會報錯
拷貝graphite的配置檔案
cd /opt/graphite/conf/
cp carbon.conf.example carbon.conf
cp storage-schemas.conf.example storage-schemas.conf 一般會改動這個檔案裡面的配置，達到自定義key存儲周期
cp relay-rules.conf.example relay-rules.conf
cp dashboard.conf.example dashboard.conf
cp graphtemplates.conf.example graphtemplates.conf
cp blacklist.conf.example blacklist.conf
cp aggregation-rules.conf.example aggregation-rules.conf
cp storage-aggregation.conf.example storage-aggregation.conf
啟動graphite套件
/opt/graphite/bin/carbon-relay.py start
/opt/graphite/bin/carbon-cache.py start
ss -lntp | egrep carbon
listen 0 50 *:7002 *:* users:(("carbon-cache.py",pid=48433,fd=11))
listen 0 50 *:2013 *:* users:(("carbon-relay.py",pid=37683,fd=13))
listen 0 50 *:2014 *:* users:(("carbon-relay.py",pid=37683,fd=15))
listen 0 50 *:2003 *:* users:(("carbon-cache.py",pid=48433,fd=13))
listen 0 50 *:2004 *:* users:(("carbon-cache.py",pid=48433,fd=14))
/opt/graphite/bin/run-graphite-devel-server.py --port=8085 --libs=/opt/graphite/webapp /opt/graphite 前台啟動
啟動打點測試程序
python /opt/graphite/examples/example-client.py
安裝nginx
yum install nginx
cd /etc/nginx/conf.d
cat graphite.conf 内容如下：
upstream graphite {
server 127.0.0.1:8085 fail_timeout=0;
}
server {
listen 80 default_server;
server_name _;
root /opt/graphite/webapp;
access_log /var/log/nginx/graphite.access.log;
error_log /var/log/nginx/graphite.error.log;
location = /favicon.ico {
return 204;
}
# serve static content from the "content" directory
location /static {
alias /opt/graphite/webapp/content;
expires max;
location / {
try_files $uri @graphite;
location @graphite {
proxy_pass_header server;
proxy_set_header host $http_host;
proxy_redirect off;
proxy_set_header x-real-ip $remote_addr;
proxy_set_header x-scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://graphite;
建立nginx相關檔案
touch /var/log/nginx/graphite.access.log
touch /var/log/nginx/graphite.error.log
chmod 640 /var/log/nginx/graphite.*
chown nginx.nginx /var/log/nginx/graphite.*
啟動nginx
nginx -t
systemctl start nginx
通路 192.168.2.4/ 如下圖
可以看到已經有資料采集到了。
我們可以到grafana裡面配下graphite的資料源，然後展示資料，類似如下：
啟動brubeck（前台） 【一款類似statsd的工具，隻是不需要nodejs環境】
yum install jansson jansson-devel
yum install libmicrohttpd libmicrohttpd-devel
git clone https://github.com/github/brubeck.git
cd brubeck/
./script/bootstrap
mkdir /usr/local/brubeck/
cp brubeck /usr/local/brubeck/
vim /usr/local/brubeck/brubeck.json 增加配置檔案如下：
{
"sharding" : false,
"server_name" : "prod-devops-graphite-01",
"dumpfile" : "/var/lib/brubeck.dump",
"capacity" : 15,
"expire" : 20,
"http" : ":8000",
"backends" : [
{
"type" : "carbon",
"address" : "localhost",
"port" : 2003,
"frequency" : 10
],
"samplers" : [
"type" : "statsd",
"address" : "0.0.0.0",
"port" : 8135,
"workers" : 8,
"multisock" : true,
"multimsg" : 8
]
【brubeck監聽在8135端口上，用于接收業務方的打點。2003 端口指的是carbon-cache的端口】
./brubeck --config brubeck.json 前台啟動程序
測試打點到brubeck
echo "db.dba.dble2.front_conn:11111|g" | nc -u -w1 127.0.0.1 8135
修改graphite存儲資料的周期
cd /opt/graphite/conf
vim storage-schemas.conf
# schema definitions for whisper files. entries are scanned in order,
# and first match wins. this file is scanned for changes every 60 seconds.
#
# [name]
# pattern = regex
# retentions = timeperpoint:timetostore, timeperpoint:timetostore, ...
# carbon's internal metrics. this entry should match what is specified in
# carbon_metric_prefix and carbon_metric_interval settings
[carbon]
pattern = ^carbon\.
retentions = 60:90d
[service-sla]
pattern = ^service.sla-pct.*
retentions = 10s:2d,1min:8d,10min:365d
[service1-rule]
pattern = ^service1\.timers\.rule.*
retentions = 10s:5min,1min:1h,10min:7d
[service1-ic]
pattern = ^service1\.counters\.ic.*
retentions = 10s:24h,1min:7d
[logstash]
pattern = service1\.counters\.logstash.*
[service1-timers]
pattern = ^service1\.timers\.*
retentions = 10s:5min,1min:1h,10min:1d,1h:365d
[service]
pattern = ^service1.*
retentions = 10s:24h,1min:6d,10min:365d
[counter]
pattern = ^.*\.counters\..*
retentions = 10s:24h,1min:7d,10min:30d
[timers]
pattern = ^.*\.timers\..*
retentions = 10s:5min,1min:1h,10min:1d,1h:30d
[db]
pattern = ^db\.*
retentions = 1min:30d
[default_10s_for_1hour]
pattern = .*
retentions = 10s:1h
vim storage-aggregation.conf 内容如下：
[min]
pattern = \.min$
xfilesfactor = 0.01
aggregationmethod = min
[max]
pattern = \.max$
aggregationmethod = max
[all_sum]
pattern = \.sum$
aggregationmethod = sum
[all_count]
pattern = \.count$
[service1_gauges]
pattern = ^service1.gauges
aggregationmethod = last
[service1_counters]
pattern = ^service1\.counters
[service1]
[default_average]
aggregationmethod = average
注意， 修改為上面的2個檔案後， 我們需要重新開機 carbon-cache程序才能生效。
建議：
1、graphite-web程序，建議通過supervisor來控制啟停。
2、個人認為brubeck比statsd更好使，例如有些機器上可能不便于部署nodejs環境