Loki安裝部署并收集K8S日志
loki 官方推薦的最佳實踐為采用 DamonSet部署 promtail 的方式,将 node 的 /var/lib/pods目錄挂載進容器内部,借助prometheus 的服務發現機制動态的為日志加上标簽,無論是資源的占用程度還是部署維護難度都是非常低。這也是主流的雲原生日志采集範式。
Loki收集k8s架構圖:

安裝Loki
采用 helm 方法部署 建立名稱空間
kubectl create namespace monitoring
安裝Loki 注意:
不做持久化:将 persistence.enabled=true 改為 persistence.enabled=false 即可
時間本地化
service.httpNodePort 指定http端口
service.grpcNodePort 指定grpc端口
serviceMonitor.enabled 讓普羅米修斯監控到名額
--set serviceMonitor.labels="prometheus-operated"
k8s版本低無法安裝修改版本:kubeVersion: 1.10.0-0
--set ingress.enabled=ture 擷取ingress名額
添加倉庫:
helm repo add kube-ops https://charts.kube-ops.io
helm repo update
部署 Loki
helm install loki -n monitoring \
--set replicas=1 \
--set promtail.enabled=true \
--set service.type=ClusterIP \
--set persistence.enabled=false \
--set extraVolumeMounts[0].name=localtime \
--set extraVolumeMounts[0].mountPath="/etc/localtime" \
--set extraVolumes[0].name=localtime \
--set extraVolumes[0].hostPath.path="/etc/localtime" \
kube-ops/loki --version 1.6.1
檢視pod狀态
[root@master ~]# kubectl get pod -n monitoring
NAME READY STATUS RESTARTS AGE
loki-0 1/1 Running 1 12h
loki-promtail-99ztv 1/1 Running 0 3m20s
loki-promtail-nb2q4 1/1 Running 0 4m50s
loki-promtail-pzxcl 1/1 Running 0 2m34s
配置Loki為資料源:确認Loki服務端口: Loki服務端口号是80
[root@master ~]# kubectl get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
loki ClusterIP 10.233.2.239 <none> 80/TCP,9095/TCP 12h
導入Dashboard
可以看到日志來了 Logging Dashboard via Loki dashboard for Grafana | Grafana Labs
[root@master ~]# kubectl exec -it grafana-757fcd5f7c-lkxrx -n ops
error: you must specify at least one command for the container
[root@master ~]# kubectl exec -it grafana-757fcd5f7c-lkxrx -n ops bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
bash-5.0$ grafana-cli plugins install grafana-piechart-panel
installing grafana-piechart-panel @ 1.6.2
from: https://grafana.com/api/plugins/grafana-piechart-panel/versions/1.6.2/download
into: /var/lib/grafana/plugins
?.Installed grafana-piechart-panel successfully
Restart grafana after installing plugins . <service grafana-server restart>
bash-5.0$ service grafana-server restart
bash: service: command not found
bash-5.0$ exit
exit
command terminated with exit code 127
[root@master ~]# kubectl delete pod grafana-757fcd5f7c-lkxrx -n ops
pod "grafana-757fcd5f7c-lkxrx" deleted