ELK RPM安裝和分析系統日志配置

1. 實驗環境

1.1 同步時間

1.1.1 安裝ntpdate指令

yum -y install ntpdate
           

1.1.2 同步時間操作

ntpdate ntp.aliyun.com
           

1.2 虛拟機準備

作業系統均為centos7

規劃：

elasticsearch 伺服器： 192.168.153.183

kibana 伺服器：192.168.153.184

logstash 伺服器： 192.168.153.185

2. elasticsearch 安裝及配置

elasticsearch伺服器IP位址 192.168.153.183

elasticsearch 需要jdk環境

2.1 安裝jdk

安裝jdk
[[email protected] elk]# rpm -ivh jdk-8u131-linux-x64_.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:jdk1.8.0_131-2000:1.8.0_131-fcs  ################################# [100%]
Unpacking JAR files...
	tools.jar...
	plugin.jar...
	javaws.jar...
	deploy.jar...
	rt.jar...
	jsse.jar...
	charsets.jar...
	localedata.jar...
[[email protected] elk]# 

驗證jdk安裝成功
[[email protected] elk]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[[email protected] elk]# 
           

2.2 安裝elasticsearch

[[email protected] elk]# rpm -ivh elasticsearch-7.3.2-x86_64.rpm
warning: elasticsearch-7.3.2-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   elasticsearch-0:7.3.2-1          ################################# [100%]
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service
Created elasticsearch keystore in /etc/elasticsearch
           

2.3 修改elasticsearch的配置檔案

[[email protected] elk]# vim /etc/elasticsearch/elasticsearch.yml 
17 cluster.name: my-application        # 叢集名稱
23 node.name: 192.168.153.183      #本節點名稱
33 path.data: /var/lib/elasticsearch   #資料存儲目錄
37 path.logs: /var/log/elasticsearch  #日志存儲目錄
55 network.host: 0.0.0.0                   #本機器所有IP都可監聽
59 http.port: 9200        					 #預設監聽端口
68 discovery.seed_hosts: ["192.168.153.183"]    # 發現的主機
72 cluster.initial_master_nodes: ["192.168.153.183"]    #主節點名稱，與上面23行的配置保持一緻
           

2.4 配置JVM（可選）

jvm 運作記憶體不足時可修改
[[email protected] elk]# vim /etc/elasticsearch/jvm.options 

 22 -Xms1g   為JVM啟動時申請的最小記憶體
 23 -Xmx1g   為JVM可申請的最大記憶體
           

2.5 啟動elasticsearch

[[email protected] elk]# systemctl start elasticsearch
           

2.6 檢視端口驗證elasticsearch是否啟動成功

9200 為elasticsearch所監聽的端口，9300 為叢集節點之間通信的端口

[[email protected] elk]# netstat -nltp|grep java
tcp6       0      0 :::9200                 :::*                    LISTEN      8920/java           
tcp6       0      0 :::9300                 :::*                    LISTEN      8920/java                
[[email protected] elk]# 
           

2.7 浏覽器輸入 IP:9200 驗證

3. kibana 安裝及配置

kibana伺服器IP位址 192.168.153.184

3.1 安裝kibana

[[email protected] ~]# rpm -ivh kibana-7.3.2-x86_64.rpm 
warning: kibana-7.3.2-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:kibana-7.3.2-1                   ################################# [100%]
[[email protected] ~]# 
           

3.2 修改kibana配置檔案

[[email protected] ~]# vim /etc/kibana/kibana.yml 
2 server.port: 5601                          # kibana預設監聽的端口
7 server.host: "192.168.153.184"    # kibana 監聽的IP位址
28 elasticsearch.hosts: ["http://192.168.153.183:9200"]    # elasticsearch 的位址和端口
114 i18n.locale: "zh-CN"   $設定kibana語言為中文
           

3.3 開啟kibana服務

[[email protected] ~]#  systemctl start kibana
           

3.4 浏覽器輸入 IP:5601 檢視

通路成功

4. logstash安裝及配置

logstash伺服器IP位址 192.168.153.185

4.1 安裝jdk環境

[[email protected] ~]# rpm -ivh jdk-8u131-linux-x64_.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:jdk1.8.0_131-2000:1.8.0_131-fcs  ################################# [100%]
Unpacking JAR files...
	tools.jar...
	plugin.jar...
	javaws.jar...
	deploy.jar...
	rt.jar...
	jsse.jar...
	charsets.jar...
	localedata.jar...
           

4.2 安裝logstash

[[email protected] ~]# rpm -ivh logstash-7.3.2.rpm 
warning: logstash-7.3.2.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:logstash-1:7.3.2-1               ################################# [100%]
Using provided startup.options file: /etc/logstash/startup.options
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/pleaserun-0.0.30/lib/pleaserun/platform/base.rb:112: warning: constant ::Fixnum is deprecated
Successfully created system startup script for Logstash
           

4.3 logstash配置檔案的位置為

/etc/logstash/logstash.yml
           

無需修改

4.4 切換 /etc/logstash/conf.d/ 目錄下，編寫以.conf結尾的配置檔案

以分析 /var/log/messages 為例

先給 /var/log/messages 增加讀的權限

[[email protected] ~]# chmod +r /var/log/messages

檢視效果
[[email protected] ~]# ll /var/log/messages
-rw-r--r--. 1 root root 136170 Jan  1 12:09 /var/log/messages
           

再編寫conf配置檔案

[[email protected] ~]# cd /etc/logstash/conf.d/
[[email protected] conf.d]# ls
system.conf
[[email protected] conf.d]# cat system.conf 
input{
  file {
    path => "/var/log/messages"
    type => "system-log"
	start_position => "beginning"
  }

}

output{
	elasticsearch {
	  hosts => "192.168.153.183:9200"  #elasticsearch伺服器的IP位址及端口号
	  index => "system-log_%{+YYYY.MM.dd}"
	}	
} 	
           

4.5 編寫 4.4 檔案之後開啟logstash服務

[[email protected] conf.d]# systemctl start logstash
           

4.6 檢視是否有logstash的日志檔案生成

[[email protected]host conf.d]# watch ls /var/log/logstash/
Every 2.0s: ls /var/log/logstash/                                                Thu Dec 31 11:37:54 2020

logstash-plain.log  
logstash-slowlog-plain.log
           

有日志檔案生成時按 Ctrl+c 退出

4.7 追蹤檢視日志是否報錯

[[email protected] conf.d]# tailf /var/log/logstash/logstash-plain.log 
[2020-12-31T09:34:09,320][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.specialized.RubyArrayOneObject) has been create for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-12-31T09:34:09,332][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x51a494ea run>"}
[2020-12-31T09:34:09,337][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_
.......
           

5. kibana 添加索引模式

5.1

5.2

5.3

5.4 建立成功

5.5

5.5 可視化的建立

1）

2）

3）檢視建立的可視化

6.分析nginx日志

6.1 在logstash伺服器上源碼安裝nginx

安裝連結>> https://blog.csdn.net/m0_46674735/article/details/111125960

6.2 在/etc/logstash/conf.d/ 目錄下，編寫以.conf結尾的配置檔案，同時分析系統日志和nginx日志

[[email protected] ~]# cd /etc/logstash/conf.d/
[[email protected] conf.d]# vim system_nginx.conf 
input {
  file {
    path => "/var/log/messages"
    type => "system.log"
    start_position => "beginning"
  }
  file {
    path => "/usr/local/nginx/logs/*.log"
    type => "nginx.log"
    start_position => "beginning"
  }
}

output {
  if [type] == "system.log" {
        elasticsearch {
          hosts => "192.168.153.183:9200"
          index => "system.log_%{+YYYY.MM.dd}"
        }
     }
  if [type] == "nginx.log" {
        elasticsearch {
          hosts => "192.168.153.183:9200"
          index => "nginx.log_%{+YYYY.MM.dd}"
        }
     }
}
           

6.3 編寫完成conf檔案後重新開機logstash服務

[[email protected] conf.d]# systemctl restart logstash
           

6.4 在kibana添加索引

1）

2）

3）

4）添加成功

5）檢視添加的nginx索引

6.5 索引運作狀态顔色為yellow的原因

隻有一台機器，卻設定了索引副本，這樣索引的副本沒有辦法配置設定，是以就是 yellow 狀态