系統版本：Ubuntu20.04.2LTS
Tshark：thark_3.2.3-1
Logstash：logstash-7.12.0-amd64.deb
Elasticsearch：elasticsearch-7.12.0-amd64.deb
Kibana：kibana-7.12.0-amd64.deb
系統結構圖參考：
1. 下載下傳logstash；
# wget https://artifacts.elastic.co/downloads/logstash/logstash-7.12.0-amd64.deb
2. 下載下傳elasticsearch；
# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-amd64.deb
3. 下載下傳kibana；
# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-amd64.deb
4. 下載下傳tsharkVM git包（主要用到裡面一個處理腳本）；
# git clone https://github.com/H21lab/tsharkVM.git
5. 修改Ubuntu的source源味國内鏡像源；
6. 更新系統;
# apt update && apt upgrade && reboot
1、 安裝tshark和java
2、 安裝ELK
1、 elasticsearch配置
# cd /etc/elasticsearch/
# vim jvm.options
# vim elasticsearch.yml
# systemctl daemon-reload
重新開機服務；
# systemctl restart elasticsearch.service
檢視服務狀态；
# systemctl status elasticsearch.service
●elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-08-13 02:55:08 UTC; 4h 7min ago
Docs: https://www.elastic.co
Main PID: 271265 (java)
Tasks: 99 (limit: 19109)
Memory: 8.6G
CGroup: /system.slice/elasticsearch.service
├─271265 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreT>
└─271470 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
Aug 13 02:54:48 elas systemd[1]: Starting Elasticsearch...
Aug 13 02:55:08 elas systemd[1]: Started Elasticsearch.
檢視服務監聽端口；
# netstat -tlunp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 661/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 780/sshd: /usr/sbin
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 265188/sshd: root@p
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 270722/node
tcp6 0 0 :::9300 :::* LISTEN 271265/java
tcp6 0 0 :::22 :::* LISTEN 780/sshd: /usr/sbin
tcp6 0 0 ::1:6010 :::* LISTEN 265188/sshd: root@p
tcp6 0 0 127.0.0.1:9600 :::* LISTEN 270442/java
tcp6 0 0 :::9200 :::* LISTEN 271265/java
udp 0 0 127.0.0.53:53 0.0.0.0:* 661/systemd-resolve
2、 logstash配置
# cd /etc/logstash/
# cd conf.d/
增加tshark資料輸入到elasticsearch配置檔案；
重新開機服務
# systemctl restart logstash.service
檢視伺服器狀态
# systemctl status logstash.service
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-08-13 02:10:24 UTC; 5h 17min ago
Main PID: 270442 (java)
Tasks: 72 (limit: 19109)
Memory: 2.6G
CGroup: /system.slice/logstash.service
└─270442 /usr/share/logstash/jdk/bin/java -Xms2g -Xmx2g -XX:+UseG1GC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.a>
Aug 13 02:10:46 elas logstash[270442]: [2021-08-13T02:10:46,739][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://loc>
Aug 13 02:10:46 elas logstash[270442]: [2021-08-13T02:10:46,798][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
Aug 13 02:10:46 elas logstash[270442]: [2021-08-13T02:10:46,802][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event fi>
Aug 13 02:10:46 elas logstash[270442]: [2021-08-13T02:10:46,857][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs:>
Aug 13 02:10:47 elas logstash[270442]: [2021-08-13T02:10:47,110][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers>
Aug 13 02:10:48 elas logstash[270442]: [2021-08-13T02:10:48,070][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.>
Aug 13 02:10:48 elas logstash[270442]: [2021-08-13T02:10:48,325][INFO ][logstash.inputs.file ][main] No sincedb_path set, generating one based on the "path" se>
Aug 13 02:10:48 elas logstash[270442]: [2021-08-13T02:10:48,352][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
Aug 13 02:10:48 elas logstash[270442]: [2021-08-13T02:10:48,409][INFO ][filewatch.observingtail ][main][6efc126c2ab1b0e66a90b494d9a18ae7dc55003ac5f9addc874a5130d9>
Aug 13 02:10:48 elas logstash[270442]: [2021-08-13T02:10:48,417][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_>
3、 kibana配置
# cd /etc/kibana/
# vim kibana.yml
# systemctl restart kibana.service
# systemctl status kibana.service
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-08-13 02:19:08 UTC; 5h 16min ago
Docs: https://www.elastic.co
Main PID: 270722 (node)
Tasks: 11 (limit: 19109)
Memory: 267.2M
CGroup: /system.slice/kibana.service
└─270722 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest=/var/log/kibana/kibana.log --pid.file=/run/kibana>
Aug 13 02:19:08 elas systemd[1]: Started Kibana.
檢視服務監聽端口
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 661/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 780/sshd: /usr/sbin
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 265188/sshd: root@p
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 270722/node
tcp6 0 0 :::9300 :::* LISTEN 271265/java
tcp6 0 0 :::22 :::* LISTEN 780/sshd: /usr/sbin
tcp6 0 0 ::1:6010 :::* LISTEN 265188/sshd: root@p
tcp6 0 0 127.0.0.1:9600 :::* LISTEN 270442/java
tcp6 0 0 :::9200 :::* LISTEN 271265/java
udp 0 0 127.0.0.53:53 0.0.0.0:* 661/systemd-resolve
1、 導出elastic-mapping；
# cd tsharkVM/Kibana
# tshark -G elastic-mapping --elastic-mapping-filter frame,eth,ip,udp,tcp,dns > custom_tshark_mapping.json
2、 處理重複字段和類型；(腳本處理完後，還有一些字段類型不比對，在logstash導入資料時會有提示，根據提示，修改後再重新導入)；
# ruby ./Public/process_tshark_mapping_json.rb
3、 導入模闆檔案；
# curl -X PUT "localhost:9200/_index_template/packets_template" -H 'Content-Type: application/json' \
-d@custom_tshark_mapping_deduplicated.json
{"acknowledged":true}
#
1、 使用tshark将資料包轉換為elasticsearch格式，由logstacsh輸出到elasticsearch；
# tshark –T ek –x –r 1030.pcap > /data/packets.json
2、 在kibana上建立index patterns;
自定義展示;