docker下logstash搭建

1.下載下傳鏡像

老生常談，沒啥好說的。

docker pull logstash:7.5.1      

2.建立挂載檔案

此處不詳談，如果有疑問可以參考上文redis安裝，有具體解釋。

mkdir -p /usr/local/logstash/conf.d
mkdir -p /usr/local/logstash/config
mkdir -p /usr/local/logstash/logs      

3.賦權

chmod -777 /usr/local/logstash      

4.挂載配置檔案

1.挂載配置檔案

将logstash.yml放入/usr/local/logstash/config/中，在容器啟動後，使用的就是該檔案配置。

logstash.yml

：

http.host: "0.0.0.0"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: "http://192.168.xx.xx:9200"  #es位址
xpack.monitoring.elasticsearch.username: "elastic"  #es xpack賬号密碼
xpack.monitoring.elasticsearch.password: "xxxx"     #es xpack賬号密碼
path.config: /usr/share/logstash/config/conf.d/*.conf
path.logs: /usr/share/logstash/logs      

注意 http.host: "0.0.0.0" 而不是指定ip

2.挂載日志收集檔案

将log_to_es.conf放入/usr/local/logstash/conf.d/，在收集日志時，使用的就是該配置。

3.log_to_es.conf配置

如果想具體了解配置含義，後續樓主會開展elk專欄，請持續關注樓主。

log_to_es.conf：

input{
        tcp {
                mode => "server"
                host => "0.0.0.0"
                port => 5000
                codec => json_lines
                type=> "datalog"
        }
        tcp {
                mode => "server"
                host => "0.0.0.0"
                port => 4999
                codec => json_lines
                type=> "loginlog"
 
        }
}
filter{
            if[type] == "loginlog"{
                grok {
                    match => {"message" => "|%{GREEDYDATA:loginMsg}|%{GREEDYDATA:timeFormat}|%{GREEDYDATA:userName}"}
                }
                if([message] =~  "^(?!.*?登入系統).*$") {
                    ### 丢棄
                    drop{}
                   }
                }
            if[type] == "datalog"{  
                grok {
                    match => {"message" => "|%{DATA:userName}|%{GREEDYDATA:operationName}|%{DATA:timeFormat}|%{DATA:ip}|%{DATA:systemType}|%{GREEDYDATA:logType}|%{GREEDYDATA:method}|%{GREEDYDATA:input}"}
                }   
            }
            ruby {
                code => "event['time'] = event['@timestamp']"
            }
            mutate
            {
                add_field => ["time", "%{@timestamp}"]
            }                
}
output{
       
                if[type] == "datalog"{
                    elasticsearch{
                        hosts=>["192.168.xx.xx:9200"]
                        user => "elastic"
                        password => "xxxx"
                        index => "xxxx-%{+YYYY.MM.dd}"
                            }
                        }
                if[type] == "loginlog"{
                    elasticsearch{
                        hosts=>["192.168.xx.xx:9200"]
                        user => "elastic"
                        password => "xxxx"
                        index => "xxxx-%{+YYYY.MM.dd}"
                            }                   
                        }
}      

5.啟動

docker run -p 5044:5044 -p 5000:5000-p 4999:4999--name=logstash \
  --restart=always --privileged=true\
  -e ES_JAVA_OPTS="-Xms1g -Xmx2g" \
  -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
  -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d \
  -v /usr/local/logstash/logs:/usr/share/logstash/logs  \
  -d logstash:7.5.1      

參數詳解：

• -p 5044:5044 -p 5000:5000-p 4999:4999 ：映射的端口号 這裡與上文log_to_es.conf input中一定要相同！！！！額外價格一個5044 為logstash位址
• --name=logstash：容器名稱
• --restart=always --privileged=true：啟動配置
• -e ES_JAVA_OPTS="-Xms1g -Xmx2g"：指定記憶體
• -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml：配置檔案挂載
• -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d：日志收集配置挂載位置
• -v /usr/local/logstash/logs:/usr/share/logstash/logs：日志挂載位置
• -d logstash:7.5.1：指定鏡像

6.驗證

通路kibana可以看到是否連接配接成功。