mtail統計nginx通路日志

mtail

是一個可以從應用程式日志中提取名額，并将其導出到時間序列資料庫或時間序列電腦中，以便配置警報和儀表盤的工具。

它的目的在于填補沒有導出自己内部狀态的應用程式和已有監控系統之間的空白，由于有自定義抽取粘合代碼，無修改應用程式或重寫相同的架構。

提取由定義了模式和動作的mtail程式來控制。

安裝方法

預編譯的

二進制檔案

推薦使用最新的生産版本二進制檔案安裝mtail。可按需選擇對應Windows，OSX和Linux等系統版本。

源碼編譯

需要安裝運作版本不低于1.9的Go環境。如果不想在本地環境中安裝Go，也可以選擇Dockerfile。

下載下傳編譯好的二進制檔案後運作即可

# nohup ./mtail -progs /path/to/mtailprograms -logs /path/to/logfile &
# cat /path/to/mtailprograms
counter nginx_line_count
#正則統計通路日志的行數，可以在grafana中配置視圖觀察通路量趨勢
/$/ {
  nginx_line_count++
}
# ./mtail -h
mtail version v3.0.0-rc12 git revision 936050df9856da258f13e5df71a5a0c7f8f6acdc go version go1.10.1

Usage:
  -address string
        Host or IP address on which to bind HTTP listener
  -alsologtostderr
        log to standard error as well as files
  -block_profile_rate int
        Nanoseconds of block time before goroutine blocking events reported. 0 turns off.  See https://golang.org/pkg/runtime/#SetBlockProfileRate
  -collectd_prefix string
        Prefix to use for collectd metrics.
  -collectd_socketpath string
        Path to collectd unixsock to write metrics to.
  -compile_only
        Compile programs only, do not load the virtual machine.
  -dump_ast
        Dump AST of programs after parse (to INFO log).
  -dump_ast_types
        Dump AST of programs with type annotation after typecheck (to INFO log).
  -dump_bytecode
        Dump bytecode of programs (to INFO log).
  -emit_prog_label
        Emit the 'prog' label in variable exports. (default true)
  -graphite_host_port string
        Host:port to graphite carbon server to write metrics to.
  -graphite_prefix string
        Prefix to use for graphite metrics.
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace
  -log_dir string
        If non-empty, write log files in this directory
  -logfds value
        List of file descriptor numbers to monitor, separated by commas.  This flag may be specified multiple times.
  -logs value
        List of log files to monitor, separated by commas.  This flag may be specified multiple times.
  -logtostderr
        log to standard error instead of files
  -metric_push_interval_seconds int
        Interval between metric pushes, in seconds. (default 60)
  -metric_push_write_deadline duration
        Time to wait for a push to succeed before exiting with an error. (default 10s)
  -mtailDebug int
        Set parser debug level.
  -mutex_profile_fraction int
        Fraction of mutex contention events reported.  0 turns off.  See http://golang.org/pkg/runtime/#SetMutexProfileFraction
  -one_shot
        Compile the programs, then read the contents of the provided logs from start until EOF, print the values of the metrics store and exit. This is a debugging flag only, not for production use.
  -one_shot_metrics
        DEPRECATED: Dump metrics (to stdout) after one shot mode.
  -override_timezone string
        If set, use the provided timezone in timestamp conversion, instead of UTC.
  -port string
        HTTP port to listen on. (default "3903")
  -progs string
        Name of the directory containing mtail programs
  -statsd_hostport string
        Host:port to statsd server to write metrics to.
  -statsd_prefix string
        Prefix to use for statsd metrics.
  -stderrthreshold value
        logs at or above this threshold go to stderr
  -syslog_use_current_year
        Patch yearless timestamps with the present year. (default true)
  -v value
        log level for V logs
  -version
        Print mtail version information.
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging