zabbix历史数据存储到elasticsearch

Zabbix历史数据存储到ES

第1章 环境说明：

[[email protected] ~]# cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

[[email protected] ~]# systemctl status firewalld.service

●firewalld.service - firewalld - dynamic firewall daemon

   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)

   Active: inactive (dead)

     Docs: man:firewalld(1)

[[email protected] ~]# getenforce

Disabled

Zabbix版本

[[email protected] ~]# rpm -qa|grep zabbix

zabbix-release-4.0-1.el7.noarch

ES版本，官方说明支持的ES版本为5.0-6.1之间，之前安装的6.2最后会不成功

[[email protected] elasticsearch]# curl http://127.0.0.1:9200

{

  "name" : "2nrv98N",

  "cluster_name" : "elasticsearch",

  "cluster_uuid" : "oyVYItFuTIuEMgagxaOzgg",

  "version" : {

    "number" : "5.6.6",

    "build_hash" : "7d99d36",

    "build_date" : "2018-01-09T23:55:47.880Z",

    "build_snapshot" : false,

    "lucene_version" : "6.6.1"

  },

  "tagline" : "You Know, for Search"

}

第2章 zabbix服务端配置

2.1修改zabbix-server配置文件，用于将zabbix-server数据库中的数值类型和文本类型的历史数据存储到ES中

[[email protected] zabbix]# cat zabbix_server.conf

HistoryStorageURL=http://your-elasticsearch.here:9200

HistoryStorageTypes=str,log,text

2.2修改zabbix前端配置文件，用于将文本，字符，日志类型的历史数据存储到ES

中

[[email protected] zabbix]# cat web/zabbix.conf.php

<?php

// Zabbix GUI configuration file.

global $DB, $HISTORY;

$DB['TYPE']     = 'MYSQL';

$DB['SERVER']   = '127.0.0.1';

$DB['PORT']     = '3306';

$DB['DATABASE'] = 'zabbix';

$DB['USER']     = 'zabbix';

$DB['PASSWORD'] = 'admin111';

// Schema name. Used for IBM DB2 and PostgreSQL.

$DB['SCHEMA'] = '';

$ZBX_SERVER      = 'localhost';

$ZBX_SERVER_PORT = '10051';

$ZBX_SERVER_NAME = 'zabbix';

$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;

$HISTORY['url'] = 'http://your-elasticsearch.here:9200';

$HISTORY['types'] = ['log', 'text', 'str'];

2.3重启zabbix-server

systemctl restart zabbix-server.service

2.4重启httpd

systemctl stop httpd.service

第3章 ES上操作

Ø  ES上面创建映射，相当于在mysql中创建数据库的操作

3.1创建text类型的映射

curl -X PUT \

 http://your-elasticsearch.here:9200/text \

 -H 'content-type:application/json' \

 -d '{

   "settings" : {

      "index" : {

         "number_of_replicas" : 1,

         "number_of_shards" : 5

      }

   },

   "mappings" : {

      "values" : {

         "properties" : {

            "itemid" : {

               "type" : "long"

            },

            "clock" : {

               "format" : "epoch_second",

               "type" : "date"

            },

            "value" : {

               "fields" : {

                  "analyzed" : {

                     "index" : true,

                     "type" : "text",

                     "analyzer" : "standard"

                  }

               },

               "index" : false,

               "type" : "text"

            }

         }

      }

   }

}'

3.2配置历史数据存储多个基于时间的索引，创建uint模版

curl -X PUT \

 http://your-elasticsearch.here:9200/_template/uint_template \

 -H 'content-type:application/json' \

 -d '{

   "template": "uint*",

   "index_patterns": ["uint*"],

   "settings" : {

      "index" : {

         "number_of_replicas" : 1,

         "number_of_shards" : 5

      }

   },

   "mappings" : {

      "values" : {

         "properties" : {

            "itemid" : {

               "type" : "long"

            },

            "clock" : {

               "format" : "epoch_second",

               "type" : "date"

            },

            "value" : {

               "type" : "long"

            }

         }

      }

   }

}'

3.3为文本索引创建模版

curl -X PUT \

 http://your-elasticsearch.here:9200/_template/text_template \

 -H 'content-type:application/json' \

 -d '{

   "template": "text*",

   "index_patterns": ["text*"],

   "settings" : {

      "index" : {

         "number_of_replicas" : 1,

         "number_of_shards" : 5

      }

   },

   "mappings" : {

      "values" : {

         "properties" : {

            "itemid" : {

               "type" : "long"

            },

            "clock" : {

               "format" : "epoch_second",

               "type" : "date"

            },

            "value" : {

               "fields" : {

                  "analyzed" : {

                     "index" : true,

                     "type" : "text",

                     "analyzer" : "standard"

                  }

               },

               "index" : false,

               "type" : "text"

            }

         }

      }

   }

}'

3.4允许ES为自动创建的索引设置有效的映射，创建pipline定义（pipline能对数据进行多种预处理操作

curl -X PUT \

 http://your-elasticsearch.here:9200/_ingest/pipeline/uint-pipeline \

 -H 'content-type:application/json' \

 -d '{

  "description": "daily uint index naming",

  "processors": [

    {

      "date_index_name": {

        "field": "clock",

        "date_formats": ["UNIX"],

        "index_name_prefix": "uint-",

        "date_rounding": "d"

      }

    }

  ]

}'

第4章 在zabbix-web页面上查看数据获取是否正常

4.1刚修改完的话需要等下才能有数据展示

还可以到数据库中查看history开头的几张表是否还有数据写入

转载于:https://blog.51cto.com/13520772/2329274