Elasticsearch 監控 Elasticsearch 叢集

文章目錄

• ​​1. Elasticsearch Stats 相關的 API​​
• ​​2. Elasticsearch Task API​​
• ​​3. The Index & Query Slow Log​​
• ​​4. 如何建立監控 Dashboard​​
• ​​5. demo​​

1. Elasticsearch Stats 相關的 API

Elasticsearch 提供了多個監控相關的 API

• Node Stats： _nodes/stats
• Cluster Stats: _cluster/stats
• Index Stats: index_name/_stats

2. Elasticsearch Task API

檢視 Task 相關的 API

• Pending Cluster Tasks API: GET _cluster/pending_tasks
• Task Management API :GET _tasks (可以用來 Cancel 一個 Task)

• GET _nodes/thread_pool
• GET _nodes/stats/thread_pool
• GET _cat/thread_pool?v
• GET _nodes/hot_threads

3. The Index & Query Slow Log

• 支援将分片上，​

  ​Search​

  ​ 和 ​

  ​Fetch​

  ​ 階段的慢 查詢寫入檔案
• 支援為​

  ​Query​

  ​ 和 ​

  ​Fetch​

  ​ 分别定義門檻值
• 索引級的動态設定，可以按需設定，或者通 過​

  ​Index Template​

  ​ 統一設定

• Slog log 檔案通過

  log4j2.properties 配置

PUT my_index/
{
  "settings": {
    "index.search.slowlog.threshold": {
      "query.warn": "10s",
      "query.info": "3s",
      "query.debug": "2s",
      "query.trace": "0s",
      "fetch.warn": "1s",
      "fetch.info": "600ms",
      "fetch.debug": "400ms",
      "fetch.trace": "0s"
    }
  }
}      

4. 如何建立監控 Dashboard

• 開發​

  ​Elasticsearch plugin​

  ​，通過讀取相關的監控 API，将資料發送到 ES，或者 TSDB
• 使用 Metricbeats 搜集相關名額
• 使用 Kibana 或 Grafana 建立 Dashboard
• 可以開發​

  ​Elasticsearch Exporter​

  ​​，通過​

  ​Prometheus​

  ​ 監控 Elasticsearch 叢集

5. demo

# Node Stats：
GET _nodes/stats

#Cluster Stats:
GET _cluster/stats

#Index Stats:
GET kibana_sample_data_ecommerce/_stats

#Pending Cluster Tasks API:
GET _cluster/pending_tasks

# 檢視所有的 tasks，也支援 cancel task
GET _tasks


GET _nodes/thread_pool
GET _nodes/stats/thread_pool
GET _cat/thread_pool?v
GET _nodes/hot_threads
GET _nodes/stats/thread_pool


# 設定 Index Slowlogs
# the first 1000 characters of the doc's source will be logged
PUT my_index/_settings
{
  "index.indexing.slowlog":{
    "threshold.index":{
      "warn":"10s",
      "info": "4s",
      "debug":"2s",
      "trace":"0s"
    },
    "level":"trace",
    "source":1000  
  }
}

# 設定查詢
DELETE my_index
//"0" logs all queries
PUT my_index/
{
  "settings": {
    "index.search.slowlog.threshold": {
      "query.warn": "10s",
      "query.info": "3s",
      "query.debug": "2s",
      "query.trace": "0s",
      "fetch.warn": "1s",
      "fetch.info": "600ms",
      "fetch.debug": "400ms",
      "fetch.trace": "0s"
    }
  }
}

GET my_index