天天看點
傳回

ES資料的操作（二）kibanakibana的安裝和使用：ES查詢的兩種方式六、ES的bool查詢 (must、should)七、ES之查詢結果過濾八、ES之精确查詢與模糊查詢九、ES的聚合查詢avg、max、min、sum十、ES的分組查詢

kibana

是一一個針對Elasticsearch的開源分析及可視化平台,用來搜尋、檢視互動存儲在Elasticsearch索引中的資料。 使用Kibana ,可以通過各種圖表進行進階資料分析及展示。Kibana讓海量資料更容易了解。它操作簡單,基于浏覽器的使用者界面可以快速建立儀表闆( dashboard )實時顯示Elasticsearch查詢動态。設定Kibana非常簡單。 無需編碼或者額外的基礎架構,幾分鐘内就可以完成Kibana安裝并啟動Elasticsearch索引監測。

kibana的安裝和使用：

https://blog.csdn.net/qq_18769269/article/details/80843810

ES查詢的兩種方式

基礎知識

指令

指令 url 解釋
put /索引名稱/類型名稱/文檔ID 建立文檔（指定文檔ID）
POST /索引名稱/索引類型 建立文檔（随機文檔ID）
POST /索引名稱/類型名稱/文檔id/_update 修改文檔
POST /索引名稱/類型名稱/_search 查詢資料
DELETE /索引名稱/類型名稱/文檔id 删除文檔/或者索引
GET /索引名稱/類型名稱/文檔id 查詢文檔通過文檔ID

字段類型

類型 對應類型 說明
字元串 text keyword text自動分詞，keyword全文比對
整型 byte short integer long
浮點型 float double half_float scaled_float
日期 date
布爾 boolean
二進制 binary
範圍 range
數組 array
對象 object
嵌套 nested
ip ip (IPv4 和 IPv6 位址)

1、查詢字元串搜尋

GET alias_productbatches/_search?q=id:1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3


{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 10.914281,
    "hits" : [
      {
        "_index" : "productbatches_v2",
        "_type" : "_doc",
        "_id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
        "_score" : 10.914281,
        "_source" : {
          "id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
          "num" : "202000004107210719150940932",
          "product_id" : "56e9dab5-8ff8-410a-8e2d-995a98eb54b1",
          "store_id" : "1471d41e-c30c-4ffb-90ba-98febbfae394",
          "store_product_id" : "5eaf6b02-c727-4bf7-b9af-15f11421f900",
          }
      }
    ]
  }
}

2、結構化查詢(單字段查詢,不能多字段組合查詢)

GET alias_productbatches/_search
{
  "query":{
    "match":{
      "id":"1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3"
    }
  }
}

3、match系列之操作 

造數：

PUT test1/doc/1
{
  "title": "中國是世界上人口最多的國家",
   "desc": "china is the most people in the world"
}
PUT test1/doc/2
{
  "title": "美國是世界上軍事實力最強大的國家",
   "desc": "The United States is the most powerful military country in the world"
}
PUT test1/doc/3
{
  "title": "北京是中國的首都",
  "desc": "Beijing is the capital of China"
}
           
match: 查詢比對key的values值

GET test1/doc/_search
{
  "query":{
    "match":{
      "title":"中國"
    }
  }
}

結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中國是世界上人口最多的國家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中國的首都"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美國是世界上軍事實力最強大的國家"
        }
      }
    ]
  }
}



比對id=1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3

match查詢中文時，會把中文拆分後進行比對查詢，如需要則使用短語查詢match_phrase

GET test1/doc/_search
{
  "query":{
    "match_phrase":{
      "title":"中國"
    }
  }
}

結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 0.5753642,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "中國是世界上人口最多的國家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中國的首都"
        }
      }
    ]
  }
}

slop:相當于正則中的中國.*?世界。這個間隔預設為0，2代表中國與世界中間數字在兩個及以下
GET test1/doc/_search
{
  "query":{
    "match_phrase": {
      "title": {
        "query": "中國世界",
        "slop":2
      }
    }
  }
}

結果：
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.7445889,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.7445889,
        "_source" : {
          "title" : "中國是世界上人口最多的國家"
        }
      }
    ]
  }
}


match_phrase_prefix（最左字首查詢）智能搜尋--以什麼開頭（主要是英文）
GET test1/doc/_search
{
  "query":{
    "match_phrase_prefix": {
      "desc": "china is"
    }
  }
}

結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.5753642,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most people in the world"
        }
      }
    ]
  }
}

max_expansions 參數了解 字首查詢會非常的影響性能，要對結果集進行限制，就加上這個參數。

GET test1/doc/_search
{
  "query": {
    "match_phrase_prefix": {
      "desc": {
        "query": "bea",
        "max_expansions":1
      }
    }
  }
}

match系列之multi_match（多字段查詢) 

multi_match是要在多個字段中查詢同一個關鍵字 除此之外，mulit_match甚至可以當做match_phrase和match_phrase_prefix使用，隻需要指定type類型即可

GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "中國",
      "fields": ["title"]
    }
  }
}

同上第一個match查詢

當設定屬性 type:phrase 時 等同于 短語查詢
GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "中國",
      "fields": ["title"],
      "type": "phrase"
    }
  }
}


當設定屬性 type:phrase_prefix時 等同于 最左字首查詢
GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "china is",
      "fields": ["desc"],
      "type": "phrase_prefix"
    }
  }
}

四、ES的排序查詢

sort：對字段進行排序

GET test1/doc/_search
GET test1/doc/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "title.keyword": {
        "order": "asc"
      }
    }
  ]
}

為什麼不能直接title呢？
檢視mapping：
{
  "test1" : {
    "mappings" : {
      "doc" : {
        "properties" : {
          "desc" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "title" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

對text進行排序需要對字段索引兩次，一次索引分詞（用于搜尋）一次索引不分詞（用于排序）

五、ES的分頁查詢

from：從哪開始查 size：傳回幾條結果

GET test1/doc/_search
{
  "query": {
    "match": {
      "title": "中國"
    }
  },
  "from": 0,
  "size": 2
}

結果：

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most people in the world"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中國的首都",
          "desc" : "Beijing is the capital of China"
        }
      }
    ]
  }
}

六、ES的bool查詢 (must、should)

must (must字段對應的是個清單，也就是說可以有多個并列的查詢條件，一個文檔滿足各個子條件後才最終傳回)

GET test1/doc/_search
{
    "query": {
        "bool": {
            "must": [
                {
                    "match": {
                        "title": "中國"
                    }
                }
            ]
        }
    }
}

結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most people in the world"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中國的首都",
          "desc" : "Beijing is the capital of China"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美國是世界上軍事實力最強大的國家",
          "desc" : "The United States is the most powerful military country in the world"
        }
      }
    ]
  }
}


多個條件：
GET test1/doc/_search
{
    "query": {
        "bool": {
            "must": [
                {
                    "match": {
                        "title": "中國"
                    }
                },
               {
                    "match_phrase_prefix": {
                        "desc": "china is"
                    }
                }
            ]
        }
    }
}

結果：
{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.258609,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.258609,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most people in the world"
        }
      }
    ]
  }
}

should (隻要符合其中一個條件就傳回)

GET test1/doc/_search
{
    "query": {
        "should": {
            "must": [
                {
                    "match": {
                        "title": "中國"
                    }
                },
               {
                    "match_phrase_prefix": {
                        "desc": "mytest"
                    }
                }
            ]
        }
    }
}

結果和查詢條件滿足中國的結果一樣

filter：滿足過濾條件 

filter(條件過濾查詢，過濾條件的範圍用range表示gt表示大于、lt表示小于、gte表示大于等于、lte表示小于等于)

GET test1/_search
{
      "query": {
        "bool": {
          "must": [
            {
              "range": {
                  "time_create": {
                      "gte": "1626667200000",
                      "lte": "1626710400000"
                  }
              }
            }
          ]
        }
      }
    }

boost：提權，控制每個查詢子句的相對權重，該值預設為1。一個大于1的boost會增加該查詢子句的相對權重

可以參考：https://www.jianshu.com/p/98888942e737

minimum_should_match：最小比對度

參考：https://blog.csdn.net/xiao_jun_0820/article/details/51095521

must_not：不等于

ES資料的操作（二）kibanakibana的安裝和使用：ES查詢的兩種方式六、ES的bool查詢 (must、should)七、ES之查詢結果過濾八、ES之精确查詢與模糊查詢九、ES的聚合查詢avg、max、min、sum十、ES的分組查詢

七、ES之查詢結果過濾

_source

GET test1/doc/_search
{
  "query": {
    "match": {
      "title": "中國"
    }
  },
  "_source": ["title"]
}

結果：
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中國是世界上人口最多的國家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中國的首都"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美國是世界上軍事實力最強大的國家"
        }
      }
    ]
  }
}


=======
次數最終的結果沒有展示desc字段

八、ES之精确查詢與模糊查詢

term:查詢查找包含文檔精确的反向索引指定的詞條。也就是精确查找。

term和match的差別是：match是經過analyer的，也就是說，文檔首先被分析器給處理了。根據不同的分析器，分析的結果也稍顯不同，然後再根據分詞結果進行比對。term則不經過分詞，它是直接去反向索引中查找了精确的值了。

GET test1/doc/_search
{
  "query": {
    "term": {
      "title": "中國"
    }
  },
  "_source": ["title"]
}
===如果這樣查詢，就無法查出結果


GET test1/doc/_search
{
  "query": {
    "term": {
      "title.keyword": "中國是世界上人口最多的國家"
    }
  }
}

======
這樣查詢title為“中國是世界上人口最多的國家”的資料,
為什麼要加keyword呢，因為索引的mapping title是text:
"keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }

多個term查詢

查詢方式一：
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "title.keyword": {
            "value": "中國是世界上人口最多的國家"
          }
        }
        },
        {"term": {
          "desc.keyword": {
            "value": "china is the most people in the world"
          }
        }}
      ]
      
    }
    
  }
}


方式二：
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {"terms": {
          "title.keyword": [
            "中國是世界上人口最多的國家",
            "北京是中國的首都"
          ]
        }}
      ]
    }
  }
}

九、ES的聚合查詢avg、max、min、sum

聚合函數的使用，一定是先查出結果，然後對結果使用聚合函數做處理

avg：求平均

max：最大值

min：最小值

sum：求和

GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs": {
      "avg": {
        "field": "score"
      }
    }
  }
}

=====
查詢結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "title" : "美國是世界上軍事實力最強大的國家",
          "desc" : "The United States is the most powerful military country in the world",
          "score" : 2.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most populous country in the world",
          "score" : 9.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "title" : "北京是中國的首都",
          "desc" : "Beijing is the capital of China",
          "score" : 5.5
        }
      }
    ]
  },
  "aggregations" : {
    "my_aggs" : {
      "value" : 5.833333333333333
    }
  }
}

=========
query:先查詢結果
aggs：對查詢出來的結果進行聚合，my_aggs是聚合的别名 avg是聚合類型，field的值是聚合的字段
其他聚合方式一樣
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs": {
      "max": {
        "field": "score"
      }
    }
  }
}

十、ES的分組查詢

在aggs的中，使用range來做分組，field是以age為分組，分組使用ranges來做，from和to是範圍

GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs_group": {
      "range": {
        "field": "score",
        "ranges": [
          {"from": 1.0,
          "to": 5.0
          },
          {"from": 5.0,
            "to": 10.0
          }
        ]
      },
      "aggs": {
        "my_agg": {
          "avg": {
            "field": "score"
          }
        }
      }
    }
  }
}

==結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "title" : "美國是世界上軍事實力最強大的國家",
          "desc" : "The United States is the most powerful military country in the world",
          "score" : 2.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "title" : "中國是世界上人口最多的國家",
          "desc" : "china is the most populous country in the world",
          "score" : 9.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "title" : "北京是中國的首都",
          "desc" : "Beijing is the capital of China",
          "score" : 5.5
        }
      }
    ]
  },
  "aggregations" : {
    "my_aggs_group" : {
      "buckets" : [
        {
          "key" : "1.0-5.0",
          "from" : 1.0,
          "to" : 5.0,
          "doc_count" : 1,
          "my_agg" : {
            "value" : 2.5
          }
        },
        {
          "key" : "5.0-10.0",
          "from" : 5.0,
          "to" : 10.0,
          "doc_count" : 2,
          "my_agg" : {
            "value" : 7.5
          }
        }
      ]
    }
  }
}
大資料測試 elasticsearch
上一篇: Python資料結構與算法之希爾排序
下一篇: MongoDB分片式安裝部署 OpenEuler/Centos

繼續閱讀