場景:
需要在用戶端構造請求調解, 調用 Elastic Search 的 API 取到結果,并且能夠使用 ES 的授權機制。
方案:
一、在用戶端構造 Low Level API 。
二、Low Level API 直接請求 ES 的 HTTP位址,傳入 ES 的使用者名,密碼。
三、Low Level API 取到 JSON 格式的傳回結果後,結果反序列化成 Java 對象。這個步驟裡面的 Java 類定義,使用前面部落格裡提到的方法, 從 json 檔案中來生成代碼。
另外也需要在用戶端, 根據不同的條件構造出來 Json 格式的請求調解。
我們分4個步驟來看。
1). 構造 Low Level API Client
public class ElasticSearchRestClient {
private static final Logger logger = LoggerFactory.getLogger(ElasticSearchRestClient.class);
private RestClient restClient;
public void init(String elasticsearchAddress, String userName, String password){
try {
// Elasticsearch叢集需要basic auth驗證。
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
//通路使用者名和密碼為您建立Elasticsearch執行個體時設定的使用者名和密碼,也是Kibana控制台的登入使用者名和密碼。
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));
// 通過builder建立rest client,配置http client的HttpClientConfigCallback。
RestClientBuilder builder = RestClient.builder(new HttpHost(elasticsearchAddress, 9200))
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
});
// lower level client
restClient = builder.build();
} catch (Exception e) {
logger.error("Failed to initialize Elastic Search client, server:{}, userName:{}", elasticsearchAddress, userName, e);
}
}
}
2).指定要搜尋的索引發出搜尋請求
// low level api:
Request request = new Request("POST", "/" + indexName + "/_search");
request.setJsonEntity(queryJson );
Response response = restClient.performRequest(request);
3).取到 JSON 格式的傳回結果,結果反序列化成 Java 對象
前面的文章有介紹,如何從 Json 格式的資料定義中生産 Java 類的代碼。
我們以 ES HTTP 請求傳回的 Json 為例,ES 傳回的 Json 格式資料 跟 Kibana 控制台 Dev Tools -> Console 裡面發送請求, 看到的傳回結果格式是一樣的。 比如發送這個請求:
GET media/_search
{
"query": {"match_all": {}}
}
能看到傳回的這個結果
右邊傳回的 Json 内容儲存為檔案 elastic.response.json, 内容如下:
{
"took" : 113,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 9,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 16.761106,
"hits" : [
{
"_index" : "media",
"_type" : "doc",
"_id" : "176",
"_score" : 16.761106,
"_source" : {
"publish_state" : "Published",
"media_id" : "226bb012c46e1ef4e52b788cf3312345",
"id" : 176,
"url" : "http://www.abcd.com/20190317152813791092",
"owner_id" : 1438747659963495,
"media_info" : "",
"description" : null,
"env" : 1,
"cover_url" : null,
"gmt_create" : "2016-03-17T07:29:18.000Z",
"gmt_modified" : "2016-06-16T06:50:48.000Z",
"state" : "Normal",
"title" : "20160317152813791092",
"tags" : null,
"@version" : "1",
"@timestamp" : "2021-04-06T14:36:17.393Z"
}
}
]
}
}
然後執行指令:
jsonschema2pojo -T json --source elastic.response.json --target java
就能夠自動生成這幾個類:
ElasticResponse.java, Hit.java, Hits.java, Shards.java, Source.java
生成的 ElasticResponse 類是這樣的,可以稍做調整,或者重構适應實際的需求:
@Generated("jsonschema2pojo")
public class ElasticResponse {
@JsonProperty("took")
private Integer took;
@JsonProperty("timed_out")
private Boolean timedOut;
@JsonProperty("_shards")
private Shards shards;
@JsonProperty("hits")
private Hits hits;
@JsonProperty("took")
public Integer getTook() {
return took;
}
@JsonProperty("took")
public void setTook(Integer took) {
this.took = took;
}
@JsonProperty("timed_out")
public Boolean getTimedOut() {
return timedOut;
}
@JsonProperty("timed_out")
public void setTimedOut(Boolean timedOut) {
this.timedOut = timedOut;
}
@JsonProperty("_shards")
public Shards getShards() {
return shards;
}
//....
}
4). 構造 JSON 格式 請求條件
以上請求發起調用的完整代碼:
public ElasticResponse search(String indexName, String queryJson){
try {
if(StringUtils.isEmpty(queryJson)){
ElasticResponse response = new ElasticResponse();
JSONObject errorInfo = new JSONObject();
errorInfo.put("message", "empty json");
response.setError(errorInfo);
response.setError(errorInfo);
return response;
}
// low level api:
Request request = new Request("POST", "/" + indexName + "/_search");
request.setJsonEntity(queryJson );
Response response = restClient.performRequest(request);
String jsonResponse = EntityUtils.toString(response.getEntity());
ElasticResponse finalResponse = JSON.parseObject(jsonResponse, ElasticResponse.class);
return finalResponse;
} catch (IOException e) {
logger.error("Failed to search for index:{}", indexName, e);
}
return null;
}
那麼這裡 json 格式的請求條件 queryJson 如何構造, 比如要查詢 指定 owner_id, 且 publish_state不是删除狀态,id 小于某個限定值的資料,可以這樣構造:
MatchQueryBuilder ownerQuery = QueryBuilders.matchQuery("owner_id", ownerId);
BoolQueryBuilder finalQuery = QueryBuilders.boolQuery();
finalQuery.must(ownerQuery);
// id <= #id#
if(upperId>1){
RangeQueryBuilder lessThanUpperIdQuery = QueryBuilders.rangeQuery("id").to(upperId, true);
finalQuery.must(lessThanUpperIdQuery);
}
// publish_state !='Deleted'
MatchQueryBuilder inDeletedQuery = QueryBuilders.matchQuery("publish_state", "Deleted");
finalQuery.mustNot(inDeletedQuery);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(finalQuery);
//ORDER BY gmt_create DESC,id DESC
FieldSortBuilder createSort = SortBuilders.fieldSort("gmt_create").order(SortOrder.DESC);
searchSourceBuilder.sort(createSort);
FieldSortBuilder idSort = SortBuilders.fieldSort("id").order(SortOrder.DESC);
searchSourceBuilder.sort(idSort);
String queryJson = searchSourceBuilder.toString();
然後就能使用這個構造出來的 queryJson 來發送 ElasticSearch 的請求了。