天天看點
傳回

Spring Boot整合ElasticSearch單/多叢集案例

Spring Boot整合ElasticSearch單個叢集和多個叢集案例分享，本文涉及内容：

  • 導入spring boot elasticsearch starter
  • 單個es叢集案例
  • 多個es叢集案例

本文内容适合于:

  • spring boot 1.x,2.x
  • elasticsearch 1.x,2.x,5.x,6.x,+

1.導入spring boot elasticsearch starter

在spring boot項目中導入spring boot elasticsearch starter

maven工程

<dependency>
            <groupId>com.bbossgroups.plugins</groupId>
            <artifactId>bboss-elasticsearch-spring-boot-starter</artifactId>
            <version>5.0.8.6</version>
        </dependency>

gradle工程

compile "com.bbossgroups.plugins:bboss-elasticsearch-spring-boot-starter:5.0.8.6"

2.建立spring boot啟動類

建立Application類：

package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;

/**
 *  @author yinbp [[email protected]]
 *  
 */

@SpringBootApplication

public class Application {

    private Logger logger = LoggerFactory.getLogger(Application.class);

   
    public static void main(String[] args) {

        SpringApplication.run(Application.class, args);
    }
}

Application類中定義了一個main方法用來啟動spring boot elasticsearch測試應用。

Application類将被用于啟動下面兩個測試用例：

  • 單es叢集測試用例
  • 多es叢集測試用例

定義elasticsearch rest client元件實列管理類ServiceApiUtil：

package org.bboss.elasticsearchtest.springboot;
/*
 *  Copyright 2008 biaoping.yin
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 */

import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.springframework.stereotype.Service;

/**
 * 管理es rest client元件執行個體
 */
@Service
public class ServiceApiUtil {


   /**
    * 擷取操作預設的es叢集的用戶端工具元件
    * @return
    */
   public ClientInterface restClient(){
      return  ElasticSearchHelper.getRestClientUtil();
   }

   /**
    * 擷取操作預設的es叢集的加載dsl配置檔案的用戶端工具元件
    * @return
    */
   public ClientInterface restDemoConfigClient(){
      return  ElasticSearchHelper.getConfigRestClientUtil("esmapper/demo.xml");
   }

   /**
    * 擷取操作logs的es叢集的用戶端工具元件
    * @return
    */
   public ClientInterface restClientLogs(){
      return  ElasticSearchHelper.getRestClientUtil("logs");
   }
   /**
    * 擷取操作logs的es叢集的加載dsl配置檔案的用戶端工具元件
    * @return
    */
   public ClientInterface restConfigClientLogs(){
      return  ElasticSearchHelper.getConfigRestClientUtil("logs","esmapper/demo.xml");
   }
}

實列管理類ServiceApiUtil将被注入到後續的測試用例中。

3.單es叢集配置和使用

3.1 配置單es叢集

修改spring boot配置檔案application.properties内容（yml格式配置檔案參考下面的内容配置）：

##ES叢集配置
spring.elasticsearch.bboss.elasticUser=elastic
spring.elasticsearch.bboss.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
#elasticsearch.rest.hostNames=127.0.0.1:9200
#elasticsearch.rest.hostNames=10.21.20.168:9200
spring.elasticsearch.bboss.elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.elasticsearch.ttl=2d
#在控制台輸出腳本調試開關showTemplate,false關閉，true打開，同時log4j至少是info級别
spring.elasticsearch.bboss.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.elasticsearch.discoverHost=false
# dsl配置檔案熱加載掃描時間間隔，毫秒為機關，預設5秒掃描一次，<= 0時關閉掃描機制
spring.elasticsearch.bboss.dslfile.refreshInterval = -1

##es client http連接配接池配置
spring.elasticsearch.bboss.http.timeoutConnection = 400000
spring.elasticsearch.bboss.http.timeoutSocket = 400000
spring.elasticsearch.bboss.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.http.retryTime = 1
spring.elasticsearch.bboss.http.maxLineLength = -1
spring.elasticsearch.bboss.http.maxHeaderCount = 200
spring.elasticsearch.bboss.http.maxTotal = 400
spring.elasticsearch.bboss.http.defaultMaxPerRoute = 200
spring.elasticsearch.bboss.http.soReuseAddress = false
spring.elasticsearch.bboss.http.soKeepAlive = false
spring.elasticsearch.bboss.http.timeToLive = 3600000
spring.elasticsearch.bboss.http.keepAlive = 3600000
spring.elasticsearch.bboss.http.keystore =
spring.elasticsearch.bboss.http.keyPassword =
# ssl 主機名稱校驗，是否采用default配置，
# 如果指定為default，就采用DefaultHostnameVerifier,否則采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.http.hostnameVerifier =

## 資料庫資料源配置，使用db-es資料導入功能時需要配置
#spring.elasticsearch.bboss.db.name = test
#spring.elasticsearch.bboss.db.user = root
#spring.elasticsearch.bboss.db.password = 123456
#spring.elasticsearch.bboss.db.driver = com.mysql.jdbc.Driver
#spring.elasticsearch.bboss.db.url = jdbc:mysql://localhost:3306/bboss
#spring.elasticsearch.bboss.db.usePool = false
#spring.elasticsearch.bboss.db.validateSQL = select 1

單ES叢集配置項都是以spring.elasticsearch.bboss開頭。

3.2 單叢集測試用例

編寫es單叢集測試用例BBossESStarterTestCase

/*
 * Copyright 1999-2018 Alibaba Group Holding Ltd.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.client.ClientInterface;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

/**
 * 單叢集示範功能測試用例，spring boot配置項以spring.elasticsearch.bboss開頭
 * 對應的配置檔案為application.properties檔案
 * @author  yinbp [[email protected]]
 */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class BBossESStarterTestCase {
	 
    @Test
    public void testBbossESStarter() throws Exception {
//        System.out.println(bbossESStarter);

		//驗證環境,擷取es狀态
		String response = serviceApiUtil.restClient().executeHttp("_cluster/state?pretty",ClientInterface.HTTP_GET);

		System.out.println(response);
		//判斷索引類型是否存在，false表示不存在，正常傳回true表示存在
		boolean exist = serviceApiUtil.restClient().existIndiceType("twitter","tweet");

		//判讀索引是否存在，false表示不存在，正常傳回true表示存在
		exist = serviceApiUtil.restClient().existIndice("twitter");

		exist = serviceApiUtil.restClient().existIndice("agentinfo");

    }


}

直接通過junit運作上述測試用例即可

4.多ES叢集測試用例

4.1 配置多es叢集

修改spring boot配置檔案application-multi-datasource.properties，内容如下：

##多叢集配置樣例，如果需要做多叢集配置，請将參照本文内容修改application.properties檔案内容
spring.elasticsearch.bboss.default.name = default
##default叢集配配置
spring.elasticsearch.bboss.default.elasticUser=elastic
spring.elasticsearch.bboss.default.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
#elasticsearch.rest.hostNames=127.0.0.1:9200
spring.elasticsearch.bboss.default.elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.default.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.default.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.default.elasticsearch.ttl=2d
#在控制台輸出腳本調試開關showTemplate,false關閉，true打開，同時log4j至少是info級别
spring.elasticsearch.bboss.default.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.default.elasticsearch.discoverHost=false

##default連接配接池配置
spring.elasticsearch.bboss.default.http.timeoutConnection = 400000
spring.elasticsearch.bboss.default.http.timeoutSocket = 400000
spring.elasticsearch.bboss.default.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.default.http.retryTime = 1
spring.elasticsearch.bboss.default.http.maxLineLength = -1
spring.elasticsearch.bboss.default.http.maxHeaderCount = 200
spring.elasticsearch.bboss.default.http.maxTotal = 400
spring.elasticsearch.bboss.default.http.defaultMaxPerRoute = 200
spring.elasticsearch.bboss.default.http.keystore =
spring.elasticsearch.bboss.default.http.keyPassword =
# ssl 主機名稱校驗，是否采用default配置，
# 如果指定為default，就采用DefaultHostnameVerifier,否則采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.default.http.hostnameVerifier =

##logs叢集配置
spring.elasticsearch.bboss.logs.name = logs
spring.elasticsearch.bboss.logs.elasticUser=elastic
spring.elasticsearch.bboss.logs.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
spring.elasticsearch.bboss.logs.elasticsearch.rest.hostNames=127.0.0.1:9200
#elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.logs.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.logs.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.logs.elasticsearch.ttl=2d
#在控制台輸出腳本調試開關showTemplate,false關閉，true打開，同時log4j至少是info級别
spring.elasticsearch.bboss.logs.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.logs.elasticsearch.discoverHost=false

##logs叢集對應的連接配接池配置
spring.elasticsearch.bboss.logs.http.timeoutConnection = 400000
spring.elasticsearch.bboss.logs.http.timeoutSocket = 400000
spring.elasticsearch.bboss.logs.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.logs.http.retryTime = 1
spring.elasticsearch.bboss.logs.http.maxLineLength = -1
spring.elasticsearch.bboss.logs.http.maxHeaderCount = 200
spring.elasticsearch.bboss.logs.http.maxTotal = 400
spring.elasticsearch.bboss.logs.http.defaultMaxPerRoute = 200
# https證書配置
spring.elasticsearch.bboss.logs.http.keystore =
spring.elasticsearch.bboss.logs.http.keyPassword =
# ssl 主機名稱校驗，是否采用default配置，
# 如果指定為default，就采用DefaultHostnameVerifier,否則采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.logs.http.hostnameVerifier =
# dsl配置檔案熱加載掃描時間間隔，毫秒為機關，預設5秒掃描一次，<= 0時關閉掃描機制
spring.elasticsearch.bboss.dslfile.refreshInterval = -1

配置說明：

上面配置了兩個叢集：default和logs

每個叢集配置項的字首為：spring.elasticsearch.bboss.叢集名字，其中的叢集名字是一個自定義的邏輯名稱，用來在client api中引用叢集。

default叢集的配置項字首為：

spring.elasticsearch.bboss.default

logs叢集的配置項字首為：

spring.elasticsearch.bboss.logs

同時每個叢集的配置項目裡面必須包含name項目的配置

default叢集name配置：

spring.elasticsearch.bboss.default.name = default

logs叢集name配置：

##logs叢集配置

spring.elasticsearch.bboss.logs.name = logs

4.2 定義加載多es叢集配置的spring boot Configuration類

建立類MultiESSTartConfigurer

package org.bboss.elasticsearchtest.springboot;
/*
 *  Copyright 2008 biaoping.yin
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 */

import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.boot.BBossESStarter;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;

/**
 * 配置多個es叢集
 * 指定多es資料源profile：multi-datasource
 */
@Configuration
@Profile("multi-datasource") 
public class MultiESSTartConfigurer {
	@Primary
	@Bean(initMethod = "start")
	@ConfigurationProperties("spring.elasticsearch.bboss.default")
	public BBossESStarter bbossESStarterDefault(){
		return new BBossESStarter();

	}

	@Bean(initMethod = "start")
	@ConfigurationProperties("spring.elasticsearch.bboss.logs")
	public BBossESStarter bbossESStarterLogs(){
		return new BBossESStarter();

	}

}

說明：

MultiESSTartConfigurer通過以下兩個方法分别加載default和logs兩個es叢集的配置

default叢集配置加載

@Primary
@Bean(initMethod = "start")
@ConfigurationProperties("spring.elasticsearch.bboss.default")
public BBossESStarter bbossESStarterDefault()
logs叢集配置加載

@Bean(initMethod = "start")
@ConfigurationProperties("spring.elasticsearch.bboss.logs")
public BBossESStarter bbossESStarterLogs()

4.3 定義多es叢集測試用例

多es叢集測試用例MultiBBossESStartersTestCase

package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.client.ClientInterface;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.junit4.SpringRunner;

/**
 * 多叢集示範功能測試用例，spring boot配置項以spring.elasticsearch.bboss.叢集名稱開頭，例如：
 * spring.elasticsearch.bboss.default 預設es叢集
 * spring.elasticsearch.bboss.logs  logs es叢集
 * 兩個叢集通過 org.bboss.elasticsearchtest.springboot.MultiESSTartConfigurer加載
 * 對應的配置檔案為application-multi-datasource.properties檔案
 * 通過ActiveProfiles指定并激活多es叢集配置：multi-datasource 
 * @author yinbp [[email protected]]
 */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
@ActiveProfiles("multi-datasource")
public class MultiBBossESStartersTestCase {

    @Test
    public void testMultiBBossESStarters() throws Exception {

		//驗證環境,擷取es狀态
		String response = serviceApiUtil.restClient().executeHttp("_cluster/state?pretty",ClientInterface.HTTP_GET);
		System.out.println(response);


		//判斷索引類型是否存在，false表示不存在，正常傳回true表示存在
		boolean exist = serviceApiUtil.restClientLogs().existIndiceType("twitter","tweet");
		System.out.println("twitter/tweet:"+exist);
		//判讀索引是否存在，false表示不存在，正常傳回true表示存在
		exist = serviceApiUtil.restClientLogs().existIndice("twitter");
		System.out.println("twitter:"+exist);
		exist = serviceApiUtil.restClientLogs().existIndice("agentinfo");
		System.out.println("agentinfo:"+exist);
    }


}

直接通過junit運作上述測試用例即可。

5.完整的demo工程

https://gitee.com/bbossgroups/eshelloword-spring-boot-starter https://github.com/bbossgroups/eshelloword-spring-boot-starter

6 開發交流

elasticsearch微信公衆号：

Spring Boot整合ElasticSearch單/多叢集案例
java 測試技術 apache 索引 spring elasticsearch搜尋引擎 ElasticSearch讀書筆記 ElasticSearch前背景 圖表ElasticSearch ElasticSearch排序
上一篇: ElasticSearch DSL Script使用案例分享
下一篇: ElasticSearch用戶端注解使用介紹

繼續閱讀