ECK安装elasticsearch集群及es配置x-pack

ECK安装elasticsearch集群

elasticsearch安装x-pack插件

Elasticsearch Springboot 开发

ECK安装elasticsearch集群

ECK为elasticsearch官方提供的在k8s集群上安装es集群的方式
因为es集群为有状态，ECK进行了很好的封装
生产环境面临大的集群状态的话，尝试通过k8s进行es集群的维护会是以后的发展趋势

ECK安装的前提条件

k8s集群
配置好pv，可以采用nfs和cephFS，pvc由ECK进行维护
配置好镜像仓库 harbor
将es的镜像从远程服务器下载下来，这样本地拉的时候会快很多

k8s集群部署NFS

k8s集群部署ECK

下载ECK定义的CRD资源和RCAB规则 kubectl apply -f https://download.elastic.co/downloads/eck/1.4.0/all-in-one.yaml
查看日志 kubectl -n elastic-system logs -f statefulset.apps/elastic-operator
创建单个节点的es operator自动创建节点

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.11.1
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

(设置node.store.allow_mmap: false对性能有影响，应针对虚拟虚拟机一节中所述的生产工作负载进行调整。)

创建成功之后监控节点的健康状态 kubectl get elasticsearch

NAME          HEALTH    NODES     VERSION   PHASE         AGE
quickstart    green     1         7.11.1     Ready         1m

查看pod kubectl get pods --selector='elasticsearch.k8s.elastic.co/cluster-name=quickstart'
访问es，ECK会给es创建cluster IP，ECK自动集成了X-PACK插件，分别通过 kubectl get svc quickstart-es-http 和 PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}') 获取IP和密码，默认端口9200 curl -u "elastic:$PASSWORD" -k "https://quickstart-es-http:9200"
kubectl port-forward service/quickstart-es-http 9200 curl -u "elastic:$PASSWORD" -k "https://localhost:9200" 得到结果

"name" : "quickstart1-es-default-0",
  "cluster_name" : "quickstart1",
  "cluster_uuid" : "cqEk6G9dQHKWL4MidTQYAw",
  "version" : {
    "number" : "7.11.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

elasticsearch安装x-pack插件

es6.8以上的版本默认集成了x-pack，但是需要配置开启

1.es证书生成

将证书文件生成到指定的目录下，这里指定为/etc/elasticsearch

cd $ES_HOME/bin
./elasticsearch-certutil ca -out /etc/elasticsearch/elastic-certificates.p12 -pass

./elasticsearch-certutil ca -out elastic-certificates.p12 -pass

此处操作为，在一台机器上生成，然后分发到集群中的其他机器上，文件要保持一致

2.配置elasticsearch.yml文件

集群中每台机器的配置文件都要添加配置

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

3.重启es集群

重启es要用普通用户,加上

-d

命令在后台运行

su elastic
bin/elasticsearch -d

4.生成密码

执行下面的命令，会让输入各种密码

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

此处输入的账号和密码是elastic:OY3iPO2AK&gSz35D

6.验证

curl localhost:9200/_cat/indices?v -u "elastic:OY3iPO2AK&gSz35D"

可以查看到索引，即密码验证成功

6.坑

1.Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

原因是集群中elasticsearch证书不一致

2.ERROR: bootstrap checks failed memory locking requested for elasticsearch process but memory is not locked

在/etc/security/limits.conf中添加配置
* soft memlock unlimited 
* hard memlock unlimited

Elasticsearch Springboot 开发

1.配置文件

application.properties

spring.data.elasticsearch.repositories.enabled=true
spring.elasticsearch.rest.uris=http://192.168.223.147:9200
spring.elasticsearch.rest.connection-timeout=30s

2.编写实体类entity

@Document

@Id

@Field

3.编写repository

继承ElasticsearchRepository

4.定义接口

Service

5.写接口实现方法

6.编写Controller

ECK安装elasticsearch集群及es配置x-pack