天天看點

Spark thriftserver連elasticsearch

1.需要将elasticsearch-hadoop-2.1.0.Beta4.jar包放入/usr/local/spark/lib,下載下傳位址為:https://www.elastic.co/products/hadoop/

2.需要在/usr/local/spark/conf的hive-site.xml中進行配置

Spark thriftserver連elasticsearch

3.啟動thriftserver,并在—jars後面帶上此jar包

./start-thriftserver.sh --master local--driver-class-path /usr/local/spark/postgresql-9.4-1201.jdbc41.jar --jars/usr/local/spark/lib/elasticsearch-hadoop-2.1.0.Beta4.jar

4.建立artists表,并将它的index名稱命名為default,type名稱命名為artists

Spark thriftserver連elasticsearch

5.可以看到它的資料,通過curl通路elasticsarch的rest api:

Spark thriftserver連elasticsearch

6.可以聯合查詢此表與已存在的表

Spark thriftserver連elasticsearch

此方法完美解決elasticsearch自身的api難以多表關聯查詢的問題。

繼續閱讀