SparkSql 用Scala代碼查詢Hive裡面的指定的表

Maven依賴

<!--添加spark對hive的支援的jar包-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.11</artifactId>
            <version>2.1.1</version>
        </dependency>      

hive-site.xml配置檔案

<?xml version="1.0" ?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
        <!-- 這個庫是中繼資料所在的庫-->
    <value>jdbc:mysql://zjj101:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>username to use against metastore database</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
    <description>password to use against metastore database</description>
  </property>


</configuration>      

代碼

package com.hive

import org.apache.spark.sql.SparkSession


object HiveRead {
  def main(args: Array[String]): Unit = {
    val spark: SparkSession = SparkSession
      .builder()
      .master("local[*]")
      .appName("HiveRead")
      // 添加支援外置hive
      .enableHiveSupport()
      .getOrCreate()

    //    spark.sql("show databases").show
    spark.sql("use gmall").show
    //    spark.sql("show tables").show
    spark.sql("select * from emp").show
    spark.close()
  }
}      

控制台列印結果

++
||
++
++

+---+----+
| id|name|
+---+----+
|  1| aaa|
|  2| bbb|
|  3| ccc|
+---+----+      

碼雲位址