天天看點

Hive最全常見錯誤及解決方案hive --service metastore &

1)SecureCRT 7.3出現亂碼或者删除不掉資料,免安裝版的SecureCRT 解除安裝或者用虛拟機直接操作或者換安裝版的SecureCRT

2)連接配接不上mysql資料庫

(1)導錯驅動包,應該把mysql-connector-java-5.1.27-bin.jar導入/opt/module/hive/lib的不是這個包。錯把mysql-connector-java-5.1.27.tar.gz導入hive/lib包下。

(2)修改user表中的主機名稱沒有都修改為%,而是修改為localhost

3)hive預設的輸入格式處理是CombineHiveInputFormat,會對小檔案進行合并。

hive (default)> set hive.input.format;

hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat

可以采用HiveInputFormat就會根據分區數輸出相應的檔案。

hive (default)> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

4)不能執行mapreduce程式

可能是hadoop的yarn沒開啟。

5)啟動mysql服務時,報MySQL server PID file could not be found! 異常。

在/var/lock/subsys/mysql路徑下建立hadoop102.pid,并在檔案中添加内容:4396

6)報service mysql status MySQL is not running, but lock file (/var/lock/subsys/mysql[失敗])異常。

解決方案:在/var/lib/mysql 目錄下建立: -rw-rw----. 1 mysql mysql 5 12月 22 16:41 hadoop102.pid 檔案,并修改權限為 777。

7)JVM堆記憶體溢出

描述:java.lang.OutOfMemoryError: Java heap space

解決:在yarn-site.xml中加入如下代碼

yarn.scheduler.maximum-allocation-mb 2048 yarn.scheduler.minimum-allocation-mb 2048 yarn.nodemanager.vmem-pmem-ratio 2.1 mapred.child.java.opts -Xmx1024m

8、執行/usr/bin/hive指令進入hive時遇到如下錯誤:

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.7.3-1.cdh5.7.3.p0.5/jars/hive-common-1.1.0-cdh5.7.3.jar!/hive-log4j.properties

Exception in thread “main” java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:540)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1493)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)

at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)

at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)

… 8 more

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)

… 13 more

Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

at org.apache.thrift.transport.TSocket.open(TSocket.java:187)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:427)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)

at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)

at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)

at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:579)

at org.apache.thrift.transport.TSocket.open(TSocket.java:182)

… 21 more

)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:474)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)

at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

… 18 more

原因:沒有正常啟動Hive的Metastore Server 程序

解決辦法:找到Metaserver Server程序所在的節點,執行如下指令:

hive --service metastore &

9、執行/usr/bin/hive指令進入hive時遇到如下錯誤:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1

at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)

at org.datanucleus.api.jdo.JDOPersistenceManager.getDataStoreConnection(JDOPersistenceManager.java:2259)

at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.trySetAnsiQuotesForMysql(MetaStoreDirectSql.java:166)

原因:資料庫驅動包版本不對應

解決辦法:更換對應的資料庫驅動

10、在hive中執行SQL時遇到如下錯誤:

Diagnostic Messages for this Task:

Container launch failed for container_1431310046688_0001_02_000013 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

This token is expired. current time is 1431311178863 found 1431311136701

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

原因:叢集時間不同步

解決辦法:同步叢集時間(date -s '2018-7-1 15:08:00)

11、

錯誤資訊:[ERROR] Terminal initialization failed; falling back to unsupported

解決方案:将hadoop中share/hadoop/yarn/lib路徑下的jline包換成hive中比對的jar包。

注意需要删除share/hadoop/yarn/lib下原有的jline包

12、

錯誤資訊:Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:

file:///usr/local/hive-0.13.0/lib/hive-hbase-handler-0.13.0.jar

解決方案:修改hive中的conf下的hive-site.xml配置檔案,hive.aux.jars.path配置改成file:///。注意中間不要有換行符,也就是不要回車。多個的話直接","号分隔開。

13、

錯誤提示:java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解決方案:拷貝hive的lib目錄下的jline高版本jar包到,hadoop的/share/hadoop/yarn/lib目錄下

14、

錯誤提示:java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解決方案:拷貝hive的lib目錄下的jline高版本jar包到,hadoop的/share/hadoop/yarn/lib目錄下