天天看点

Hive最全常见错误及解决方案hive --service metastore &

1)SecureCRT 7.3出现乱码或者删除不掉数据,免安装版的SecureCRT 卸载或者用虚拟机直接操作或者换安装版的SecureCRT

2)连接不上mysql数据库

(1)导错驱动包,应该把mysql-connector-java-5.1.27-bin.jar导入/opt/module/hive/lib的不是这个包。错把mysql-connector-java-5.1.27.tar.gz导入hive/lib包下。

(2)修改user表中的主机名称没有都修改为%,而是修改为localhost

3)hive默认的输入格式处理是CombineHiveInputFormat,会对小文件进行合并。

hive (default)> set hive.input.format;

hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat

可以采用HiveInputFormat就会根据分区数输出相应的文件。

hive (default)> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

4)不能执行mapreduce程序

可能是hadoop的yarn没开启。

5)启动mysql服务时,报MySQL server PID file could not be found! 异常。

在/var/lock/subsys/mysql路径下创建hadoop102.pid,并在文件中添加内容:4396

6)报service mysql status MySQL is not running, but lock file (/var/lock/subsys/mysql[失败])异常。

解决方案:在/var/lib/mysql 目录下创建: -rw-rw----. 1 mysql mysql 5 12月 22 16:41 hadoop102.pid 文件,并修改权限为 777。

7)JVM堆内存溢出

描述:java.lang.OutOfMemoryError: Java heap space

解决:在yarn-site.xml中加入如下代码

yarn.scheduler.maximum-allocation-mb 2048 yarn.scheduler.minimum-allocation-mb 2048 yarn.nodemanager.vmem-pmem-ratio 2.1 mapred.child.java.opts -Xmx1024m

8、执行/usr/bin/hive命令进入hive时遇到如下错误:

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.7.3-1.cdh5.7.3.p0.5/jars/hive-common-1.1.0-cdh5.7.3.jar!/hive-log4j.properties

Exception in thread “main” java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:540)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1493)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)

at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)

at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)

… 8 more

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)

… 13 more

Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

at org.apache.thrift.transport.TSocket.open(TSocket.java:187)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:427)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)

at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1491)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:67)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)

at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2935)

at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2954)

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:579)

at org.apache.thrift.transport.TSocket.open(TSocket.java:182)

… 21 more

)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:474)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:237)

at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

… 18 more

原因:没有正常启动Hive的Metastore Server 进程

解决办法:找到Metaserver Server进程所在的节点,执行如下命令:

hive --service metastore &

9、执行/usr/bin/hive命令进入hive时遇到如下错误:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1

at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)

at org.datanucleus.api.jdo.JDOPersistenceManager.getDataStoreConnection(JDOPersistenceManager.java:2259)

at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.trySetAnsiQuotesForMysql(MetaStoreDirectSql.java:166)

原因:数据库驱动包版本不对应

解决办法:更换对应的数据库驱动

10、在hive中执行SQL时遇到如下错误:

Diagnostic Messages for this Task:

Container launch failed for container_1431310046688_0001_02_000013 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

This token is expired. current time is 1431311178863 found 1431311136701

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

原因:集群时间不同步

解决办法:同步集群时间(date -s '2018-7-1 15:08:00)

11、

错误信息:[ERROR] Terminal initialization failed; falling back to unsupported

解决方案:将hadoop中share/hadoop/yarn/lib路径下的jline包换成hive中匹配的jar包。

注意需要删除share/hadoop/yarn/lib下原有的jline包

12、

错误信息:Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:

file:///usr/local/hive-0.13.0/lib/hive-hbase-handler-0.13.0.jar

解决方案:修改hive中的conf下的hive-site.xml配置文件,hive.aux.jars.path配置改成file:///。注意中间不要有换行符,也就是不要回车。多个的话直接","号分隔开。

13、

错误提示:java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解决方案:拷贝hive的lib目录下的jline高版本jar包到,hadoop的/share/hadoop/yarn/lib目录下

14、

错误提示:java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

解决方案:拷贝hive的lib目录下的jline高版本jar包到,hadoop的/share/hadoop/yarn/lib目录下