Hive下载安装

下载地址

下载地址

安装步骤

解压hive到指定文件夹

1
$ tar -xzvf apache-hive-3.1.2-bin.tar.gz

在/conf文件夹下配置hive-site.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.118.8:23306/hive_test</value>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value>
</property>

<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
</configuration>

如果没有该文件,直接新建即可!该配置文件主要配置了mysql 的连接地址!

导入mysql驱动到lib文件夹

导入mysql-connector-java-5.1.20.jar到hive的lib文件夹下。

初始化hive在mysql中的表

1
$ schematool -dbType mysql -initSchema

启动停止hive

进入hive客户端

1
$ hive

启动hiveServer

1
$ hiveserver2 &

hiveServer是hive提供的jdbc连接hive的一种方式

测试

在hive中创建一个库

1
2
3
1 hive> create database hive_1;
2 OK
3 Time taken: 1.432 seconds

查看hive中的库

1
2
3
1 hive> create database hive_1;
2 OK
3 Time taken: 1.432 seconds

切换到创建的库上

1
2
3
1 hive> create database hive_1;
2 OK
3 Time taken: 1.432 seconds

在当前库创建一张表

1
2
3
1 hive> create database hive_1;
2 OK
3 Time taken: 1.432 seconds

查看当前库里面所有的表

1
2
3
4
hive> show tables;
OK
hive_01
Time taken: 0.107 seconds, Fetched: 1 row(s)

通过mysql查询创建的表

1
select * from TBLS;

通过hsfs的web界面查看我们创建的表

1585207863239
1585207863239

以上就是hive 的安装步骤!!!

遇到问题

启动hive报错

报错详情

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/software/hive/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/software/hadoop/hadoop-3.1.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:518)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:536)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:430)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5141)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5104)
at org.apache.hive.beeline.HiveSchemaTool.<init>(HiveSchemaTool.java:96)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1473)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)

错误原因

guava版本太低造成的。

错误解决

提升版本,删除lib文件夹下的guava原jar包,然后去maven仓库下载最新jar包放入即可!

创建库报错

错误详情

1
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

错误原因

hive数据库格式化失败

错误详情

1
2
3
4
5
6
7
8
9
10
11
12
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/software/hive/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/software/hadoop/hadoop-3.1.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://219.145.62.237:23306/hive_test
Metastore Connection Driver : com.MySQL.jdbc.Driver
Metastore connection User: root
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.MySQL.jdbc.Driver
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

错误原因

hive/lib文件夹下面没有导入mysql 的驱动。

还有可能是配置文件错误了。

错误解决

导入mysql驱动,用官方的配置文件,不要在网上乱粘!!!

HiveServer2启动报错

错误详情

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74) ~[hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.initTriggers(TezSessionPoolManager.java:207) ~[hive-exec-3.1.2.jar:3.1.2]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:114) ~[hive-exec-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:839) ~[hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:822) ~[hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:745) ~[hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1037) [hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140) [hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305) [hive-service-3.1.2.jar:3.1.2]
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149) [hive-service-3.1.2.jar:3.1.2]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.3.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.3.jar:?]
Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.api.TezConfiguration
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_212]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_212]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_212]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_212]
... 16 more

错误原因

这种问题是从机上运行的Container试图使用过多的内存,而被NodeManager kill掉了。

错误解决

关掉虚拟内存检查,修改yarn-site.xml文件(添加相关代码)!!

1
2
cd /opt/module/hadoop/etc/hadoop
vim yarn-site.xml

关掉虚拟机内存检查!!

1
2
3
4
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>

参考文档

官方文档


扫描以下公众号关注小猿↓↓↓↓↓↓↓↓

更多资讯请在简书、微博、今日头条、掘金、CSDN都可以通过搜索“Share猿”找到小猿哦!!!