EMR的hive包升级和gateway启动hs2
yanqianglifei 2019-12-30
升级:
mkdir /opt/apps/hive
cd /opt/apps/hive
wget http://emr.oss-cn-hangzhou.aliyuncs.com/ecm-repo/hive/opay/apache-hive-2.3.5-bin.tar.gz
md5sum apache-hive-2.3.5-bin.tar.gz
229e32d99d8d288bdece8c897d6cc625 apache-hive-2.3.5-bin.tar.gz
tar xvf apache-hive-2.3.5-bin.tar.gz
1. Gateway升级Hive包
a.登录Gateway机器，将新的hive包解压到一个目录，比如/opt/apps/hive
b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
c. unlink /usr/lib/hive-current
d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
e. 建立其它软连接
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar
ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar
f. 测试一下是否正常
2. HiveServer升级Hive包
a. 登录emr-header-1, 将新的hive包解压到一个目录，比如/opt/apps/hive
b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
c. unlink /usr/lib/hive-current
d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
e. 建立其它软连接
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar
ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar
f. 登录emr-header-2，重复a~d的操作
g. 页面控制台重启HiveServer
h. 使用beeline/hue测试是否正常
gateway上启动hs2: (可选)
拷贝Ranger相关配置文件
以root用户执?以下命令
scp :/etc/ecm/hive-conf/ranger-hive-audit.xml /etc/ecm/hive-conf
scp :/etc/ecm/hive-conf/ranger-hive-security.xml /etc/ecm/hive-conf
scp :/etc/ecm/hive-conf/ranger-policymgr-ssl.xml /etc/ecm/hive-conf
scp :/etc/ecm/hive-conf/ranger-security.xml /etc/ecm/hive-conf
chown hadoop:hadoop /etc/ecm/hive-conf/ranger-*
启动HiveServer2
以hadoop用户执行以下命令
