hadoop集群搭建问题集锦
1.集群启动后,各DataNode成功启动但是在查询LiveNodes时与实际数量(3个)不符
查看日志所示错误:
2017-06-06 16:35:08,850 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-749503087-220.192.10.13-1488958653636 (Datanode Uuid 013abfe9-a8d8-4bcc-b173-db0fc8e582f8) service to hadoop05/220.192.10.14:9000 Blockpool ID mismatch: previously connected to Blockpool ID BP-749503087-220.192.10.13-1488958653636 but now connected to Blockpool ID BP-749503087-220.192.10.14-1488958653636
进入Namenode所在节点主机(220.192.10.14),修改/hadoop/hadoop-2.7.3/tmp/dfs/name/current/VERSION的Blockpool ID,将BP-749503087-220.192.10.14-1488958653636修改为BP-749503087-220.192.10.13-14889586 53636即可,重启集群可看到三台DataNode正常显示:
2.hive使用beeline时无法登录:
错误:Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate root
解决办法:在hadoop的配置文件core-site.xml中添加以下配置,若有其他节点,将其拷贝至各节点,重启hadoop集群即可。
<!--root指登录使用主机的用户-->
<property>
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.root.groups</name>
<value>*</value>
</property>