HBase集群搭建1Master、2Slaves
HBase集群
Master.Hadoop 10.9.148.21 master(root,123456) /softs/hbase /softs/ hbase/logs
Slave1.Hadoop 10.9.148.26 slave1(root,123456) /softs/hbase /softs/ hbase/logs
Slave2.Hadoop 10.9.148.23 slave2(root,123456) /softs/hbase /softs/hbase/logs
hbase-1.2.6-bin.tar.gz
826 scp [email protected]:/softs/
827 scp [email protected]:/softs/
Master slave 相同:
tar –zxf hbase-1.2.6-bin.tar.gz
830 mv hbase-1.2.6 hbase
833 cd hbase
836 vi /etc/profile
添加内容:
#hbase
exportHBASE_HOME=/softs/hbase
exportPATH=$PATH:$HBASE_HOME
#stop hbase zookeeperrunning
exportHBASE_MANAGES_ZK=false
vi /etc/hosts
#zookeeper MasterSlave IP address
10.9.148.21 master
10.9.148.26 slave1
10.9.148.23 slave2
837 source /etc/profile
839 vi conf/hbase-site.xml
添加内容:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://10.9.148.21:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.master</name>
<value>master</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master:2181,slave1:2181,slave2:2181</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/softs/zookeeper/data</value>
</property>
</configuration>
840 vi conf/regionservers
添加内容:
slave1
slave2
vi conf/hbase-env.sh
添加内容:
export JAVA_HOME=/usr/local/jdk1.7
export HBASE_MANAGES_ZK=false
启动HBase集群
启动之前,在Hadoop的文件系统中创建hbase目录
850 start-all.sh
查看./logs目录下的
vihbase/logs/hbase-root-master-Master.Hadoop.log
发现slave1和slave2没有启动,错误提示:连接master超时(或者hbase集群节点HRegionServer启动后自动关闭)
出现问题的原因:3台主机的系统时钟不一致,相差超过30s导致。
解决问题:使用工具,统一时间(集群时钟管理工具)
在这里,我是用的是直接设置时钟:
1052 date -s 11:41:00
1053 clock -w
重新启动集群
1057 bin/start-hbase.sh
验证:
Master:
[[email protected] softs]# jps
3717 SecondaryNameNode
4606 QuorumPeerMain
13543 Jps
6909 HMaster
6756 NameNode
3911 ResourceManager
[[email protected] softs]#
Slave1:
[[email protected] hbase]# jps
24344 Jps
4148 SecondaryNameNode
3058 QuorumPeerMain
3981 DataNode
4310 ResourceManager
4427 NodeManager
23638 HRegionServer
[[email protected] hbase]#
Slave2:
[[email protected] hbase]# jps
3596 QuorumPeerMain
4048 DataNode
11495 HRegionServer
12157 Jps
[[email protected] hbase]#
在浏览器访问主节点16010端口(即http://10.9.148.21:16010/),可以产看HBase Web UI
在hadoop的文件系统中:
HBase启动几次失败,后来改正HBase配置后,依旧失败。可以试着将hadoop的HBase的文件目录删掉重建,再重新启动HBase集群。这样可以成功。
HBaseJava使用
Maven依赖:
<!-- hbase -->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-it</artifactId>
<version>1.2.6</version>
</dependency>
<!--spring -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context-support</artifactId>
<version>3.2.13.RELEASE</version>
</dependency>
本地计算机C:\WINDOWS\system32\drivers\etc\hosts
增加:
10.9.148.21 master
10.9.148.26 slave1
10.9.148.23 slave2
为HBase Java API的配置文件做准备:
static{
conf=HBaseConfiguration.create();
conf.set("hbase.rootdir","hdfs://10.9.148.21:9000/hbase");
conf.set("hbase.master","10.9.148.21");
conf.set("hbase.zookeeper.quorum","10.9.148.21,10.9.148.26,10.9.148.23");
}
Java操作HBase集群,可以创建表删除表。当进行对表的put、get、admin.tableExists(tablename)等时,运行java代码出错。查看zookeeper日志发现:
2017-08-03 18:42:58,643 [myid:1] - INFO [QuorumPeer[myid=1]/0.0.0.0:2181:[email protected]]- Getting a diff from the leader 0x1000000a7
2017-08-03 18:42:58,644 [myid:1] - INFO [QuorumPeer[myid=1]/0.0.0.0:2181:[email protected]] - Snapshotting:0x1000000a7 to ./data/version-2/snapshot.1000000a7
2017-08-03 18:43:00,905 [myid:1] - INFO [/10.9.148.21:3888:[email protected]]- Received connection request /10.9.148.23:43171
2017-08-03 18:43:00,908 [myid:1] - INFO [WorkerReceiver[myid=1]:[email protected]] - Notification: 1(message format version), 3 (n.leader), 0x1000000a7 (n.zxid), 0x1 (n.round),LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) FOLLOWING (my state)
2017-08-03 18:43:00,922 [myid:1] - INFO [WorkerReceiver[myid=1]:[email protected]] - Notification: 1(message format version), 3 (n.leader), 0x1000000a7 (n.zxid), 0x2 (n.round),LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) FOLLOWING (my state)
2017-08-03 18:43:23,050 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]] -Accepted socket connection from /10.9.148.21:20942
2017-08-03 18:43:23,063 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]] - Clientattempting to establish new session at /10.9.148.21:20942
2017-08-03 18:43:23,065 [myid:1] - WARN [QuorumPeer[myid=1]/0.0.0.0:2181:[email protected]] - Got zxid 0x200000001expected 0x1
2017-08-03 18:43:23,071 [myid:1] - INFO [CommitProcessor:1:[email protected]] -Established session 0x15da7b0f5d30000 with negotiated timeout 800000 for client/10.9.148.21:20942
2017-08-03 18:43:24,321 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]]- Accepted socket connection from /10.9.148.23:37715
2017-08-03 18:43:24,323 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]] - Clientattempting to establish new session at /10.9.148.23:37715
2017-08-03 18:43:24,326 [myid:1] - INFO [CommitProcessor:1:[email protected]] -Established session 0x15da7b0f5d30001 with negotiatedtimeout 800000 for client /10.9.148.23:37715
2017-08-03 18:43:25,495 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]]- Accepted socket connection from /10.9.148.21:20951
2017-08-03 18:43:25,496 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:[email protected]] - Clientattempting to establish new session at /10.9.148.21:20951
2017-08-03 18:43:25,499 [myid:1] - INFO [CommitProcessor:1:[email protected]] -Established session 0x15da7b0f5d30002 with negotiated timeout 800000 for client/10.9.148.21:20951
Zoomkeeper连接超时。
解决方法:
在运行java代码的windows本地计算机中:C:\WINDOWS\system32\drivers\etc\hosts
增加:
10.9.148.21 Master.Hadoop
10.9.148.26 Slave1.Hadoop
10.9.148.23 Slave2.Hadoop
Master.Hadoop Slave1.Hadoop Slave2.Hadoop分别为三台服务器的主机名。
如此操作成功:
常用操作:
对表的操作
创建表