Hbase1.3.1集群
-
- Hbase1.3.1集群
集群规划：
主机名 IP 安装的软件 运行的进程
hdp-ha-01 192.168.1.63 jdk、hadoop、zookeeper、hbase NameNode,SecondaryNameNode,NodeManager,ResourceManager,QuorumPeerMain、hmaster
hdp-ha-02 192.168.1.63 jdk、hadoop、zookeeper、hbase DataNode、NodeManager、QuorumPeerMain、、hregionserver
hdp-ha-03 192.168.1.64 jdk、hadoop、zookeeper、hbase DataNode、NodeManager、QuorumPeerMain、hregionserver
-
-
- 准备工作
-
- 安装jdk
- Hadoop集群
- Zookeeper集群
安装步骤：
（1）下载hbase安装包，上传hbase安装包到linux
（2）解压
（3）配置hbase集群，要修改3个文件
注意：要把hadoop的hdfs-site.xml和core-site.xml 放到hbase/conf下
（3.1）修改hbase-env.sh
|
export JAVA_HOME=/usr/java/jdk1.7.0_55 //告诉hbase使用外部的zk export HBASE_MANAGES_ZK=false |
（3.2）修改 hbase-site.xml
|
<configuration> <!-- 指定hbase在HDFS上存储的路径 --> <property> <name>hbase.rootdir</name> <value>hdfs://ns1/hbase</value> </property> <!-- 指定hbase是分布式的 --> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <!-- 指定zk的地址，多个用“,”分割 --> <property> <name>hbase.zookeeper.quorum</name> <value>hdp-server-01:2181, hdp-server-02:2181, hdp-server-03:2181</value> </property> </configuration> |
（3.3）修改 regionservers
|
weekend03 weekend04 weekend05 weekend06 |
(3.4) 拷贝hbase到其他节点
scp -r /weekend/hbase-0.96.2-hadoop2/ weekend02:/weekend/
scp -r /weekend/hbase-0.96.2-hadoop2/ weekend03:/weekend/
scp -r /weekend/hbase-0.96.2-hadoop2/ weekend04:/weekend/
scp -r /weekend/hbase-0.96.2-hadoop2/ weekend05:/weekend/
scp -r /weekend/hbase-0.96.2-hadoop2/ weekend06:/weekend/
(4) 将配置好的HBase拷贝到每一个节点并同步时间。
(5) 启动所有的hbase进程
首先启动zk集群
./zkServer.sh start
启动hbase集群
start-dfs.sh
启动hbase，在主节点上运行：
start-hbase.sh
(6) 通过浏览器访问hbase管理页面
192.168.1.201:60010
(7) 为保证集群的可靠性，要启动多个HMaster
hbase-daemon.sh start master
-
-
- 解压缩到/usr/local目录
-
[[email protected] download]#
[[email protected] download]# tar -zxf hbase-1.3.1-bin.tar.gz -C /usr/local/
[[email protected] download]# cd /usr/local/
[[email protected] local]# ls
bin games include libexec share ucenter1.5 vmware-tools-distrib
etc hadoop-2.6.2 lib redis-3.2.8 src ucenter_home2.0 zookeeper-3.4.6
fail2ban-0.8.14 hbase-1.3.1 lib64 sbin tomcat7 VMwareTools-9.6.2-1688356.tar.gz
[[email protected] local]#
[[email protected] local]# cd hbase-1.3.1/
[[email protected] hbase-1.3.1]# ls
bin conf LEGAL LICENSE.txt NOTICE.txt xuegod65
CHANGES.txt hbase-webapps lib logs README.txt
[[email protected] hbase-1.3.1]#
-
-
- 设置环境变量
-
[[email protected] hbase-1.3.1]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.7.0_15
export JRE_HOME=/usr/java/jdk1.7.0_15/jre
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
export HADOOP_HOME=/usr/local/hadoop-2.6.2
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
export ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.6
export PATH=$ZOOKEEPER_HOME/bin::$PATH
export HBASE_HOME=/usr/local/hbase-1.3.1
export PATH=$HBASE_HOME/bin::$PATH
[[email protected] hbase-1.3.1]#
-
-
- 修改hbase-env.sh配置文件
-
[[email protected] hbase-1.3.1]# ls
bin conf LEGAL LICENSE.txt NOTICE.txt xuegod65
CHANGES.txt hbase-webapps lib logs README.txt
[[email protected] hbase-1.3.1]# cd conf/
[[email protected] conf]# ls
core-site.xml hbase-env.sh hdfs-site.xml
hadoop-metrics2-hbase.properties hbase-policy.xml log4j.properties
hbase-env.cmd hbase-site.xml regionservers
[[email protected] conf]#
修改hbase-env.sh
[[email protected] conf]# vim hbase-env.sh
#添加一下内容:
# The java implementation to use. Java 1.7+ required.
export JAVA_HOME=/usr/java/jdk1.7.0_15
# Extra Java CLASSPATH elements. Optional.
#设置到Hadoop的etc/hadoop目录是用来引导Hbase找到Hadoop,即hbase和hadoop进行关联
export HBASE_CLASSPATH=/usr/local/hadoop-2.6.2/etc/hadoop
export HBASE_MANAGES_ZK=false 默认为true,使用自带的zookeeper,如果设置为false,需要自己配置zookeeper.
[[email protected] conf]#
-
-
- 修改hbase-site.xml配置文件
-
[[email protected] conf]#
[[email protected] conf]# vim hbase-site.xml
<configuration>
<!-- 指定hbase在HDFS上存储的路径，hbase.rootdir的前端与$HADOOP_HOME/conf/core-site.xml的fs.defaultFS一致 -->
<property>
<name>hbase.rootdir</name>
<value>hdfs://192.168.3.65:9000/hbase</value>
</property>
<!-- 指定hbase是分布式的 -->
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<!--本地文件系统的临时文件夹。可以修改到一个更为持久的目录上。(/tmp会在重启时清除) -->
<property>
<name>hbase.tmp.dir</name>
<value>/home/hbase/tmp</value>
</property>
<!--如果只设置单个 Hmaster，那么 hbase.master 属性参数需要设置为 master5:60000 (主机名:60000) -->
<!--如果要设置多个 Hmaster，那么我们只需要提供端口 60000，因为选择真正的 master 的事情会有 zookeeper 去处理 -->
<property>
<name>hbase.master</name>
<value>60000</value>
</property>
<!--这个参数用户设置 ZooKeeper 快照的存储位置，默认值为 /tmp，显然在重启的时候会清空。因为笔者的 ZooKeeper 是独立安装的，所以这里路径是指向了 $ZOOKEEPER_HOME/conf/zoo.cfg 中 dataDir 所设定的位置 -->
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/zookeeper/data</value>
</property>
<!--客户端连接 ZooKeeper服务地址，这里有几个集群zookeeper就配置几个，多个以逗号分隔，zookeeper集群最好是奇数个，由于zookeeper的选举机>制是过半原则，偶数集群选举时间比较长 -->
<property>
<name>hbase.zookeeper.quorum</name>
<value>xuegod63:2181,xuegod64:2181,xuegod65:2181</value>
</property>
<!--表示客户端连接 ZooKeeper 的端口 -->
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<!--ZooKeeper 会话超时。Hbase 把这个值传递改 zk 集群，向它推荐一个会话的最大超时时间 -->
<property>
<name>zookeeper.session.timeout</name>
<value>120000</value>
</property>
<!--当 regionserver 遇到 ZooKeeper session expired ， regionserver 将选择 restart 而不是 abort -->
<property>
<name>hbase.regionserver.restart.on.zk.expire</name>
<value>true</value>
</property>
</configuration>
[[email protected] conf]#
-
-
- 拷贝hadoop目录下hdfs-site.xml和core-site.xml 放到hbase/conf下
-
要把hadoop的hdfs-site.xml和core-site.xml 放到hbase/conf下
[[email protected] conf]# cp /usr/local/hadoop-2.6.2/etc/hadoop/hdfs-site.xml ./
[[email protected] conf]# cp /usr/local/hadoop-2.6.2/etc/hadoop/core-site.xml ./
[[email protected] conf]# ls
core-site.xml hbase-env.sh hdfs-site.xml
hadoop-metrics2-hbase.properties hbase-policy.xml log4j.properties
hbase-env.cmd hbase-site.xml regionservers
[[email protected] conf]# vim core-site.xml
[[email protected] conf]#
-
-
- 修改regionservers文件
-
Master只能有一个，如果启动多个master，则其他的都为standby。Regionserver可以有多个。
编辑所有机器的HBase的HMasters和HRegionServers。修改/conf 文件夹下的regionservers文件。添加DataNode的IP即可，通常是将regionserver与datanode服务器放在一起
[[email protected] conf]#
[[email protected] conf]# ls
core-site.xml hbase-env.sh hdfs-site.xml
hadoop-metrics2-hbase.properties hbase-policy.xml log4j.properties
hbase-env.cmd hbase-site.xml regionservers
[[email protected] conf]# vim regionservers
xuegod63
xuegod64
xuegod65
[[email protected] conf]#
-
-
- 将hbase拷贝到其他服务器
-
[[email protected] local]#
[[email protected] local]# scp -r hbase-1.3.1/ xuegod64:/usr/local/
[[email protected] local]# scp -r hbase-1.3.1/ xuegod65:/usr/local/
[[email protected] local]#
[[email protected]xuegod64 local]# ls
bin hadoop-2.7.3.tar lib64 share zookeeper-3.4.6
etc hbase-1.3.1 libexec src
games include redis-3.2.8 VMwareTools-9.6.2-1688356.tar.gz
hadoop-2.6.2 lib sbin vmware-tools-distrib
[[email protected] local]#
[[email protected]xuegod65 ~]# cd /usr/local/
[[email protected] local]# ls
bin hadoop-2.6.2 lib64 share ucenter_home2.0
etc hbase-1.3.1 libexec src VMwareTools-9.6.2-1688356.tar.gz
fail2ban-0.8.14 include redis-3.2.8 tomcat7 vmware-tools-distrib
games lib sbin ucenter1.5 zookeeper-3.4.6
[[email protected] local]#
-
-
- 启动hbase服务
-
[[email protected] local]# start-hbase.sh
starting master, logging to /usr/local/hbase-1.3.1/logs/hbase-root-master-xuegod63.out
xuegod65: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-xuegod65.out
xuegod64: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-xuegod64.out
xuegod63: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-xuegod63.out
[[email protected] local]# jps
7026 NameNode
6814 QuorumPeerMain
8258 HRegionServer
8304 Jps
8133 HMaster
7270 SecondaryNameNode
7440 ResourceManager
7539 NodeManager
[[email protected] local]#
Xuegod64服务：
[[email protected] ~]# jps
5656 HRegionServer
6616 Jps
5307 DataNode
5188 QuorumPeerMain
5411 NodeManager
[[email protected] ~]#
Xuegod65服务：
[[email protected] local]# jps
4435 Jps
4275 HRegionServer
4030 NodeManager
3833 QuorumPeerMain
[[email protected] local]#
注意：在集群使用命令start-hbase.sh启动的服务为Hmaster,并且master只能有一个，其他的为standby, HregionServer则为hbase中regionserver配置文件中的配置的主机。可以为多个。
可以通过手动启动standby节点，如下：
[[email protected] hbase-1.3.1]# hbase-daemon.sh start master
starting master, logging to /usr/local/hbase-1.3.1/logs/hbase-root-master-xuegod63.out
[[email protected] hbase-1.3.1]#
-
-
- 查看hbase服务是否已经启动
-
[[email protected] hbase-1.3.1]# jps
7026 NameNode
6814 QuorumPeerMain
9028 Jps
8258 HRegionServer
7270 SecondaryNameNode
7440 ResourceManager
8797 Hmaster #表示hbase已经启动成功
7539 NodeManager
[[email protected] hbase-1.3.1]#
-
-
- 登录zookeeper查看hbase
-
[[email protected] hbase-1.3.1]#
[[email protected] hbase-1.3.1]# zkCli.sh
Connecting to localhost:2181
2017-09-25 23:18:29,809 [myid:] - INFO [main:[email protected]] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2017-09-25 23:18:29,823 [myid:] - INFO [main:[email protected]] - Client environment:host.name=xuegod63
2017-09-25 23:18:29,824 [myid:] - INFO [main:[email protected]] - Client environment:java.version=1.7.0_15
2017-09-25 23:18:29,834 [myid:] - INFO [main:[email protected]] - Client environment:java.vendor=Oracle Corporation
2017-09-25 23:18:29,834 [myid:] - INFO [main:[email protected]] - Client environment:java.home=/usr/java/jdk1.7.0_15/jre
2017-09-25 23:18:29,834 [myid:] - INFO [main:[email protected]] - Client environment:java.class.path=/usr/local/zookeeper-3.4.6/bin/../build/classes:/usr/local/zookeeper-3.4.6/bin/../build/lib/*.jar:/usr/local/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/usr/local/zookeeper-3.4.6/bin/../conf:/usr/java/jdk1.7.0_15/lib:/usr/java/jdk1.7.0_15/jre/lib:
2017-09-25 23:18:29,834 [myid:] - INFO [main:[email protected]] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-09-25 23:18:29,835 [myid:] - INFO [main:[email protected]] - Client environment:java.io.tmpdir=/tmp
2017-09-25 23:18:29,835 [myid:] - INFO [main:[email protected]] - Client environment:java.compiler=<NA>
2017-09-25 23:18:29,835 [myid:] - INFO [main:[email protected]] - Client environment:os.name=Linux
2017-09-25 23:18:29,835 [myid:] - INFO [main:[email protected]] - Client environment:os.arch=amd64
2017-09-25 23:18:29,836 [myid:] - INFO [main:[email protected]] - Client environment:os.version=2.6.32-431.el6.x86_64
2017-09-25 23:18:29,836 [myid:] - INFO [main:[email protected]] - Client environment:user.name=root
2017-09-25 23:18:29,836 [myid:] - INFO [main:[email protected]] - Client environment:user.home=/root
2017-09-25 23:18:29,836 [myid:] - INFO [main:[email protected]] - Client environment:user.dir=/usr/local/hbase-1.3.1
2017-09-25 23:18:29,841 [myid:] - INFO [main:[email protected]] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 [email protected]
Welcome to ZooKeeper!
2017-09-25 23:18:29,917 [myid:] - INFO [main-SendThread(localhost:2181):[email protected]] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-09-25 23:18:29,945 [myid:] - INFO [main-SendThread(localhost:2181):[email protected]] - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
JLine support is enabled
2017-09-25 23:18:29,990 [myid:] - INFO [main-SendThread(localhost:2181):[email protected]] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15eb96af2fd0008, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[hbase, zookeeper]
[zk: localhost:2181(CONNECTED) 1] ls /hbase
[meta-region-server, backup-masters, switch, table, draining, region-in-transition, table-lock, running, master, namespace, hbaseid, online-snapshot, replication, splitWAL, recovering-regions, rs, flush-table-proc]
[zk: localhost:2181(CONNECTED) 2]
-
-
- 关闭hbase服务
-
[[email protected] hbase-1.3.1]#
[[email protected] hbase-1.3.1]# stop-hbase.sh
stopping hbase.....................
[[email protected] hbase-1.3.1]#
-
-
- 测试hbase服务
-
浏览器输入：http://192.168.3.63:16010/ hbase默认端口是16010。
-
-
- hbase通过脚本复制到其他节点
-
创建脚本：
[[email protected] ~]# vim cphbase.sh
#!/bin/bash
if [ "$#" != "2" ]; then
echo "usage:$(basename $0) <dir-name> <ln-name>"
echo "example:$(basename $0) hbase-1.3 hbase"
fi
SRC_PATH="/usr/local/$1/conf/regionservers"
for srv in $(cat $SRC_PATH);
do
echo "sending command to $srv...";
rsync -vaz --exclude='logs/*' /usr/local/$1 $srv:/root/test/
# ssh $srv "rm -rf /usr/local/$2;ln -s /usr/local/$1 /usr/local/$2"
done
echo "done.............";
~
[[email protected] ~]#
执行复制操作：
[[email protected] ~]# sh cphbase.sh hbase-1.3.1 hbase-1.3.1