黑猴子的家:Cloudera Manager 部署 CDH
1、官方网址安装指南
https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_b.html
2、服务器环境基本操作
(1)修改静态ip
(2)禁用selinux
(3)修改主机名
(4)修改主机名IP映射 hosts文件
(5)关闭防火墙
(6)安装httpd服务
(7)禁用ipv6
https://www.jianshu.com/p/76a415f9fb0f
(8)安装mod_ssl服务
https://www.jianshu.com/p/460cb8e845d3
(9)设置用户最大可打开文件数,进程数
https://www.jianshu.com/p/6700e6464975
(10)克隆虚拟机
https://www.jianshu.com/p/dbf1ea1ed326
(11)配置ssh免密登录
https://www.jianshu.com/p/2e7b920ce353
(12)安装jdk
https://www.jianshu.com/p/567aa4b2e6a0
(13)ntp时间同步
https://www.jianshu.com/p/930bb81b3eb1
(14)MariaDB-10.1.22_64安装
https://www.jianshu.com/p/ad3843a97230
3、Cloudera Manager下载地址
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.14.2_x86_64.tar.gz
http://archive.cloudera.com/cm5/installer/5.14.2/cloudera-manager-installer.bin
http://archive.cloudera.com/cm5/repo-as-tarball/5.14.2/[cm5.14.2-centos7.tar.gz
http://archive.cloudera.com/cdh5/parcels/5.14.2/manifest.json
http://archive.cloudera.com/cdh5/parcels/5.14.2/[CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.14.2/[CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel
4、安装依赖环境
[[email protected] ~]# yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs redhat-lsb
尖叫提示:所有节点都执行
5、解压
[[email protected] software]# tar -xzvf cloudera-manager-centos7-cm5.14.2_x86_64.tar.gz -C /opt/
6、添加mysql驱动
[[email protected] software]# cp mysql-connector-java-5.1.27-bin.jar /opt/cm-5.14.2/share/cmf/lib/
7、把CDH文件放入仓库
[[email protected] software]# cp -r parcel/* /opt/cloudera/parcel-repo/
[[email protected] software]# cd /opt/cloudera/parcel-repo/
[[email protected] parcel-repo]# ll
CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel
CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel
manifest.json
[[email protected] parcel-repo]# mv CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel.sha1 CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel.sha
8、配置从节点cloudera-manger-agent指向主节点服务器
[[email protected] ~]# vim /opt/cm-5.14.2/etc/cloudera-scm-agent/config.ini
server_host=node1
9、分发cm-5.14.2
[[email protected] opt]# mkdir -p /opt/cm-5.14.2/run/cloudera-scm-agent
[[email protected] opt]# rsync -rvl cm-5.14.2/ [email protected]:/opt/cm-5.14.2/
[[email protected] opt]# rsync -rvl cm-5.14.2/ [email protected]:/opt/cm-5.14.2/
[[email protected] opt]# rsync -rvl cm-5.14.2/ [email protected]:/opt/cm-5.14.2/
10、添加cloudera-scm用户组
[[email protected] opt]# useradd --system --home=/opt/cm-5.14.2/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
[[email protected] opt]# chown -R cloudera-scm:cloudera-scm cloudera/
[[email protected] opt]# chown -R cloudera-scm:cloudera-scm cm-5.14.1/
尖叫提示:所有节点都操作
11、创建CDH内置数据库cm
[[email protected] opt]# /opt/cm-5.14.2/share/cmf/schema/scm_prepare_database.sh mysql scm –hnode1 -uroot –p000000 --scm-host node1 scm scm scm
12、启动服务
[[email protected] ~]# cd /opt/cm-5.7.2/etc/init.d
[[email protected] init.d]# cloudera-scm-server start
[[email protected] init.d]# cloudera-scm-server status
[[email protected] init.d]# cloudera-scm-agent start
[[email protected] init.d]# cloudera-scm-agent status
[[email protected] init.d]# cloudera-scm-agent start
[[email protected] init.d]# cloudera-scm-agent status
[[email protected] init.d]# cloudera-scm-agent start
[[email protected] init.d]# cloudera-scm-agent status
尖叫提示:启动一个master和所有agent包括主节点上的agent
13、打开web界面,端口号:7180
http://node1:7180
尖叫提示: 用户名,密码均为admin
14、点击继续按钮
15、选择Cloudera Express → Continue
16、选择当前管理的主机
17、集群安装,选择CDH版本和Parcel存储库
18、安装选中的Parcel
19、检测主机正确性
20、集群安装,显示检查主机的正确性
21、swappiness 问题解决
[[email protected] ~]# cat /proc/sys/vm/swappiness
[[email protected] ~]# sysctl -w vm.swappiness=0
[[email protected] ~]# echo 'vm.swappiness=0' >> /etc/sysctl.conf
[[email protected] ~]# sysctl -p
尖叫提示:echo 0 > /proc/sys/vm/swappiness 也可以解决,记得sysctl -p 使其生效
22、大透明问题解决
[[email protected] cloudera]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[[email protected] cloudera]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] cloudera]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'>> /etc/rc.local
[[email protected] cloudera]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'>> /etc/rc.local
[[email protected] cloudera]# cat /etc/rc.local
尖叫提示:所有节点都操作
23、集群设置,选择安装的服务
24、集群设置,节点分配
尖叫提示:有一些需要用到mysql的服务,需要添加时,再去创建Database,比如Monitor服务
mysql> create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;