HDP2.6 安装部署文档

一、服务器环境配置

1 服务器列表

名称

地址

操作系统

root密码

master

192.168.217.130

Centos 7

root

slave1

192.168.217.131

Centos 7

root

slave2

192.168.217.132

Centos 7

root

2 配置ssh信任

ssh-******

cat .ssh/*.pub > .ssh/authorized_keys

scp -r .ssh [email protected]:/root/

scp -r .ssh [email protected]:/root/

3 更改hosts

在master 机器上执行如下命令

vi  /etc/hosts

192.168.217.130  master.bigdata.com master

192.168.217.131  slave1.bigdata.com slave1

192.168.217.132  slave2.bigdata.com slave2

scp /etc/hosts [email protected]:/etc/hosts

scp /etc/hosts [email protected]:/etc/hosts

4 配置防火墙

iptables –F

systemctl disable firewalld

service firewalld stop

setenforce 0

5 配置阿里yum源(可选)

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

yum makecache

6 安装mysql数据库

本实验在mysql服务安装在了master节点,也可选择其他节点或机器。

具体步骤如下:

wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm

rpm -ivh mysql57-community-release-el7-8.noarch.rpm

yum install mysql-server

[[email protected] ~]# service mysqld  start

[[email protected] ~]# grep "password" /var/log/mysqld.log

[[email protected] ~]# mysql -u root -p

Enter password:

mysql> set password = password("[email protected]#");

mysql> grant all privileges on *.* to [email protected]"%" identified by "[email protected]#" ;

mysql> flush privileges;

7 时钟同步

yum install -y ntp

systemctl enable ntpd

8 关闭SELinux、配置umask 值

setenforce 0

echo umask 0022 >> /etc/profile

9 安装mysql-java驱动

yum installmysql-connector-java.noarch

二、配置本地yum库

1 服务安装配置

yum install yum-utils createrepo

yum install httpd

service httpd start

2 下载镜像包,解压到指定目录

操作系统

HDP/Ambari

版本

库名称

下载链接

CentOS 7

2.6.4.0

HDP

http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0/HDP-2.6.4.0-centos7-rpm.tar.gz

HDP-UTILS

http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz

HDP-GPL

http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0/HDP-GPL-2.6.4.0-centos7-rpm.tar.gz

CentOS 7

2.6.1.5

Ambari

http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.5/ambari-2.6.1.5-centos7.tar.gz

下载后把镜像包解压缩,并且移动到/var/www/html/hdp 目录下;

tar -zxvf ambari-2.6.1.5-centos7.tar.gz

完成后目录结构如下:

HDP2.6 安装部署文档

三、安装配置ambari Server

1 配置ambari 的yum源(在master节点)

wget  -nv  http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.6.1.5/

ambari.repo -O /etc/yum.repos.d/ambari.repo

2 修改ambari.repo配置信息

HDP2.6 安装部署文档

3 执行ambari server安装

yum install ambari-server

4 配置ambari server

[[email protected] yum.repos.d]# ambari-server setup

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)? y

Customize user account for ambari-server daemon [y/n] (n)?

Adjusting ambari-server permissions and ownership...

Checking firewall status...

Checking JDK...

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

==============================================================================

Enter choice (1): 1

…………

Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz

Installing JDK to /usr/jdk64/

Successfully installed JDK to /usr/jdk64/

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip

Installing JCE policy...

Checking GPL software agreement...

GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html

Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)?

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)? y

Configuring database...

==============================================================================

Choose one of the following options:

[1] - PostgreSQL (Embedded)

[2] - Oracle

[3] - MySQL / MariaDB

[4] - PostgreSQL

[5] - Microsoft SQL Server (Tech Preview)

[6] - SQL Anywhere

[7] - BDB

==============================================================================

Enter choice (1): 3

Hostname (localhost): master.bigdata.com

Port (3306):

Database name (ambari):

Username (ambari):

Enter Database Password (bigdata): BigData123_

Configuring ambari database...

Configuring remote database connection properties...

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql (此处转下表的数据库操作)

Proceed with configuring remote database connection properties [y/n] (y)? y

Extracting system views...

ambari-admin-2.6.1.5.3.jar

Ambari Server 'setup' completed successfully.

数据库操作

[[email protected] ~]# mysql -u root -p

Enter password:

mysql> create database ambari;

mysql> grant all privileges on ambari.* to [email protected]"%" identified by "BigData123_";

mysql> use ambari;

Database changed

mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

Query OK, 0 rows affected (0.00 sec)

5 服务启停

启动:ambari-server start

停止:ambari-server stop

启动成功后可以通过浏览器访问管理界面。

URL: 192.168.217.130:8080

用户名:admin

密码:admin

HDP2.6 安装部署文档

四、HDP集权部署

1 登录ambari server 管理界面

1.在浏览器打开http://192.168.217.130:8080

2.输入用户名密码  默认admin/admin

HDP2.6 安装部署文档

2 进入集群安装向导

HDP2.6 安装部署文档

3 输入集群名称

HDP2.6 安装部署文档

4 选择hdp版本

在HDP/centos7/2.6.4.0-91/HDP-2.6.4.0-91.xml文件基础上,修改baseurl。

HDP2.6 安装部署文档

选择增加版本、上传修改后的版本文件。

HDP2.6 安装部署文档

增加版本后,直接下一步

HDP2.6 安装部署文档

5 确认主机列表(并上传id_rsa文件)

HDP2.6 安装部署文档

6 主机注册、主机环境检查

HDP2.6 安装部署文档

7 选择安装部署组件

HDP2.6 安装部署文档

8 分配服务角色

HDP2.6 安装部署文档

9 设置slave节点和客户端

HDP2.6 安装部署文档

10 配置hive组件

HDP2.6 安装部署文档

选择已有的mysql数据库,填入相关信息(用户名密码数据库地址)

数据库操作

[[email protected] ~]# mysql -u root -p

Enter password:

mysql> create database hive;

mysql> grant all privileges on hive.* to [email protected]"%" identified by "BigData123_";

mysql> flush privileges;

Ambari-server设置

[[email protected] bin]# rpm -qa |grep mysql-connector

mysql-connector-java-5.1.25-3.el7.noarch

[[email protected] bin]# rpm -ql mysql-connector-java-5.1.25-3.el7.noarch

……

/usr/share/java/mysql-connector-java.jar

……

[[email protected] java]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

Using python  /usr/bin/python

Setup ambari-server

Copying /usr/share/java/mysql-connector-java.jar to /var/lib/ambari-server/resources

If you are updating existing jdbc driver jar for mysql with mysql-connector-java.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.

JDBC driver was successfully initialized.

Ambari Server 'setup' completed successfully.

测试数据库连接。

HDP2.6 安装部署文档

11 设置Ambari Metrics的admin密码

HDP2.6 安装部署文档

密码:bigdata

12 设置smartsence 的admin的密码

HDP2.6 安装部署文档

密码:bigdata

13 确认安装信息  点击下一步

HDP2.6 安装部署文档

14 执行安装

HDP2.6 安装部署文档

15 安装成功,点击完成进入图形界面

HDP2.6 安装部署文档