一、下载相关安装包
- https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm
- https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm
- https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm
- https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm
- https://archive.cloudera.com/cdh6/6.1.0/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel
- https://archive.cloudera.com/cdh6/6.1.0/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha256
- https://archive.cloudera.com/cdh6/6.1.0/parcels/manifest.json
二、准备四台服务器( 密码设置相同
)(1主3从)
密码设置相同
主机 | IP |
---|---|
cdh-master | 192.168.30.110 |
cdh-slave-1 | 192.168.30.111 |
cdh-slave-2 | 192.168.30.112 |
cdh-slave-3 | 192.168.30.113 |
三、修改主机名(所有节点)
-
hostnamectl set-hostname *上述对应的主机名*
- 将以下内容添加到 /etc/hosts 文件:
192.168.30.110 cdh-master
192.168.30.111 cdh-slave-1
192.168.30.112 cdh-slave-2
192.168.30.113 cdh-slave-3
四、关闭防火墙和 selinux(所有节点)
- 执行关闭命令:
systemctl stop firewalld.service
- 执行开机禁用防火墙命令:
systemctl disable firewalld.service
- 关闭selinux:
将vi /etc/sysconfig/selinux
改为SELINUX=enforcing
SELINUX=disabled
五、时钟同步(所有节点)
- 安装NTP
yum -y install ntp
- 编辑 /etc/ntp.conf 文件添加 NTP 服务器
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org
server 3.cn.pool.ntp.org
- 启动 NTP
systemctl start ntpd
- 手工同步网络时间
ntpdate -u 0.cn.pool.ntp.org
- 同步系统时钟
hwclock --systohc
- 开机自启动
chkconfig ntpd on
六、禁用透明大页面压缩(所有节点)
- 打开编辑
文件:/etc/rc.local
添加以下内容:vi /etc/rc.local
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
- 赋予执行权限:
chmod +x /etc/rc/local
七、安装jdk(所有节点)
- 切换到刚刚下载文件的目录
-
yum -y install oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm
- 在
末尾添加以下内容:/etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
八、安装 cloudera manager 包(所有节点)
- 切换到刚刚下载文件的目录
-
yum -y install cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm
-
yum -y install cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm
- 将
文件中/etc/cloudera-scm-agent/config.ini
改为server_host=localhost
server_host=cdh-master
九、 主节点
安装 mysql(cdh-master)
主节点
- 下载 mysql 驱动
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
- 解压
tar zxvf mysql-connector-java-5.1.46.tar.gz
- 创建目录
mkdir -p /usr/share/java/
- 拷贝至目录
cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
- 下载 mysql yum 文件
wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
-
yum -y install mysql57-community-release-el7-10.noarch.rpm
-
yum -y install mysql-community-server
- 启动mysql
systemctl start mysqld
- 设置开机自启动
systemctl enable mysqld
-
查看并记住密码grep "password" /var/log/mysqld.log
- 登录mysql
输入 刚刚看到的密码mysql -u root -p
- 修改密码
ALTER USER 'root'@'localhost' IDENTIFIED BY 'Qaz!@#$';
- 创建 cloudera-manager-server 数据库
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
- 创建 scm 库连接信息
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'Abc123!@#';
- 刷新
flush privileges;
- 退出mysql
exit;
十、 主节点
安装 cloudera manager-server(cdh-master)
主节点
- 切换至文件下载目录
yum -y install cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm
- 将已下载的
和CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel
拷贝至manifest.json
目录/opt/cloudera/parcel-repo/
- 生成 parcel 包的 sha 值:
sha1sum /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel
- 将 sha 值填入到
文件中:CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha
vi /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha
- 启动 Cloudera Manager Server
systemctl start cloudera-scm-server
十一、启动 cloudera-manager-agent(所有节点)
-
systemctl start cloudera-scm-agent
- 查看主节点启动日志(cdh-master) cloudera-scm-server 启动比较慢
看到下面的信息,说明启动完成tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
十二、登录 web
- http://cdh-master:7180
- 账号/密码:
admin / admin