天天看点

cloudera-cdh-6.1.0-centos-7 安装五、时钟同步(所有节点)十、主节点安装 cloudera manager-server(cdh-master)十一、启动 cloudera-manager-agent(所有节点)

一、下载相关安装包

  1. https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm
  2. https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm
  3. https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm
  4. https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm
  5. https://archive.cloudera.com/cdh6/6.1.0/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel
  6. https://archive.cloudera.com/cdh6/6.1.0/parcels/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha256
  7. https://archive.cloudera.com/cdh6/6.1.0/parcels/manifest.json

二、准备四台服务器(

密码设置相同

)(1主3从)

主机 IP
cdh-master 192.168.30.110
cdh-slave-1 192.168.30.111
cdh-slave-2 192.168.30.112
cdh-slave-3 192.168.30.113

三、修改主机名(所有节点)

  • hostnamectl set-hostname *上述对应的主机名*

  • 将以下内容添加到 /etc/hosts 文件:

192.168.30.110 cdh-master

192.168.30.111 cdh-slave-1

192.168.30.112 cdh-slave-2

192.168.30.113 cdh-slave-3

四、关闭防火墙和 selinux(所有节点)

  1. 执行关闭命令:

    systemctl stop firewalld.service

  2. 执行开机禁用防火墙命令:

    systemctl disable firewalld.service

  3. 关闭selinux:

    vi /etc/sysconfig/selinux

    SELINUX=enforcing

    改为

    SELINUX=disabled

五、时钟同步(所有节点)

  1. 安装NTP

    yum -y install ntp

  2. 编辑 /etc/ntp.conf 文件添加 NTP 服务器

server 0.cn.pool.ntp.org

server 1.cn.pool.ntp.org

server 2.cn.pool.ntp.org

server 3.cn.pool.ntp.org

  1. 启动 NTP

    systemctl start ntpd

  2. 手工同步网络时间

    ntpdate -u 0.cn.pool.ntp.org

  3. 同步系统时钟

    hwclock --systohc

  4. 开机自启动

    chkconfig ntpd on

六、禁用透明大页面压缩(所有节点)

  1. 打开编辑

    /etc/rc.local

    文件:

    vi /etc/rc.local

    添加以下内容:

    echo never > /sys/kernel/mm/transparent_hugepage/defrag

    echo never > /sys/kernel/mm/transparent_hugepage/enabled

  2. 赋予执行权限:

    chmod +x /etc/rc/local

七、安装jdk(所有节点)

  • 切换到刚刚下载文件的目录
  • yum -y install oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm

  • /etc/profile

    末尾添加以下内容:

    export JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera

    export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH

    export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

八、安装 cloudera manager 包(所有节点)

  • 切换到刚刚下载文件的目录
  • yum -y install cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm

  • yum -y install cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm

  • /etc/cloudera-scm-agent/config.ini

    文件中

    server_host=localhost

    改为

    server_host=cdh-master

九、

主节点

安装 mysql(cdh-master)

  • 下载 mysql 驱动

    wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz

  • 解压

    tar zxvf mysql-connector-java-5.1.46.tar.gz

  • 创建目录

    mkdir -p /usr/share/java/

  • 拷贝至目录

    cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

  • 下载 mysql yum 文件

    wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm

  • yum -y install mysql57-community-release-el7-10.noarch.rpm

  • yum -y install mysql-community-server

  • 启动mysql

    systemctl start mysqld

  • 设置开机自启动

    systemctl enable mysqld

  • grep "password" /var/log/mysqld.log

    查看并记住密码
  • 登录mysql

    mysql -u root -p

    输入 刚刚看到的密码
  • 修改密码

    ALTER USER 'root'@'localhost' IDENTIFIED BY 'Qaz!@#$';

  • 创建 cloudera-manager-server 数据库

    CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

  • 创建 scm 库连接信息

    GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'Abc123!@#';

  • 刷新

    flush privileges;

  • 退出mysql

    exit;

十、

主节点

安装 cloudera manager-server(cdh-master)

  • 切换至文件下载目录
    yum -y install cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm
               
  • 将已下载的

    CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel

    manifest.json

    拷贝至

    /opt/cloudera/parcel-repo/

    目录
  • 生成 parcel 包的 sha 值:

    sha1sum /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel

  • 将 sha 值填入到

    CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha

    文件中:

    vi /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha

  • 启动 Cloudera Manager Server

    systemctl start cloudera-scm-server

十一、启动 cloudera-manager-agent(所有节点)

  • systemctl start cloudera-scm-agent

  • 查看主节点启动日志(cdh-master) cloudera-scm-server 启动比较慢

    tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

    看到下面的信息,说明启动完成

    INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

十二、登录 web

  • http://cdh-master:7180
  • 账号/密码:

    admin / admin

cdh

继续阅读