天天看點

CENTOS6.5安裝CDH5.12.1(一)

溫馨提示:要看高清無碼套圖,請使用手機打開并單擊圖檔放大檢視。

1.概述

本文檔主要描述離線環境下,CentOS6.5作業系統部署CDH5.12.1企業版的過程。本安裝文檔主要分為4個步驟:

1.前期準備(包括hostname、關閉防火牆、關閉SELinux、時鐘同步等)

2.安裝Cloudera Manager Server

3.安裝CDH叢集

4.叢集完整性檢查(包括HDFS檔案系統、MapReduce、Hive等服務是否可以正常運作)

這篇文檔将着重介紹Cloudera管理器與CDH的安裝,并基于以下假設:

1.作業系統版本:CentOS6.5

2.MySQL版本:5.1.73

3.CM版本:CM 5.12.1

4.CDH版本:CDH 5.12.1

5.采用root對叢集進行部署

6.您已經下載下傳CDH和CM的安裝包(CDH使用parcels包方式安裝)

2.前期準備

2.1hostname及hosts配置

叢集中各個節點之間能互相通信使用靜态IP位址。IP位址和主機名通過/etc/hosts配置,主機名/etc/hostname進行配置(推薦hostname使用FQDN完全域名的方式配置)。

以伺服器(172.31.6.148)為例:

  • hostname配置

修改/etc/sysconfig/network檔案:

[root@ip-172-31-6-148~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=ip-172-31-6-148.fayson.com

CENTOS6.5安裝CDH5.12.1(一)

注意:修改hostname後需要重新開機伺服器。

檢查hostname是否修改成功:

[root@ip-172-31-6-148~]# hostnameip-172-31-6-148.fayson.com

[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • hosts配置

/etc/hosts檔案如下:

[root@ip-172-31-6-148~]# more /etc/hosts127.0.0.1               localhost.localdomain localhost::1             localhost6.localdomain6 localhost6172.31.6.148  ip-172-31-6-148.fayson.com172.31.5.190  ip-172-31-5-190.fayson.com172.31.10.118 ip-172-31-10-118.fayson.com172.31.9.33   ip-172-31-9-33.fayson.com[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)

以上兩步操作,在叢集中其它節點做相應配置。

2.2禁用SELinux

在所有節點執行sudo setenforce 0 指令

[root@ip-172-31-6-148~]# sudo setenforce 0

修改叢集所有節點的/etc/selinux/config檔案,内容如下:

[root@ip-172-31-6-148~]# vim /etc/selinux/config # This file controls the state ofSELinux on the system.

# SELINUX= can take one of these three values:

#     enforcing - SELinux security policyis enforced.

#     permissive - SELinux printswarnings instead of enforcing.

#     disabled - No SELinux policy isloaded.SELINUX=disabled# SELINUXTYPE= can take one of these twovalues:

#     targeted - Targeted processes areprotected,

#     mls - Multi Level Securityprotection.SELINUXTYPE=targeted

CENTOS6.5安裝CDH5.12.1(一)

2.3關閉防火牆

在叢集所有節點執行如下操作,并永久關閉防火牆

[root@ip-172-31-6-148~]# service iptables stopiptables: Setting chains to policy ACCEPT: filter          [  OK  ]iptables: Flushing firewall rules:                         [  OK  ]iptables: Unloading modules:                               [  OK  ][root@ip-172-31-6-148 ~]# chkconfig iptables off[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)

2.4配置作業系統本地yum源

  • 挂載作業系統iso檔案

[root@ip-172-31-6-148~]# mkdir /mnt/iso[root@ip-172-31-6-148 ~]# mount -o loop CentOS-6.5-x86_64-minimal.iso/mnt/iso/[root@ip-172-31-6-148 ~]# df -hFilesystem                           Size  UsedAvail Use% Mounted on/dev/xvde                             99G 3.4G   91G   4% /tmpfs                                7.4G     0  7.4G   0%/dev/shm/root/CentOS-6.5-x86_64-minimal.iso  398M  398M    0 100% /mnt/iso[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 配置作業系統repo

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo[localrepo]name=CentOS6.5

baseurl=file:///mnt/iso

gpgcheck=false

enabled=true

CENTOS6.5安裝CDH5.12.1(一)

檢視yum源清單是否有建立的localrepo

[root@ip-172-31-6-148~]# yum repolistLoaded plugins: fastestmirror, prestoLoading mirror speeds from cached hostfile

 * base: mirror.0x.sg

 * extras: mirror.0x.sg

 * updates: mirror.0x.sglocalrepo                                                                                     | 3.6 kB     00:00... localrepo/primary_db                                                                          | 492 kB    00:00 ... repo id                                         reponame                                                       statusbase                                            CentOS-6 - Base                                                6,706cloudera-manager                                Cloudera Manager, Version 5.12.0                                    7cmrepo                                          cm_repo                                                            7extras                                          CentOS-6 - Extras                                                 45localrepo                                       CentOS6.5                                                        248updates                                         CentOS-6 - Updates                                               589repolist: 7,602[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)

2.5安裝http服務

  • 使用yum安裝http服務

在叢集其中一節點上安裝http服務,執行如下指令:

[root@ip-172-31-6-148~]# yum -y install httpd

CENTOS6.5安裝CDH5.12.1(一)

如上圖所示則表明安裝成功。

  • 将httpd服務加入系統自啟動服務并設定開機啟動

[root@ip-172-31-6-148~]# chkconfig --add httpd[root@ip-172-31-6-148 ~]# chkconfig httpd on[root@ip-172-31-6-148 ~]# chkconfig --list |grep httpdhttpd           0:off   1:off   2:on   3:on    4:on    5:on   6:off[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 啟動http服務

[root@ip-172-31-6-148~]# service httpd startStarting httpd:                                            [  OK  ]

CENTOS6.5安裝CDH5.12.1(一)

2.6重做作業系統yum源

在2.5安裝了httpd服務後,重新制作作業系統的yum源,使用http的方式,操作如下:

  • 在/var/www/html目錄下建立centos6.5目錄

[root@ip-172-31-6-148html]# mkdir centos6.5[root@ip-172-31-6-148 html]# lltotal 4drwxr-xr-x 2 rootroot 4096 Sep  4 16:04 centos6.5[root@ip-172-31-6-148 html]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 将/mnt/iso目錄下的所有檔案拷貝至/var/www/html/centos6.5目錄下

[root@ip-172-31-6-148html]# scp -r /mnt/iso/*/var/www/html/centos6.5/

CENTOS6.5安裝CDH5.12.1(一)
  • 修改/etc/yum.repo.d/os.repo配置

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo [localrepo]name=CentOS6.5

baseurl=http://ip-172-31-6-148.fayson.com/centos6.5

gpgcheck=false

enabled=true

CENTOS6.5安裝CDH5.12.1(一)
  • 檢查yum源是否配置正常
CENTOS6.5安裝CDH5.12.1(一)

将/etc/yum.repo.d/os.repo配置檔案同步到叢集所有節點的/etc/yum.repo.d/目錄下

2.7叢集時鐘同步

在叢集的所有伺服器上安裝ntp服務,用于叢集時鐘同步,此處選擇ip-172-31-6-148.fayson.com伺服器作為本地NTP伺服器,叢集其它節點與其保持同步,配置如下:

  • 在叢集所有節點安裝ntp服務,指令如下

[root@ip-172-31-6-148~]# yum -y install ntp

CENTOS6.5安裝CDH5.12.1(一)

如上圖顯示,則說明安裝成功。

  • 将ntpd加入系統自啟動服務并設定開機啟動

在所有節點執行如下操作:

[root@ip-172-31-6-148~]# chkconfig --add ntpd[root@ip-172-31-6-148 ~]# chkconfig ntpd on

CENTOS6.5安裝CDH5.12.1(一)
  • 在ip-172-31-6-148.fayson.com上配置與自己同步

[root@ip-172-31-6-148~]# vim /etc/ntp.conf ...# Use public servers from thepool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

#server 0.centos.pool.ntp.org iburst

#server 1.centos.pool.ntp.org iburst

#server 2.centos.pool.ntp.org iburst

#server 3.centos.pool.ntp.org iburstserver  127.127.1.0     #local clockfudge   127.127.1.0 stratum 10

CENTOS6.5安裝CDH5.12.1(一)
  • 叢集其它節點配置如下

[root@ip-172-31-5-190~]# vim /etc/ntp.conf ...# Use public servers from thepool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

#server 0.centos.pool.ntp.org iburst

#server 1.centos.pool.ntp.org iburst

#server 2.centos.pool.ntp.org iburst

#server 3.centos.pool.ntp.org iburstserver 172.31.6.148 iburst

CENTOS6.5安裝CDH5.12.1(一)
  • 重新開機叢集所有節點的ntpd服務

[root@ip-172-31-6-148~]# service ntpd restartShutting down ntpd:                                        [  OK  ]Starting ntpd:                                             [  OK  ][root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 驗證時鐘同步,在所有節點執行ntpq -p指令

[root@ip-172-31-6-148~]# ntpq -p

     remote           refid      stt when poll reach   delay   offset jitter

==============================================================================*LOCAL(0)        .LOCL.         10 l   15   64    7    0.000   0.000   0.001[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txtip-172-31-5-190.fayson.com "ntpq-p"

     remote           refid      st t when pollreach   delay   offset jitter

==============================================================================*ip-172-31-6-148 LOCAL(0)        11 u   68  128 377    0.274    1.005   0.243[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txtip-172-31-10-118.fayson.com "ntpq-p"              

     remote           refid      stt when poll reach   delay   offset jitter

==============================================================================*ip-172-31-6-148 LOCAL(0)        11 u  75 128  377    0.285    1.214  0.213[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txt ip-172-31-9-33.fayson.com"ntpq -p"               

     remote           refid      stt when poll reach   delay   offset jitter

==============================================================================*ip-172-31-6-148 LOCAL(0)        11 u    1   64 367    0.294   -9.249   1.511[root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)

注:标記部分,有“*”顯示則表示同步成功。

2.8安裝MySQL

  • 安裝MySQL

[root@ip-172-31-6-148~]# yum -y install mysql mysql-server

CENTOS6.5安裝CDH5.12.1(一)

如上截圖則表示MySQL安裝成功。

  • 将mysqld加入系統自啟動服務并設定開機啟動

[root@ip-172-31-6-148~]# chkconfig --add mysqld[root@ip-172-31-6-148 ~]# chkconfig mysqld on

  • 啟動并配置Mysql

[root@ip-172-31-6-148~]# service mysqld start...

                                                          [ OK  ]Starting mysqld:                                           [  OK  ]

CENTOS6.5安裝CDH5.12.1(一)
  • 初始化MySQL

[root@ip-172-31-6-148~]# mysql_secure_installation NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL

      SERVERS IN PRODUCTIONUSE!  PLEASE READ EACH STEP CAREFULLY!In order to log into MySQL to secure it, we'll need the current

password for the root user.  If you've just installed MySQL, andyou haven't set the root password yet, the passwordwill be blank,

so you should just press enter here.

Enter current password for root (enter for none):

OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL

root user without the proper authorisation.

Set root password? [Y/n] yNew password:

Re-enter new password:

Password updated successfully!

Reloading privilege tables..

 ... Success!

By default, a MySQL installation has an anonymous user, allowing anyone

to log into MySQL without having to have a user account created for

them.  This is intended only for testing,and to make the installation

go a bit smoother.  You should removethem before moving into a

production environment.

Remove anonymous users? [Y/n] y ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] n ... skipping.

By default, MySQL comes with a database named 'test' that anyone can

access.  This is also intended only fortesting, and should be removed

before moving into a production environment.

Remove test database and access to it? [Y/n] y - Dropping test database...

 ... Success!

 - Removing privileges on test database...

 ... Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? [Y/n] y ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MySQLinstallation should now be secure.Thanks for usingMySQL!

[root@ip-172-31-6-148 ~]# 

注意:标紅部分為輸入項,出現“Thanks for using MySQL!”則表示初始化成功。

  • 建立CM及CDH服務的資料庫

建庫語句:

create database metastore default character set utf8;CREATE USER 'hive'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';FLUSH PRIVILEGES;create database cm default character set utf8;CREATE USER 'cm'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';FLUSH PRIVILEGES;create database am default character set utf8;CREATE USER 'am'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON am. * TO 'am'@'%';FLUSH PRIVILEGES;create database rm default character set utf8;CREATE USER 'rm'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';FLUSH PRIVILEGES;create database hue default character set utf8;CREATE USER 'hue'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';FLUSH PRIVILEGES;create database oozie default character set utf8;CREATE USER 'oozie'@'%' IDENTIFIED BY 'password';GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';FLUSH PRIVILEGES;

指令行操作如下:

[root@ip-172-31-6-148~]# mysql -uroot -pEnter password: ...

mysql> create database metastore default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'hive'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql> mysql> create databasecm default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'cm'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql> create databaseam default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'am'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON am. * TO 'am'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql> mysql> create databaserm default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'rm'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql> mysql> create databasehue default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'hue'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql> mysql> create databaseoozie default character set utf8;Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'oozie'@'%' IDENTIFIEDBY 'password';Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

mysql>

  • 安裝jdbc驅動

[root@ip-172-31-6-148~]# mkdir -p /usr/share/java[root@ip-172-31-6-148 ~]# mv mysql-connector-java-5.1.34.jar/usr/share/java/[root@ip-172-31-6-148 ~]# cd /usr/share/java/[root@ip-172-31-6-148 java]# ln -s mysql-connector-java-5.1.34.jar mysql-connector-java.jar         [root@ip-172-31-6-148 java]# lltotal 940-rwxr-xr-x. 1 root root 960372 Aug 30 15:16 mysql-connector-java-5.1.34.jarlrwxrwxrwx  1root root     31 Sep 4 16:32 mysql-connector-java.jar -> mysql-connector-java-5.1.34.jar[root@ip-172-31-6-148 java]# 

CENTOS6.5安裝CDH5.12.1(一)

3.Cloudera Manager安裝

3.1配置CM本地repo源

CM5.12.1下載下傳位址:

​​http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm​​http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/jdk-6u31-linux-amd64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

  • 将以上7個檔案下載下傳至伺服器的/var/www/html/cm5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cm5.12.1[root@ip-172-31-6-148 ~]# cd /var/www/html/cm5.12.1/[root@ip-172-31-6-148 cm5.12.1]# lltotal 942088-rw-r--r-- 1 rootroot   9676820 Aug 30 14:45cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm-rw-r--r-- 1 rootroot 709818804 Aug 30 14:45 cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm-rw-r--r-- 1 rootroot      8692 Aug 30 14:45cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm-rw-r--r-- 1 rootroot     10600 Aug 30 14:45cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm-rw-r--r-- 1 rootroot  31918952 Aug 30 14:45enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm-rw-r--r-- 1 rootroot  71204325 Aug 30 14:45jdk-6u31-linux-amd64.rpm-rw-r--r-- 1 rootroot 142039186 Aug 30 14:45 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm[root@ip-172-31-6-148 cm5.12.1]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 在/var/www/html/cm5.12.1目錄下執行指令

[root@ip-172-31-6-148 cm5.12.1]# pwd/var/www/html/cm5.12.1

[root@ip-172-31-6-148 cm5.12.1]#createrepo .Spawning worker 0 with 7 pkgsWorkers FinishedGathering worker resultsSaving Primary metadataSaving file lists metadataSaving other metadataGenerating sqlite DBsSqlite DBs complete[root@ip-172-31-6-148 cm5.12.1]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 确認http是否能正常通路
CENTOS6.5安裝CDH5.12.1(一)
  • 在/etc/yum.repo.d/目錄下增加cm.repo檔案,内容如下

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/cm.repo[cmrepo]name=CM5.12.1

baseurl=http://ip-172-31-6-148.fayson.com/cm5.12.1

gpgcheck=false

enabled=true

CENTOS6.5安裝CDH5.12.1(一)
  • 驗證CM源是否配置成功

[root@ip-172-31-6-148~]# yum repolist...

cmrepo                                             CM5.12.1                                                        7localrepo                                          CentOS6.5                                                     248

CENTOS6.5安裝CDH5.12.1(一)
  • 驗證安裝JDK

[root@ip-172-31-6-148 ~]#yum -y install oracle-j2sdk1.7-1.7.0+update67-1

3.2配置http通路CDH Parcel包

CDH5.12.1下載下傳位址:

​​http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel​​http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1

http://archive.cloudera.com/cdh5/parcels/5.12.1/manifest.json

  • 将以上3個檔案下載下傳至/var/www/html/cdh5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cdh5.12.1[root@ip-172-31-6-148 ~]# cd /var/www/html/cdh5.12.1/[root@ip-172-31-6-148 cdh5.12.1]# lltotal 1582508-rw-r--r-- 1 rootroot 1620405492 Aug 30 19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel-rw-r--r-- 1 rootroot         41 Aug 30 19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1-rw-r--r-- 1 rootroot      72312 Aug 30 19:11 manifest.json[root@ip-172-31-6-148 cdh5.12.1]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 确認http是否能正常通路
CENTOS6.5安裝CDH5.12.1(一)

3.3安裝Cloudera Manager Server

  • 通過yum安裝ClouderaManager Server

[root@ip-172-31-6-148~]# yum -y installcloudera-manager-server

CENTOS6.5安裝CDH5.12.1(一)

如上圖所示則表明安裝成功。

  • 初始化CM資料庫

[root@ip-172-31-6-148~]#/usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm password

JAVA_HOME=/usr/java/jdk1.7.0_67-clouderaVerifying that we can write to /etc/cloudera-scm-serverCreating SCM configuration file in /etc/cloudera-scm-serverExecuting:  /usr/java/jdk1.7.0_67-cloudera/bin/java-cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/*com.cloudera.enterprise.dbutil.DbCommandExecutor/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.[                          main] DbCommandExecutor              INFO  Successfully connected to database.

All done, your SCM database is configured correctly![root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 啟動ClouderaManager Server

[root@ip-172-31-6-148~]# service cloudera-scm-server startStarting cloudera-scm-server:                              [  OK  ][root@ip-172-31-6-148 ~]#

  • 檢查端口是否監聽

[root@ip-172-31-6-148~]# netstat -lnpt | grep 7180tcp        0      0 0.0.0.0:7180               0.0.0.0:*                  LISTEN      4056/java           [root@ip-172-31-6-148 ~]# 

CENTOS6.5安裝CDH5.12.1(一)
  • 通過http:// 52.221.194.202:7180/cmf/login通路CM控制台
CENTOS6.5安裝CDH5.12.1(一)

4.CDH安裝

4.1CDH叢集安裝向導

1.admin/admin登入到CM

2.同意license協定,點選繼續

CENTOS6.5安裝CDH5.12.1(一)

3.選擇60試用,點選繼續

CENTOS6.5安裝CDH5.12.1(一)

4.點選“繼續”

CENTOS6.5安裝CDH5.12.1(一)

5.輸入主機ip或者名稱,點選搜尋找到主機後點選繼續

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

6.點選“繼續”

CENTOS6.5安裝CDH5.12.1(一)

7.使用parcel選項,點選“更多選項”,點選“-”删除其它所有位址,輸入

http:// ip-172-31-6-148.fayson.com/cdh5.12.1/點選“儲存更改”

CENTOS6.5安裝CDH5.12.1(一)

8.選擇自定義存儲庫,輸入cm的http位址

CENTOS6.5安裝CDH5.12.1(一)

9.點選“繼續”,進入下一步安裝jdk

CENTOS6.5安裝CDH5.12.1(一)

10.點選“繼續”,進入下一步,預設多使用者模式

CENTOS6.5安裝CDH5.12.1(一)

11.點選“繼續”,進入下一步配置ssh賬号密碼

CENTOS6.5安裝CDH5.12.1(一)

12.點選“繼續”,進入下一步,安裝Cloudera Manager相關到各個節點

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

13.點選“繼續”,進入下一步安裝cdh到各個節點

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

14.點選“繼續”,進入下一步主機檢查,確定所有檢查項均通過

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

點選完成進入服務安裝向導。

4.2叢集設定安裝向導

1.選擇需要安裝的服務

此處使用自定義服務,如下圖

CENTOS6.5安裝CDH5.12.1(一)

2.點選“繼續”,進入叢集角色配置設定

HDFS角色配置設定:

CENTOS6.5安裝CDH5.12.1(一)

Hive角色配置設定:

CENTOS6.5安裝CDH5.12.1(一)

Cloudera Manager Service角色配置設定:

CENTOS6.5安裝CDH5.12.1(一)

Spark角色配置設定:(Spark on Yarn是以沒有spark的master和worker角色)

CENTOS6.5安裝CDH5.12.1(一)

Yarn角色配置設定:

CENTOS6.5安裝CDH5.12.1(一)

Zookeeper角色配置設定:(至少3個Server)

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

3.角色配置設定完成點選“繼續”,進入下一步,測試資料庫連接配接

CENTOS6.5安裝CDH5.12.1(一)

4.測試成功,點選“繼續”,進入目錄設定,此處使用預設預設目錄,根據實際情況進行目錄修改

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

5.點選“繼續”,等待服務啟動成功

CENTOS6.5安裝CDH5.12.1(一)
CENTOS6.5安裝CDH5.12.1(一)

6.點選“繼續”,顯示叢集安裝成功

CENTOS6.5安裝CDH5.12.1(一)

7.安裝成功後進入home管理界面

CENTOS6.5安裝CDH5.12.1(一)

醉酒鞭名馬,少年多浮誇! 嶺南浣溪沙,嘔吐酒肆下!摯友不肯放,資料玩的花!

溫馨提示:要看高清無碼套圖,請使用手機打開并單擊圖檔放大檢視。

歡迎關注Hadoop實操,第一時間,分享更多Hadoop幹貨,喜歡請關注分享。

繼續閱讀