【Hadoop】 分布式Hadoop叢集安裝配置

第一次安裝hadoop 叢集，記錄其安裝過程！

1 前期準備，配置所有的機器的/etc/hosts 檔案

# cat /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1       rac localhost

::1             rac3 localhost

10.250.7.225    rac1

10.250.7.249    rac2

10.250.7.241    rac3

10.250.7.220    rac4 

1 擷取hadoop安裝檔案

可以從一下網址擷取hadoop 安裝檔案 hadoop-0.20.2.tar.gz：

http://apache.etoak.com//hadoop/common/ 

http://mirror.bjtu.edu.cn/apache//hadoop/common/ 

http://labs.renren.com/apache-mirror//hadoop/common/ 

#tar zxvf hadoop-0.20.2.tar.gz

#mv hadoop-0.20.2.tar.gz hadoop

#cd hadoop/conf

配置conf檔案夾裡面的core-site.xml,hdfs-site.xml,mapread-site.xml,mapred-site.xml，hadoop-env.sh。具體各個配置的含義請參考Hadoop幫助文檔。

core-site.xml檔案

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    fs.default.name

    hdfs://rac2:9000

<b>2.其次編輯所有節點的hdfs-site.xml，指令如下：</b>

#vi /root/hadoop/conf/hdfs-site.xml

 dfs.name.dir

 /opt/hadoop/NameData

 dfs.permissions

 false

 dfs.replication

 1

<b>3 編輯所有節點mapred-site.xml檔案：</b>

#vi /home/hadoop/conf/mapred-site.xml

mapred.job.tracker

192.168.0.10:9001

<b>4 在所有節點編輯hadoop-env.sh </b>

export HADOOP_HOME=/root/hadoop

export HADOOP_CONF_DIR=$HADOOP_HOME/conf 

export PATH=$PATH:$HADOOP_HOME/bin 

export JAVA_HOME=/usr/java/jdk1.6.0_29 

export CLASSHOME=/usr/java/jdk1.6.0_29/lib/tools.jar:/usr/java/jdk1.6.0_29/lib/dt.jar

<b>5 在所有節點修改masters ，slaves 檔案，其中masters 檔案寫入master 節點的ip，slaves 檔案寫入slaves 節點的ip</b>

[root@rac1 conf]#  cat masters 

10.250.7.225

[root@rac1 conf]# cat slaves 

10.250.7.220

10.250.7.249

10.250.7.241

<b>6 啟動hadoop</b>

[root@rac1 bin]# sh start-all.sh 

starting namenode, logging to /root/hadoop/logs/hadoop-root-namenode-rac1.out

10.250.7.220: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac4.out

10.250.7.241: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac3.out

10.250.7.249: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac2.out

10.250.7.225: starting secondarynamenode, logging to /root/hadoop/logs/hadoop-root-secondarynamenode-rac1.out

jobtracker running as process 20175. Stop it first.

10.250.7.220: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac4.out

10.250.7.241: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac3.out

10.250.7.249: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac2.out