Hadoop2.5+HA+zk3.4.6叢集搭建

部署分布圖

1.core-site.xml
<configuration>
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://bjsxt</value>
</property>
<property>
   <name>ha.zookeeper.quorum</name>
   <value>hadoop1:2181,hadoop2:2181,hadoop3:2181</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/opt/hadoop</value>
</property>
</configuration>
           

2.hdfs-site.xml
<configuration>
<property>
  <name>dfs.nameservices</name>
  <value>bjsxt</value>
</property>
<property>
  <name>dfs.ha.namenodes.bjsxt</name>
  <value>nn1,nn2</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.bjsxt.nn1</name>
  <value>hadoop1:8020</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.bjsxt.nn2</name>
  <value>hadoop2:8020</value>
</property>
<property>
  <name>dfs.namenode.http-address.bjsxt.nn1</name>
  <value>hadoop1:50070</value>
</property>
<property>
  <name>dfs.namenode.http-address.bjsxt.nn2</name>
  <value>hadoop2:50070</value>
</property>
<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://hadoop2:8485;hadoop3:8485;hadoop4:8485/bjsxt</value
>
</property>
<property>
  <name>dfs.client.failover.proxy.provider.bjsxt</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverPr
oxyProvider</value>
</property>
<property>
  <name>dfs.ha.fencing.methods</name>
  <value>sshfence</value>
</property>
<property>
  <name>dfs.ha.fencing.ssh.private-key-files</name>
  <value>/root/.ssh/id_dsa</value>
</property>
<property>
  <name>dfs.journalnode.edits.dir</name>
  <value>/opt/hadoop/data</value>
</property>
<property>
   <name>dfs.ha.automatic-failover.enabled</name>
   <value>true</value>
</property>
</configuration>
           

準備zookeeper
a)三台zookeeper：hadoop1，hadoop2，hadoop3
b)編輯zoo.cfg配置檔案
i.修改dataDir=/opt/zookeeper
ii.server.1=hadoop1::
server.2=hadoop2::
server.3=hadoop3::
c)在dataDir目錄中建立一個myid的檔案，檔案内容為，，
           

4.配置hadoop中的slaves
5.啟動三個zookeeper：./zkServer.sh start
啟動三個JournalNode：./hadoop-daemon.sh start journalnode
在其中一個namenode上格式化：hdfs namenode -format
           

把剛剛格式化之後的中繼資料拷貝到另外一個namenode上
a)啟動剛剛格式化的namenode
b)在沒有格式化的namenode上執行：hdfs namenode –bootstrapStandby
c)啟動第二個namenode
在其中一個namenode上初始化zkfc：hdfs zkfc –formatZK
hdfs zkfc -formatZK
停止上面節點：stop-dfs.sh
全面啟動：start-dfs.sh