通過secondary namenode恢複崩潰的namenode

模拟namenode崩潰，将name目錄的内容全部删除，然後通過secondary namenode恢複namenode。

環境：os：centos 6.5 x64 & soft：hadoop 1.2.1

1、進入name目錄下，删除name目錄内容。

[huser@master name]$ pwd

/home/huser/hadoop/tmp/dfs/name

[huser@master name]$ ll

drwxrwxr-x 2 huser huser 4096 4月 16 20:16

current

drwxrwxr-x 2 huser huser 4096 4月 16 17:24 image

-rw-rw-r-- 1 huser

huser 0 4月 16 20:10 in_use.lock

drwxrwxr-x 2 huser huser 4096 4月 16 18:55

previous.checkpoint

[huser@master name]$ rm -r *

[huser@master name]$ ls

2、停止叢集，然後重新開機叢集，發現namenode失敗。

[huser@master hadoop-1.2.1]$ bin/stop-all.sh

[huser@master hadoop-1.2.1]$ bin/start-all.sh 

[huser@master

hadoop-1.2.1]$ jps

7160 secondarynamenode

7229 jobtracker

7369 jps

3、停止叢集格式化namenode。

[huser@master hadoop-1.2.1]$ bin/hadoop namenode -format

14/04/16 21:17:39

info namenode.namenode: startup_msg:

/************************************************************

startup_msg:

starting namenode

startup_msg: host = master/192.168.1.115

args = [-format]

startup_msg: version = 1.2.1

startup_msg: build =

https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152;

compiled by ‘mattf‘ on mon jul 22 15:23:09 pdt 2013

startup_msg: java =

1.7.0_51

************************************************************/

re-format

filesystem in /home/huser/hadoop/tmp/dfs/name ? (y or n) y

14/04/16 21:17:42

info util.gset: computing capacity for map blocksmap

14/04/16 21:17:42 info

util.gset: vm type = 64-bit

14/04/16 21:17:42 info util.gset: 2.0% max

memory = 1013645312

14/04/16 21:17:42 info util.gset: capacity = 2^21 =

2097152 entries

14/04/16 21:17:42 info util.gset: recommended=2097152,

actual=2097152

14/04/16 21:17:43 info namenode.fsnamesystem:

fsowner=huser

supergroup=supergroup

ispermissionenabled=true

dfs.block.invalidate.limit=100

isaccesstokenenabled=false accesskeyupdateinterval=0 min(s),

accesstokenlifetime=0 min(s)

14/04/16 21:17:43 info namenode.fseditlog:

dfs.namenode.edits.toleration.length = 0

14/04/16 21:17:43 info

namenode.namenode: caching file names occuring more than 10 times

14/04/16

21:17:43 info common.storage: image file

/home/huser/hadoop/tmp/dfs/name/current/fsimage of size 111 bytes saved in 0

seconds.

14/04/16 21:17:43 info namenode.fseditlog: closing edit log:

position=4, editlog=/home/huser/hadoop/tmp/dfs/name/current/edits

21:17:43 info namenode.fseditlog: close success: truncate to 4,

editlog=/home/huser/hadoop/tmp/dfs/name/current/edits

14/04/16 21:17:44 info

common.storage: storage directory /home/huser/hadoop/tmp/dfs/name has been

successfully formatted.

14/04/16 21:17:44 info namenode.namenode:

shutdown_msg:

shutting down namenode at

master/192.168.1.115

4、從datanode節點擷取namespace的id。

[huser@master hadoop-1.2.1]$ ssh slave1

[huser@slave1 current]$ pwd

/home/huser/hadoop/tmp/dfs/data/current

[huser@slave1 current]$ ll

-rw-rw-r-- 1 huser huser 49184 4月 16 18:43

blk_-1800088935645150399

-rw-rw-r-- 1 huser huser 395 4月 16 18:43

blk_-1800088935645150399_1013.meta

-rw-rw-r-- 1 huser huser 25 4月 16

18:43 blk_269963827714855400

-rw-rw-r-- 1 huser huser 11 4月 16 18:43

blk_269963827714855400_1014.meta

-rw-rw-r-- 1 huser huser 16353 4月 16 18:43

blk_4611281727215307463

-rw-rw-r-- 1 huser huser 135 4月 16 18:43

blk_4611281727215307463_1015.meta

-rw-rw-r-- 1 huser huser 769 4月 16 19:32

dncp_block_verification.log.curr

-rw-rw-r-- 1 huser huser 158 4月 16 19:51

version

[huser@slave1 current]$ cat version

#wed apr 16 19:51:23 cst

2014

namespaceid=589801292

storageid=ds-1065963269-192.168.1.111-50010-1397640950581

ctime=0

storagetype=data_node

layoutversion=-41

5、修改namenode的version檔案中namespaceid。

[huser@slave1 current]$ exit

logout

[huser@master current]$ pwd

/home/huser/hadoop/tmp/dfs/name/current

[huser@master current]$ vi version

#wed apr 16 21:17:43 cst

storagetype=name_node

6、删除namenode節點下的fsinage檔案。

[huser@master current]$ rm fsimage

[huser@master current]$

ll

-rw-rw-r-- 1 huser huser 4 4月 16 21:17 edits

huser 8 4月 16 21:17 fstime

-rw-rw-r-- 1 huser huser 100 4月 16 21:30

7、複制secondarynamenode節點的fsimage檔案到namenode節點下。

pwd

/home/huser/hadoop/tmp/dfs/namesecondary/current

current]$ ll

-rw-rw-r-- 1 huser huser 4 4月 16 20:16 edits

-rw-rw-r-- 1

huser huser 2259 4月 16 20:16 fsimage

-rw-rw-r-- 1 huser huser 8 4月 16

20:16 fstime

-rw-rw-r-- 1 huser huser 100 4月 16 20:16 version

[huser@master current]$ cp fsimage

/home/huser/hadoop/tmp/dfs/name/current/

[huser@master current]$ cd

-rw-rw-r-- 1 huser huser 4 4月 16 21:17 edits

huser 2259 4月 16 21:37 fsimage

-rw-rw-r-- 1 huser huser 8 4月 16 21:17

fstime

-rw-rw-r-- 1 huser huser 100 4月 16 21:30 version

8、重新開機叢集并檢查運作情況。

[huser@master hadoop-1.2.1]$ jps

7927 secondarynamenode

7773

namenode

8017 jobtracker

8123 jps