天天看点

Hadoop 中的datanode无法启动

问题描述:

-- ::, WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/opt/hdfs/data/
java.io.IOException: Incompatible clusterIDs in /opt/hdfs/data: namenode clusterID = CID-a768407-d698---fadd282d8631; datanode clusterID = CID-3667822c-ea1a-4c25-a685-5ff336eb4317
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:)
    at java.lang.Thread.run(Thread.java:)
-- ::, WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool ID needed, but service not yet registered with NN, trace:
java.lang.Exception
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.hasBlockPoolId(BPOfferService.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shouldRetryInit(BPOfferService.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.shouldRetryInit(BPServiceActor.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:)
    at java.lang.Thread.run(Thread.java:)
-- ::, ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid dc086d-c141-de7-a943-c8c97d635dac) service to masternode/: Exiting. 
java.io.IOException: All specified directories are failed to load.
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:)
    at java.lang.Thread.run(Thread.java:)
           

解决步骤:

1.namenode

more /opt/hdfs/name/current/VERSION 
复制:clusterID=CID-a768407-d698---fadd282d8631
           

2.datanode

vi /opt/hdfs/data/current/VERSION
粘贴:clusterID=CID-a768407-d698---fadd282d8631
           

然后重启即可。

jps显示正常

继续阅读