Hadoop運維

簡單記錄幾個hdfs的運維指令

//檢視hdfs的狀态，是否有missing block，corrupt block等，也可以看datanode的狀态
hdfs dfsadmin -report      

//檢視hdfs根目錄下是否有檔案處于missing,currupt狀态，而且不是under replica的
hadoop fsck / | egrep -v '^\.+$' | grep -v eplica      

//檢視某個檔案中，包含的block
hadoop fsck /path/to/corrupt/file -locations -blocks -files      

送出一個hadoop wordcount作業，在mapreduce v1中

ssh <gateway_host>
find / -name hadoop-*-examples.jar
touch input
cat a>>input
cat b>>input
hadoop fs -put input /tmp/input
hadoop jar  /<find-dir>/hadoop-mapreduce-examples.jar wordcount /tmp/input /tmp/output       

E-mail: [email protected]

https://www.linkedin.com/in/huahuiyang/