在windows上用eclipse远程运行hadoop上的wordcount程序出现的问题，求解决

WordCount源代码如下：

1. package org.apache.hadoop.examples;
2. import java.io.IOException;
3. import java.util.StringTokenizer;
4. import org.apache.hadoop.conf.Configuration;
5. import org.apache.hadoop.fs.Path;
6. import org.apache.hadoop.io.IntWritable;
7. import org.apache.hadoop.io.Text;
8. import org.apache.hadoop.mapreduce.Job;
9. import org.apache.hadoop.mapreduce.Mapper;
10. import org.apache.hadoop.mapreduce.Reducer;
11. import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
12. import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
13. import org.apache.hadoop.util.GenericOptionsParser;
14. public class WordCount {
15.   public static class TokenizerMapper 
16.        extends Mapper<Object, Text, Text, IntWritable>{
17.     private final static IntWritable one = new IntWritable(1);
18.     private Text word = new Text();
19.     public void map(Object key, Text value, Context context
20.                     ) throws IOException, InterruptedException {
21.       StringTokenizer itr = new StringTokenizer(value.toString());
22.       while (itr.hasMoreTokens()) {
23.         word.set(itr.nextToken());
24.         context.write(word, one);
25.       }
26.     }
27.   }
28.   public static class IntSumReducer 
29.        extends Reducer<Text,IntWritable,Text,IntWritable> {
30.     private IntWritable result = new IntWritable();
31.     public void reduce(Text key, Iterable<IntWritable> values, 
32.                        Context context
33.                        ) throws IOException, InterruptedException {
34.       int sum = 0;
35.       for (IntWritable val : values) {
36.         sum += val.get();
37.       }
38.       result.set(sum);
39.       context.write(key, result);
40.     }
41.   }
42.   public static void main(String[] args) throws Exception {
43.     Configuration conf = new Configuration();
44.     conf.set("mapred.job.tracker", "192.168.80.100:9001");
45.     String[] ars=new String[]{"in","newout"};
46.     String[] otherArgs = new GenericOptionsParser(conf, ars).getRemainingArgs();
47.     if (otherArgs.length != 2) {
48.       System.err.println("Usage: wordcount <in> <out>");
49.       System.exit(2);
50.     }
51.     Job job = new Job(conf, "wordcount");
52.     job.setJarByClass(WordCount.class);
53.     job.setMapperClass(TokenizerMapper.class);
54.     job.setCombinerClass(IntSumReducer.class);
55.     job.setReducerClass(IntSumReducer.class);
56.     job.setOutputKeyClass(Text.class);
57.     job.setOutputValueClass(IntWritable.class);
58.     FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
59.     FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
60.     System.exit(job.waitForCompletion(true) ? 0 : 1);
61.   }
62. }

运行run on hadoop后报错内容如下：

13/12/22 18:49:11 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).

13/12/22 18:49:11 INFO mapred.JobClient: Cleaning up the staging area hdfs://hadoop001:9000/usr/local/hadoop/tmp/mapred/staging/root/.staging/job_201312221708_0002

13/12/22 18:49:11 ERROR security.UserGroupInformation: PriviledgedActionException as:root cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/E:/hadoop/eclipse/workspace/WordCount/in

Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/E:/hadoop/eclipse/workspace/WordCount/in

at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)

at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)

at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1024)

at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1041)

at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)

at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)

at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Unknown Source)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)

at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)

at org.apache.hadoop.examples.WordCount.main(WordCount.java:69)

解决办法：

捣鼓了一整天终于找到解决的办法了，原来是我之前参考的《hadoop集群第七期》http://www.cnblogs.com/xia520pi/archive/2012/05/20/2510723.html

注意第52和53行，然后运行的时候老是出错，如前面所述的错误，然后百度了半天才知道是hadoop-core-1.1.2.jar这个包的问题，org.apache.hadoop.fs里面的一个类名为

FileUtl.class中的一个方法checkReturnValue搞的鬼，这个方法是检查Windows下文件权限问题，在Linux下可以正常运行，不存在这样的问题。

因此将此方法注释掉就OK了，但是为了方便，还是从网上重新找了hadoop-core-1.1.2.jar下载下来，将原先的hadoop-core-1.1.2.jar替换掉，然后再在eclipse里面运行

wordcount.java就成功了。

纪念一下：

总结：网上别人写的东西有好有坏，得学会亲自动手解决。这里我留下代码修改后的hadoop-core-1.1.2.jar的下载地址，有需要的童鞋可以下载替换原先的jar包试试，嘿嘿！

http://download.csdn.net/detail/qq_417174491/6755193