第1节 MapReduce入门:11、mapreduce程序的入门-2
1.5、WordCount示例编写
详见代码
提醒:本地运行完成之后,就可以打成jar包放到服务器上面去运行了,实际工作当中,都是将代码打成jar包,开发main方法作为程序的入口,然后放到集群上面去运行。
打包步骤如下:
会产生2个jar包,一个是包含所有依赖jar包的jar包,一个是不含依赖包的jar包(03hdfs-1.0-SNAPSHOT.jar),我们使用不含依赖包的jar包(original-03hdfs-1.0-SNAPSHOT.jar):
改名为wordcount.jar,拷贝到node01节点/export/servers/路径下,然后执行jar包:
[[email protected] servers]# hadoop jar wordcount.jar cn.itcast.wordcount.JobMain
如下报错:
1.Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://node01:8020/wordcountout2 already exists
原因:目标文件夹的路径已存在。
2.19/06/12 19:47:18 INFO mapreduce.Job: Job job_1560328381126_0001 failed with state FAILED due to: Task failed task_1560328381126_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
查看jobhistory的报错日志为:
2019-06-12 19:47:15,657 WARN [uber-SubtaskRunner] org.apache.hadoop.mapred.LocalContainerLauncher: Exception running local (uberized) 'child' : java.lang.RuntimeException: java.lang.ClassNotFoundException: Class cn.itcast.wordcount.WordCountMapper not found
原因:jobMain.java中缺少一句话:job.setJarByClass(JobMain.class);