sqoop1.99.7 mysql->hdfs job执行报错

问题描述:

sqoop2.0版本,mysql到hdfs抽取数据执行job时报错,由于网上sqoop2的相关资料实在太少,这个问题困扰了我好久,所以贴出来给大家一起避坑

报错如下:

java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at org.apache.sqoop.job.mr.MRUtils.initConnectorClassLoaders(MRUtils.java:51)
at org.apache.sqoop.job.mr.SqoopNullOutputFormat$SqoopDestroyerOutputCommitter.setupJob(SqoopNullOutputFormat.java:64)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more
2021-04-06 12:16:15,551 FATAL [CommitterEvent Processor #0] org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[CommitterEvent Processor #0,5,main] threw an Error. Shutting down now...
java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at org.apache.sqoop.job.mr.MRUtils.initConnectorClassLoaders(MRUtils.java:51)
at org.apache.sqoop.job.mr.SqoopNullOutputFormat$SqoopDestroyerOutputCommitter.setupJob(SqoopNullOutputFormat.java:64)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:255)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more


解决方法:
将sqoop安装目录下的commons-lang-2.5.jar这个包(server shell tools这三个文件夹下的lib包里都有,随便找一个就行)拷贝到hadoop安装目录下的/share/hadoop/common/lib里面就解决了

我与你的问题一样,按上面的操作,还是没能解决,望回复。