的Hadoop - UnsupportedOperationException异常:未经DistributedFileSystem
问题描述:
我已经在Eclipse项目下面的代码,当我导出到一个jar文件并运行Hadoop的节点上的Hadoop - UnsupportedOperationException异常:未经DistributedFileSystem
String hdrsUrl = "hdfs://servername:8020";
Configuration conf = new Configuration();
conf.set("fs.defaultFS", hdrsUrl);
FileSystem fs = FileSystem.get(conf);
我收到以下错误
实施15/04/17 08:21:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.UnsupportedOperationException: Not implemented by the DistributedFileSystem FileSystem implementation
at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:216)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2564)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2574)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
at com.express.BrierleyProcessor.App.main(App.java:33)
... 5 more
虽然,当我在节点上创建一个java文件,编译它,设置类路径,并执行它的工作没有问题。
我错过了什么?
答
当运行下面的命令:
[[email protected] bin]$ hadoop version
Hadoop 2.6.0.2.2.0.0-2041
Subversion [email protected]:hortonworks/hadoop.git -r 7d56f02902b436d46efba030651a2fbe7c1cf1e9
Compiled by jenkins on 2014-11-19T19:42Z
Compiled with protoc 2.5.0
From source with checksum f0c0406cc910a79f206d2ee4c2a68773
This command was run using /usr/hdp/2.2.0.0-2041/hadoop/hadoop-common-2.6.0.2.2.0.0-2041.jar
所显示的版本号(2.6.0.2.2.0.0-2041)具有以下含义:
2.6.0 - hadoop的版本
2.2.0.0-2041 - HDP版本。
因此,你应该使用Hadoop的jar文件的2.2版本,而不是2.6版本
HDP 2.2.0对应的Hadoop 2.6.0。你能否澄清是什么导致你使用hadoop 2.2版? – thinkmassive