hadoop示例没有运行在amazon ec2
问题描述:
我在3个Ubuntu 12.10实例,1个主服务器和2个从服务器的amazon ec2上使用hadoop-1.0.4,就在〜目录下。 现在start-all.sh和stop-all.sh可以,但是当我在master或slave上运行jps时,它不会打印任何东西。然后我测试的hadoop例子:hadoop示例没有运行在amazon ec2
~/hadoop$ bin/hadoop jar hadoop-examples-1.0.4.jar pi 10 10000
它显示
Exception in thread "main" java.io.IOException: Permission denied
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createTempFile(File.java:1879)
at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
但是我搭配chmod 777 -R TMP TMP的文件夹。
~/hadoop$ sudo bin/hadoop jar hadoop-examples-1.0.4.jar pi 10 10000
使用sudo,它产生
13/05/12 03:58:11 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated.
Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to
override properties of core-default.xml, mapred-default.xml
and hdfs-default.xml respectively
Number of Maps = 10
Samples per Map = 10000
13/05/12 03:58:12 WARN fs.FileSystem: "54.235.101.85:50001" is a deprecated
filesystem name. Use "hdfs://54.235.101.85:50001/" instead.
13/05/12 03:58:13 INFO ipc.Client: Retrying connect to server:
hdmaster/54.235.101.85:50001. Already tried 0 time(s).
13/05/12 03:58:14 INFO ipc.Client: Retrying connect to server:
hdmaster/54.235.101.85:50001. Already tried 1 time(s).
13/05/12 03:58:15 INFO ipc.Client: Retrying connect to server:
hdmaster/54.235.101.85:50001. Already tried 2 time(s).
然后连接失败。那么问题是什么?我应该把sudo运行的例子吗?非常感谢。
答
我想,问题是,54.235.101.85是假设是一个公共IP地址。在所有节点中使用ifconfig
以获取IP地址列表并检查以10.x.x.x/172.x.x.x/192.x.x.x开头的IP。如果发现任何错误,请相应地修改所有节点中的配置文件。
+0
太棒了!非常感谢你! – user1261718 2013-05-13 21:59:01
是54.235.101.85可访问吗? – 2013-05-13 14:39:55
@JtheRocker现在不再需要sudo了,我修正了一些拼写错误的端口号,问题仍然是java.net.BindException:问题绑定到hdmaster/54.235.101.85:50000:无法分配请求的地址。我刚刚从主服务器中删除了/ tmp中的文件,并且jps和secondarynamenode对jps做出了响应,但仍然没有名称节点和数据节点。 – user1261718 2013-05-13 16:59:41
您是否尝试禁用防火墙? – 2013-05-13 17:19:49