无法在EC2上启动Spark的“start-all.sh”(rhel7)
问题描述:
我试图通过在EC2实例(RHEL 7)中触发/sbin/start-all.sh
来运行独立的Spark-2.1.1。每当它运行时,它会询问root @ localhost的密码,甚至难以给出正确的密码,它会引发我 - [email protected]'s password: localhost: Permission denied, please try again.
错误。无法在EC2上启动Spark的“start-all.sh”(rhel7)
不管这个错误,当我在控制台中点击jps
,我可以看到Master正在运行。
[email protected]# jps
27863 Master
28093 Jps
而且我查了日志,发现这个 -
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/06/12 15:36:15 INFO Master: Started daemon with process name: [email protected]
17/06/12 15:36:15 INFO SignalUtils: Registered signal handler for TERM
17/06/12 15:36:15 INFO SignalUtils: Registered signal handler for HUP
17/06/12 15:36:15 INFO SignalUtils: Registered signal handler for INT
17/06/12 15:36:15 WARN Utils: Your hostname, localhost.org.xxxxxxxxx.com resolves to a loopback address: 127.0.0.1; using localhost ip instead (on interface eth0)
17/06/12 15:36:15 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
17/06/12 15:36:16 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/06/12 15:36:16 INFO SecurityManager: Changing view acls to: root
17/06/12 15:36:16 INFO SecurityManager: Changing modify acls to: root
17/06/12 15:36:16 INFO SecurityManager: Changing view acls groups to:
17/06/12 15:36:16 INFO SecurityManager: Changing modify acls groups to:
17/06/12 15:36:16 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
17/06/12 15:36:16 INFO Utils: Successfully started service 'sparkMaster' on port 7077.
17/06/12 15:36:16 INFO Master: Starting Spark master at spark://localhost.org.xxxxxxxxx.com:7077
17/06/12 15:36:16 INFO Master: Running Spark version 2.1.1
17/06/12 15:36:16 INFO Utils: Successfully started service 'MasterUI' on port 8080.
17/06/12 15:36:16 INFO MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at http://localhost:8080
17/06/12 15:36:16 INFO Utils: Successfully started service on port 6066.
17/06/12 15:36:16 INFO StandaloneRestServer: Started REST server for submitting applications on port 6066
17/06/12 15:36:16 INFO Master: I have been elected leader! New state: ALIVE
我试图找出为什么我无法启动我的工作节点。有人可以帮我解决这个问题吗?谢谢。
答
检查您的主机名是否正确解析。 如果您使用的是localhost
,请确保它已在/etc/hosts
文件中解析。
让我知道这是否有帮助。干杯。
我实际上使用的是系统IP地址而不是'localhost'(只是因安全原因而重命名)。是的,主机名被正确解析,问题仍然存在。 – Panchu
的是,尝试在你的'的/ etc/hosts'文件解决'localhost',看看它的工作原理,这样的: '192.168.y.xxx \t hostname' '192.168.y.xxx \t localhost'。 如果确实如此,则必须对属性SPARK_LOCAL_IP的spark conf(spark-env.sh)进行更改。 –