spark 启动
总用量 0
[[email protected] logs]$ kill -9 32377[[email protected] logs]$ jps
29777 DFSZKFailoverController
29362 NameNode
29075 JournalNode
32547 Jps
31837 ResourceManager
[[email protected] logs]$ start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/logs/spark-wangshumin-org.apache.spark.deploy.master.Master-1-CentOSNode1.out
[[email protected] logs]$ ll
总用量 4
-rw-rw-r--. 1 wangshumin wangshumin 1534 3月 11 17:56 spark-wangshumin-org.apache.spark.deploy.master.Master-1-CentOSNode1.out
[[email protected] logs]$ cat spark-wangshumin-org.apache.spark.deploy.master.Master-1-CentOSNode1.out
Spark Command: /home/wangshumin/ekl/jdk1.8.0_162/bin/java -cp /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/conf/:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/spark-assembly-1.6.2-hadoop2.6.0.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/home/wangshumin/hadoop-2.6.5/etc/hadoop/ -Xms1g -Xmx1g org.apache.spark.deploy.master.Master --ip CentOSNode1 --port 7077 --webui-port 8080
========================================
18/03/11 17:56:08 INFO master.Master: Registered signal handlers for [TERM, HUP, INT]
18/03/11 17:56:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/03/11 17:56:10 INFO spark.SecurityManager: Changing view acls to: wangshumin
18/03/11 17:56:10 INFO spark.SecurityManager: Changing modify acls to: wangshumin
18/03/11 17:56:10 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(wangshumin); users with modify permissions: Set(wangshumin)
18/03/11 17:56:12 INFO util.Utils: Successfully started service 'sparkMaster' on port 7077.
18/03/11 17:56:12 INFO master.Master: Starting Spark master at spark://CentOSNode1:7077
18/03/11 17:56:12 INFO master.Master: Running Spark version 1.6.2
18/03/11 17:56:12 INFO server.Server: jetty-8.y.z-SNAPSHOT
18/03/11 17:56:12 INFO server.AbstractConnector: Started [email protected]:8080
18/03/11 17:56:12 INFO util.Utils: Successfully started service 'MasterUI' on port 8080.
18/03/11 17:56:12 INFO ui.MasterWebUI: Started MasterWebUI at http://192.168.72.141:8080
18/03/11 17:56:12 INFO server.Server: jetty-8.y.z-SNAPSHOT
18/03/11 17:56:12 INFO server.AbstractConnector: Started [email protected]:6066
18/03/11 17:56:12 INFO util.Utils: Successfully started service on port 6066.
18/03/11 17:56:12 INFO rest.StandaloneRestServer: Started REST server for submitting applications on port 6066
18/03/11 17:56:13 INFO master.Master: I have been elected leader! New state: ALIVE
[[email protected] logs]$
drwxr-xr-x. 13 wangshumin wangshumin 222 3月 11 09:36 spark-1.6.2-bin-hadoop2.6
[[email protected] spark]$ cd spark-1.6.2-bin-hadoop2.6/[[email protected] spark-1.6.2-bin-hadoop2.6]$ ll
总用量 1384
drwxr-xr-x. 2 wangshumin wangshumin 4096 3月 11 09:12 bin
-rw-r--r--. 1 wangshumin wangshumin 1343562 3月 11 09:12 CHANGES.txt
drwxr-xr-x. 2 wangshumin wangshumin 264 3月 11 13:15 conf
drwxr-xr-x. 3 wangshumin wangshumin 19 3月 11 09:12 data
drwxr-xr-x. 3 wangshumin wangshumin 79 3月 11 09:12 ec2
drwxr-xr-x. 3 wangshumin wangshumin 17 3月 11 09:12 examples
drwxr-xr-x. 2 wangshumin wangshumin 237 3月 11 09:16 lib
-rw-r--r--. 1 wangshumin wangshumin 17352 3月 11 09:12 LICENSE
drwxr-xr-x. 2 wangshumin wangshumin 4096 3月 11 09:12 licenses
drwxrwxr-x. 2 wangshumin wangshumin 4096 3月 11 12:11 logs
-rw-r--r--. 1 wangshumin wangshumin 23529 3月 11 09:12 NOTICE
drwxr-xr-x. 6 wangshumin wangshumin 119 3月 11 09:12 python
drwxr-xr-x. 3 wangshumin wangshumin 17 3月 11 09:12 R
-rw-r--r--. 1 wangshumin wangshumin 3359 3月 11 09:16 README.md
-rw-r--r--. 1 wangshumin wangshumin 120 3月 11 09:12 RELEASE
drwxr-xr-x. 2 wangshumin wangshumin 4096 3月 11 09:12 sbin
[[email protected] spark-1.6.2-bin-hadoop2.6]$ ./sbin/start-slave.sh spark://CentOSNode1:7077
starting org.apache.spark.deploy.worker.Worker, logging to /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/logs/spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out
failed to launch org.apache.spark.deploy.worker.Worker:
full log in /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/logs/spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out
[[email protected] spark-1.6.2-bin-hadoop2.6]$ cat /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/logs/spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out
Spark Command: /home/wangshumin/ekl/jdk1.8.0_162/bin/java -cp /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/conf/:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/spark-assembly-1.6.2-hadoop2.6.0.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/home/wangshumin/hadoop-2.6.5/etc/hadoop/ -Xms1g -Xmx1g org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://CentOSNode1:7077
========================================
[[email protected] spark-1.6.2-bin-hadoop2.6]$ cd /home/wangshumin/spark/spark-1.6.2-bin-hadoop2.6/logs
[[email protected] logs]$ ll
总用量 24
-rw-rw-r--. 1 wangshumin wangshumin 1225 3月 11 17:58 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out
-rw-rw-r--. 1 wangshumin wangshumin 2435 3月 11 12:11 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out.1
-rw-rw-r--. 1 wangshumin wangshumin 2439 3月 11 11:44 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out.2
-rw-rw-r--. 1 wangshumin wangshumin 2439 3月 11 10:26 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out.3
-rw-rw-r--. 1 wangshumin wangshumin 2439 3月 11 10:19 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out.4
-rw-rw-r--. 1 wangshumin wangshumin 2439 3月 11 09:55 spark-wangshumin-org.apache.spark.deploy.worker.Worker-1-CentOSNode3.out.5
[[email protected] logs]$