cdh5.13启动spark-shell报错

[[email protected] ~]# spark-shell
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
____ __
/ / ___ / /
\ / _ / _ `/ __/ '/
/
/ .__/_,// //_\ version 1.6.0
/
/

Using Scala version 2.10.5 (Java HotSpot™ 64-Bit Server VM, Java 1.8.0_151)
Type in expressions to have them evaluated.
Type :help for more information.
19/05/20 01:38:32 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: Required executor memory (1024+384 MB) is above the max threshold (1024 MB) of this cluster! Please check the values of ‘yarn.scheduler.maximum-allocation-mb’ and/or ‘yarn.nodemanager.resource.memory-mb’.
at org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:281)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:140)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:151)
at org.apache.spark.SparkContext.(SparkContext.scala:538)
at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1022)
at line3.line3.readiwCiwCiwC.(:15)
at line3.line3.read$$iwC.(:25)
解决方法

cdh5.13启动spark-shell报错

cdh5.13启动spark-shell报错

再次登录会报权限问题
[[email protected] ~]# spark-shell
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
____ __
/ / ___ / /
\ / _ / _ `/ __/ '/
/
/ .__/_,// //_\ version 1.6.0
/
/

Using Scala version 2.10.5 (Java HotSpot™ 64-Bit Server VM, Java 1.8.0_151)
Type in expressions to have them evaluated.
Type :help for more information.
19/05/20 01:56:28 ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)

解决方法:进行切换用户