win10启动hadoop时遇到的坑
目录
2、 util.ExitUtil: Exiting with status 1: java.lang.NullPointerException
5、Error occurred during initialization of VM
背景
昨日编译好了hadoop3.0.2源码,今日启动之,孰料想,又出现一堆错误
1、 Exception in thread "main" java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.
启动不了nameNode,解决方法:
在hadoop文件夹\hadoop-3.0.2\etc\hadoop\core-site.xml的<configuration>标签下,添加如下内容
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
2、 Exiting with status 1: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory D:\tmp\hadoop-123\dfs\name is in an inconsistent state: storage directory does not exist or is not accessible.
dfs\name文件夹没有创建。解决方法:
在某处创建dfs文件夹,然后在dfs文件夹里创建name和data文件夹,并且在hadoop文件夹\etc\hadoop\core-site.xml的<configuration>标签下,添加如下内容:
<property>
<name>hadoop.tmp.dir</name>
<value>D:/ProgramCoding/hadoop-3.0.2/workspace</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/data</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/name</value>
3、 Exiting with status 1: java.io.IOException: All specified directories are not accessible or do not exist.
解决方法:
检查core_site.xml文件里所有文件分隔符,必须是/而不是windows的\
4、NameNode is not formatted.
解决方法:
在cmd里运行命令hadoop namenode -format格式化namenode
格式化namenode时,又遇到一些问题
1、URI has an authority component
打开hadoop文件夹\etc\hadoop\hdfs-site.xml,在<configuration>标签里,输入如下内容
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/namenode/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/namenode/data</value>
</property>
前提是保证两个<value>标签下的文件夹都存在,如不存在,就创建
2、 util.ExitUtil: Exiting with status 1: java.lang.NullPointerException
java.lang.NullPointerException
at java.io.File.(File.java:277)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.setStorageDirectories(NNStorage.java:300)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.(NNStorage.java:161)
at org.apache.hadoop.hdfs.server.namenode.FSImage.(FSImage.java:127)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:829)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1213)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)
解决方法:检查hadoop文件夹\hadoop-3.0.2\etc\hadoop\hdfs-site.xml文件里<value>的值,绝对路径前面必须加上file:/
解决完这两个问题后,就可以正常格式化namenode了
5、Error occurred during initialization of VM
Could not reserve enough space for 1024000KB object heap
内存预留过大问题,解决方法:
修改文件hadoop文件夹\etc\hadoop\hadoop-env.cmd,添加或修改HADOOP_HEAPSIZE的值为200(默认1000)
启动Hadoop
解决完上面问题后,就可以start-all了
出现四个窗口,命令运行不结束就行