在nodemanager启动时关闭它

byqmnocz  于 2021-05-29  发布在  Hadoop
关注(0)|答案(3)|浏览(645)

我在笔记本电脑上运行了hadoop。当hadoop启动时，我执行命令start-all.cmd。然后从4个守护进程开始。显示4个进程中的3个
shutdown\u msg:正在关闭桌面上的namenode-t7r9jv1/192.168.1.100
我如何避免这种情况

STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = DESKTOP-T7R9JV1/192.168.1.101
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 2.9.1  
19/09/08 22:03:13 INFO namenode.NameNode: createNameNode []
    19/09/08 22:03:14 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
    19/09/08 22:03:14 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
    19/09/08 22:03:14 INFO impl.MetricsSystemImpl: NameNode metrics system started
    19/09/08 22:03:14 INFO namenode.NameNode: fs.defaultFS is hdfs://0.0.0.0:19000
    19/09/08 22:03:14 INFO namenode.NameNode: Clients are to use 0.0.0.0:19000 to access this namenode/service.
    19/09/08 22:03:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    19/09/08 22:03:15 INFO util.JvmPauseMonitor: Starting JVM pause monitor
    19/09/08 22:03:15 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:50070
    19/09/08 22:03:15 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
    19/09/08 22:03:15 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
    19/09/08 22:03:15 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined
    19/09/08 22:03:15 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
    19/09/08 22:03:15 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
    19/09/08 22:03:15 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
    19/09/08 22:03:15 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
    19/09/08 22:03:16 INFO http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
    19/09/08 22:03:16 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
    19/09/08 22:03:16 INFO http.HttpServer2: Jetty bound to port 50070
    19/09/08 22:03:16 INFO mortbay.log: jetty-6.1.26
    19/09/08 22:03:16 INFO mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
    19/09/08 22:03:17 ERROR common.Util: Syntax error in URI C:\BigData\hadoop-2.9.1\data\namenode. Please check hdfs configuration.
    java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\BigData\hadoop-2.9.1\data\namenode
            at java.net.URI$Parser.fail(URI.java:2848)
            at java.net.URI$Parser.checkChars(URI.java:3021)
            at java.net.URI$Parser.parse(URI.java:3058)
            at java.net.URI.<init>(URI.java:588)
            at org.apache.hadoop.hdfs.server.common.Util.stringAsURI(Util.java:49)
            at org.apache.hadoop.hdfs.server.common.Util.stringCollectionAsURIs(Util.java:99)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1462)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1417)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkConfiguration(FSNamesystem.java:617)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:669)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
    19/09/08 22:03:17 WARN common.Util: Path C:\BigData\hadoop-2.9.1\data\namenode should be specified as a URI in configuration files. Please update hdfs configuration.
    19/09/08 22:03:17 ERROR common.Util: Syntax error in URI C:\BigData\hadoop-2.9.1\data\namenode. Please check hdfs configuration.
    java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\BigData\hadoop-2.9.1\data\namenode
            at java.net.URI$Parser.fail(URI.java:2848)
            at java.net.URI$Parser.checkChars(URI.java:3021)
            at java.net.URI$Parser.parse(URI.java:3058)
            at java.net.URI.<init>(URI.java:588)
            at org.apache.hadoop.hdfs.server.common.Util.stringAsURI(Util.java:49)
            at org.apache.hadoop.hdfs.server.common.Util.stringCollectionAsURIs(Util.java:99)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1462)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceEditsDirs(FSNamesystem.java:1507)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceEditsDirs(FSNamesystem.java:1476)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkConfiguration(FSNamesystem.java:619)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:669)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
    19/09/08 22:03:17 WARN common.Util: Path C:\BigData\hadoop-2.9.1\data\namenode should be specified as a URI in configuration files. Please update hdfs configuration.
    19/09/08 22:03:17 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
    19/09/08 22:03:17 WARN namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!
    19/09/08 22:03:17 ERROR common.Util: Syntax error in URI C:\BigData\hadoop-2.9.1\data\namenode. Please check hdfs configuration.
    java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\BigData\hadoop-2.9.1\data\namenode
            at java.net.URI$Parser.fail(URI.java:2848)
            at java.net.URI$Parser.checkChars(URI.java:3021)
            at java.net.URI$Parser.parse(URI.java:3058)
            at java.net.URI.<init>(URI.java:588)
            at org.apache.hadoop.hdfs.server.common.Util.stringAsURI(Util.java:49)
            at org.apache.hadoop.hdfs.server.common.Util.stringCollectionAsURIs(Util.java:99)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1462)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1417)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:670)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
    19/09/08 22:03:17 WARN common.Util: Path C:\BigData\hadoop-2.9.1\data\namenode should be specified as a URI in configuration files. Please update hdfs configuration.
    19/09/08 22:03:17 ERROR common.Util: Syntax error in URI C:\BigData\hadoop-2.9.1\data\namenode. Please check hdfs configuration.
    java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\BigData\hadoop-2.9.1\data\namenode
            at java.net.URI$Parser.fail(URI.java:2848)
            at java.net.URI$Parser.checkChars(URI.java:3021)
            at java.net.URI$Parser.parse(URI.java:3058)
            at java.net.URI.<init>(URI.java:588)
            at org.apache.hadoop.hdfs.server.common.Util.stringAsURI(Util.java:49)
            at org.apache.hadoop.hdfs.server.common.Util.stringCollectionAsURIs(Util.java:99)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1462)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceEditsDirs(FSNamesystem.java:1507)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceEditsDirs(FSNamesystem.java:1476)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:670)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
    19/09/08 22:03:17 WARN common.Util: Path C:\BigData\hadoop-2.9.1\data\namenode should be specified as a URI in configuration files. Please update hdfs configuration.
    19/09/08 22:03:17 INFO namenode.FSEditLog: Edit logging is async:true
    19/09/08 22:03:17 INFO namenode.FSNamesystem: KeyProvider: null
    19/09/08 22:03:17 INFO namenode.FSNamesystem: fsLock is fair: true
    19/09/08 22:03:17 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
    19/09/08 22:03:17 INFO namenode.FSNamesystem: fsOwner             = User (auth:SIMPLE)
    19/09/08 22:03:17 INFO namenode.FSNamesystem: supergroup          = supergroup
    19/09/08 22:03:17 INFO namenode.FSNamesystem: isPermissionEnabled = true
    19/09/08 22:03:17 INFO namenode.FSNamesystem: HA Enabled: false
    19/09/08 22:03:17 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
    19/09/08 22:03:17 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
    19/09/08 22:03:17 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Sep 08 22:03:17
    19/09/08 22:03:17 INFO util.GSet: Computing capacity for map BlocksMap
    19/09/08 22:03:17 INFO util.GSet: VM type       = 32-bit
    19/09/08 22:03:17 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB
    19/09/08 22:03:17 INFO util.GSet: capacity      = 2^22 = 4194304 entries
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
    19/09/08 22:03:17 WARN conf.Configuration: No unit for dfs.heartbeat.interval(3) assuming SECONDS
    19/09/08 22:03:17 WARN conf.Configuration: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
    19/09/08 22:03:17 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
    19/09/08 22:03:17 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
    19/09/08 22:03:17 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: defaultReplication         = 1
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: maxReplication             = 512
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: minReplication             = 1
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
    19/09/08 22:03:17 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
    19/09/08 22:03:17 INFO namenode.FSNamesystem: Append Enabled: true
    19/09/08 22:03:17 INFO util.GSet: Computing capacity for map INodeMap
    19/09/08 22:03:17 INFO util.GSet: VM type       = 32-bit
    19/09/08 22:03:17 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB
    19/09/08 22:03:17 INFO util.GSet: capacity      = 2^21 = 2097152 entries
    19/09/08 22:03:17 INFO namenode.FSDirectory: ACLs enabled? false
    19/09/08 22:03:17 INFO namenode.FSDirectory: XAttrs enabled? true
    19/09/08 22:03:17 INFO namenode.NameNode: Caching file names occurring more than 10 times
    19/09/08 22:03:17 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: falseskipCaptureAccessTimeOnlyChange: false
    19/09/08 22:03:17 INFO util.GSet: Computing capacity for map cachedBlocks
    19/09/08 22:03:17 INFO util.GSet: VM type       = 32-bit
    19/09/08 22:03:17 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB
    19/09/08 22:03:17 INFO util.GSet: capacity      = 2^19 = 524288 entries
    19/09/08 22:03:17 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
    19/09/08 22:03:17 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
    19/09/08 22:03:17 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
    19/09/08 22:03:17 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
    19/09/08 22:03:17 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
    19/09/08 22:03:17 INFO util.GSet: Computing capacity for map NameNodeRetryCache
    19/09/08 22:03:17 INFO util.GSet: VM type       = 32-bit
    19/09/08 22:03:17 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
    19/09/08 22:03:17 INFO util.GSet: capacity      = 2^16 = 65536 entries
    19/09/08 22:03:17 ERROR namenode.NameNode: Failed to start namenode.
    java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
            at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
            at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:606)
            at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:1006)
            at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:558)
            at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:518)
            at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:370)
            at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:226)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1048)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
    19/09/08 22:03:17 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
    19/09/08 22:03:17 INFO namenode.NameNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down NameNode at DESKTOP-T7R9JV1/192.168.1.101
  ************************************************************/
hadoopcommand-prompt

3条答案

按热度按时间
zzzyeukh

zzzyeukh1#

在我的问题中，我从网上找到了更多的解决方法。
我的第一个问题是我在windows中安装了32位java版本。有我的环境变量 JAVA_HOME 路径设置为 C:\Progra~2\Java\<JDK version> . 然后我通过设置 JAVA_HOME 作为 C:\Progra~1\Java\<JDK version> .
将64位java版本设置为 JAVA_HOME 我已经跑了 start-all.cmd 再一次。那么除了 namenode ，所有其他的执事都工作了。跑 namenode 我遵循了这些步骤。
以管理员身份打开cmd。
键入并运行 stop-all.cmd hadoop namenode –format start-all.cmd 它100%解决了我的问题，对我来说非常有效。

赞(0）分享  回复(0）举报  2021-05-29
sqyvllje

sqyvllje2#

必须通过以下方式格式化namenode hadoop namenode –format 然后重新启动服务器

赞(0）分享  回复(0）举报  2021-05-29
7rfyedvj

7rfyedvj3#

索引2处不透明部分中的非法字符
索引2是反斜杠，它不是有效的uri字符
在配置文件中，需要对文件uri使用正向斜杠和协议
例如，改变

C:\BigData\hadoop-2.9.1\data\namenode

到

file:/C:/BigData/hadoop-2.9.1/data/namenode
赞(0）分享  回复(0）举报  2021-05-29
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com