livy开始抛出java.lang.illegalstateexception:未能创建子事件循环

6ovsh4lw  于 2021-07-13  发布在  Spark
关注(0)|答案(0)|浏览(365)

我正在使用emr 6.2.0和livy 0.7.0
我有一个工人，他一直在启动spark会话，并对其完成情况进行投票。这运行了一段时间，但突然利维开始扔

java.lang.IllegalStateException: failed to create a child event loop
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:88)
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:47)
        at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:59)
        at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:77)
        at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:72)
        at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:59)
        at org.apache.livy.rsc.RSCClient.<init>(RSCClient.java:81)
        at org.apache.livy.rsc.RSCClientFactory.createClient(RSCClientFactory.java:76)
        at org.apache.livy.LivyClientBuilder.build(LivyClientBuilder.java:130)
        at org.apache.livy.server.interactive.InteractiveSession$.$anonfun$create$1(InteractiveSession.scala:111)
        at scala.Option.orElse(Option.scala:447)
        at org.apache.livy.server.interactive.InteractiveSession$.create(InteractiveSession.scala:81)
        at org.apache.livy.server.interactive.InteractiveSessionServlet.createSession(InteractiveSessionServlet.scala:55)
        at org.apache.livy.server.interactive.InteractiveSessionServlet.createSession(InteractiveSessionServlet.scala:40)
        at org.apache.livy.server.SessionServlet.$anonfun$new$16(SessionServlet.scala:139)
        at org.scalatra.ScalatraBase.liftAction(ScalatraBase.scala:280)
        at org.scalatra.ScalatraBase.$anonfun$invoke$1(ScalatraBase.scala:274)
        at org.scalatra.ApiFormats.withRouteMultiParams(ApiFormats.scala:183)
        at org.scalatra.ApiFormats.withRouteMultiParams$(ApiFormats.scala:169)
        at org.apache.livy.server.JsonServlet.withRouteMultiParams(JsonServlet.scala:39)
        at org.scalatra.ScalatraBase.invoke(ScalatraBase.scala:273)
        at org.scalatra.ScalatraBase.invoke$(ScalatraBase.scala:272)
        at org.apache.livy.server.JsonServlet.invoke(JsonServlet.scala:39)
        at org.scalatra.ScalatraBase.$anonfun$runRoutes$3(ScalatraBase.scala:248)
        at scala.Option.flatMap(Option.scala:271)
        at org.scalatra.ScalatraBase.$anonfun$runRoutes$1(ScalatraBase.scala:246)
        at scala.collection.immutable.Stream.flatMap(Stream.scala:493)
        at org.scalatra.ScalatraBase.runRoutes(ScalatraBase.scala:245)
        at org.scalatra.ScalatraBase.runRoutes$(ScalatraBase.scala:243)
        at org.apache.livy.server.JsonServlet.runRoutes(JsonServlet.scala:39)
        at org.scalatra.ScalatraBase.runActions$1(ScalatraBase.scala:169)
        at org.scalatra.ScalatraBase.$anonfun$executeRoutes$6(ScalatraBase.scala:181)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at org.scalatra.ScalatraBase.cradleHalt(ScalatraBase.scala:199)
        at org.scalatra.ScalatraBase.executeRoutes(ScalatraBase.scala:181)
        at org.scalatra.ScalatraBase.executeRoutes$(ScalatraBase.scala:150)
        at org.apache.livy.server.JsonServlet.executeRoutes(JsonServlet.scala:39)
        at org.scalatra.ScalatraBase.$anonfun$handle$1(ScalatraBase.scala:123)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
        at org.scalatra.DynamicScope.withResponse(DynamicScope.scala:79)
        at org.scalatra.DynamicScope.withResponse$(DynamicScope.scala:77)
        at org.apache.livy.server.JsonServlet.withResponse(JsonServlet.scala:39)
        at org.scalatra.DynamicScope.$anonfun$withRequestResponse$1(DynamicScope.scala:59)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
        at org.scalatra.DynamicScope.withRequest(DynamicScope.scala:70)
        at org.scalatra.DynamicScope.withRequest$(DynamicScope.scala:68)
        at org.apache.livy.server.JsonServlet.withRequest(JsonServlet.scala:39)
        at org.scalatra.DynamicScope.withRequestResponse(DynamicScope.scala:58)
        at org.scalatra.DynamicScope.withRequestResponse$(DynamicScope.scala:56)
        at org.apache.livy.server.JsonServlet.withRequestResponse(JsonServlet.scala:39)
        at org.scalatra.ScalatraBase.handle(ScalatraBase.scala:123)
        at org.scalatra.ScalatraBase.handle$(ScalatraBase.scala:119)
        at org.apache.livy.server.JsonServlet.org$scalatra$servlet$ServletBase$$super$handle(JsonServlet.scala:39)
        at org.scalatra.servlet.ServletBase.handle(ServletBase.scala:53)
        at org.scalatra.servlet.ServletBase.handle$(ServletBase.scala:46)
        at org.apache.livy.server.SessionServlet.org$scalatra$MethodOverride$$super$handle(SessionServlet.scala:45)
        at org.scalatra.MethodOverride.handle(MethodOverride.scala:35)
...skipping...
21/02/05 14:15:57 INFO LineBufferedStream: java.lang.IllegalArgumentException: Unexpected client ID '40f2d9b7-6460-45b1-9521-6a92ff733766' in SASL handshake.
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.Utils.checkArgument(Utils.java:42)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.RpcServer$SaslServerHandler.update(RpcServer.java:288)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:61)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:36)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
21/02/05 14:15:57 INFO LineBufferedStream:      at java.lang.Thread.run(Thread.java:748)
21/02/05 14:15:57 INFO LineBufferedStream: 21/02/05 14:15:57 WARN DefaultChannelPipeline: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
21/02/05 14:15:57 INFO LineBufferedStream: java.lang.IllegalArgumentException: Unexpected client ID '40f2d9b7-6460-45b1-9521-6a92ff733766' in SASL handshake.
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.Utils.checkArgument(Utils.java:42)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.RpcServer$SaslServerHandler.update(RpcServer.java:288)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:61)
21/02/05 14:15:57 INFO LineBufferedStream:      at org.apache.livy.rsc.rpc.SaslHandler.channelRead0(SaslHandler.java:36)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
21/02/05 14:15:57 INFO LineBufferedStream:      at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
21/02/05 14:15:57 INFO LineBufferedStream:      at java.lang.Thread.run(Thread.java:748)
21/02/05 14:15:58 WARN ContextLauncher: Child process exited with code 1.
21/02/05 14:15:58 ERROR rejectedExecution: Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated
        at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:821)
        at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:327)
        at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:320)
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:746)
        at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:760)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122)
        at org.apache.livy.rsc.ContextLauncher$ChildProcess.fail(ContextLauncher.java:417)
        at org.apache.livy.rsc.ContextLauncher$ChildProcess.access$1100(ContextLauncher.java:369)
        at org.apache.livy.rsc.ContextLauncher$ChildProcess$1.run(ContextLauncher.java:400)
        at org.apache.livy.rsc.ContextLauncher$ChildProcess$2.run(ContextLauncher.java:451)
        at java.lang.Thread.run(Thread.java:748)

同时运行的会话并不多。最多可同时运行5个会话。作业完成后，工作进程将删除所有旧会话，以确保一次打开的会话不会太多。
糟糕的是，在worker停止创建新会话后，它不会自动恢复。只在livy服务器重新启动后工作。

hadoopnettyapache-sparkamazon-emrlivy

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com