I am trying to understand the warning, every time i am seeing the below exception when i run my spark job .I am seeing this in 2 nodes of my 3 node cluster.But as i said its just warn , job succeeds how ever.
com.datastax.driver.core.exceptions.ConnectionException: [x.x.x.x/x.x.x.x:9042] Pool was closed during initializationCASSANDRA LOG
INFO [SharedPool-Worker-1] 2017-07-17 22:25:48,716 Message.java:605 - Unexpected exception during request; channel = [id: 0xf0ee1096, /x.x.x.x:54863 => /x.x.x.x:9042] io.netty.channel.unix.Errors$NativeIoException: readAddress() failed: Connection timed out at io.netty.channel.unix.Errors.newIOException(Errors.java:105) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.unix.Errors.ioResult(Errors.java:121) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.unix.FileDescriptor.readAddress(FileDescriptor.java:134) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.epoll.AbstractEpollChannel.doReadBytes(AbstractEpollChannel.java:239) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:822) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:348) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) ~[netty-all-4.0.34.Final.jar:4.0.34.Final] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
1条答案
按热度按时间lndjwyie1#
错误的核心是“连接超时”。我建议排除Cassandra集群的网络连接故障，从简单的工具如
ping，telnet和nc开始。一些潜在的原因：您提到该问题是间歇性的（“在我的3节点群集的2个节点中看到此问题”），并且不会导致作业失败。这可能表示上面列出的任何问题仅在群集中的一部分节点上发生。（如果与所有节点的连接都中断，则作业可能会失败。）