cassandra readtimeoutexception获取表中的所有记录,即使在重写超时之后也是如此

w8biq8rn  于 2021-06-15  发布在  Cassandra
关注(0)|答案(1)|浏览(434)

因此,我尝试使用具有以下查询级别设置的java datastax 3.1驱动程序获取cassandra表中的所有记录(~10k记录):

// |query| reads all records in a table (select * from table)
// Setting read timeout as 10 minutes
query.setReadTimeoutMillis(600000);
// Set page size as 1000
query.setFetchSize(1000);

cassandra服务器设置为默认值(所有默认超时)。它是一个三节点集群。
驱动程序能够成功读取前1000条记录,但每次驱动程序试图获取下1000条记录时,我都会出现以下错误。

com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded)
    at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:88)
    at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:25)
    at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
    at com.datastax.driver.core.ArrayBackedResultSet$MultiPage.prepareNextRow(ArrayBackedResultSet.java:312)
    at com.datastax.driver.core.ArrayBackedResultSet$MultiPage.isExhausted(ArrayBackedResultSet.java:269)
    at com.datastax.driver.core.ArrayBackedResultSet$1.hasNext(ArrayBackedResultSet.java:143)
    ... <some more stacktrace of custom codebase trying to read all rows in a table>
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded)
    at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:115)
    at com.datastax.driver.core.Responses$Error.asException(Responses.java:124)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:500)
    at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1012)
    at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:935)
    at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:328)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:321)
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:328)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:321)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:328)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:321)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:328)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:321)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1280)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:328)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:890)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:564)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:505)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:419)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded)
    at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:62)
    at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37)
    at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:277)
    at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:257)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
    ... 19 common frames omitted

因此,读取表中所有记录的服务进入崩溃循环,每次都有相同的异常。
令我惊讶的是,这个异常总是在读取第1000条记录后的10秒内发生。默认情况下,10sec是cassandra服务器端超时,对于我的查询,应该将其覆盖为600sec(显然没有发生这种情况)。有人能解释一下为什么会这样吗?
p、 在苦苦寻找根源之后,我重新启动了Cassandra。瞧!,之后一切都很顺利,所有的记录都是一页一页地读,处理起来没有任何问题。

gdrx4gfi

gdrx4gfi1#

这个 select * from table 是错误的模式-cassandra没有针对它进行优化,因此它需要从所有节点获取数据,并通过协调器节点进行流处理。在cassandra中访问数据最有效的方法是指定分区键(可能在集群列上附加条件)。
如果您确实需要扫描集群中的所有数据,那么您需要将is实现为一组令牌范围查询,这些查询将遍历集群中的所有范围(您可以通过 Metadata java或python驱动程序的类)。

相关问题