处理消息时发生thrift传输错误

atmip9wb  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(404)

我正在运行cassandra1.2.6和apachespark 0.8.0。
在这里,我使用spark的newapihadooprdd从cassandra创建一个rdd。
当我从spark appln运行作业时,我发现cassandra正在记录以下错误消息

INFO 21:36:28,629 Listening for thrift clients...
DEBUG 21:36:29,730 Disseminating load info ...
DEBUG 21:36:57,844 Started replayAllFailedBatches
DEBUG 21:36:57,845 forceFlush requested but everything is clean in batchlog
DEBUG 21:36:57,846 Finished replayAllFailedBatches
DEBUG 21:37:29,731 Disseminating load info ...
DEBUG 21:37:57,846 Started replayAllFailedBatches
DEBUG 21:37:57,847 forceFlush requested but everything is clean in batchlog
DEBUG 21:37:57,847 Finished replayAllFailedBatches
DEBUG 21:38:29,732 Disseminating load info ...
DEBUG 21:38:57,847 Started replayAllFailedBatches
DEBUG 21:38:57,849 forceFlush requested but everything is clean in batchlog
DEBUG 21:38:57,849 Finished replayAllFailedBatches
DEBUG 21:39:29,732 Disseminating load info ...
DEBUG 21:39:57,849 Started replayAllFailedBatches
DEBUG 21:39:57,850 forceFlush requested but everything is clean in batchlog
DEBUG 21:39:57,850 Finished replayAllFailedBatches
DEBUG 21:39:57,956 computing ranges for -3011659447910895493
DEBUG 21:40:00,043 Thrift transport error occurred during processing of message.
org.apache.thrift.transport.TTransportException
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
    at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:22)
    at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
DEBUG 21:40:00,048 Thrift transport error occurred during processing of message.
org.apache.thrift.transport.TTransportException
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
    at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:22)
    at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
DEBUG 21:40:03,071 execute_cql3_query
DEBUG 21:40:03,090 request complete
DEBUG 21:40:03,253 prepare_cql3_query
DEBUG 21:40:03,267 execute_prepared_cql3_query
DEBUG 21:40:03,275 request complete
DEBUG 21:40:03,291 prepare_cql3_query
DEBUG 21:40:03,310 execute_prepared_cql3_query
DEBUG 21:40:03,312 request complete
DEBUG 21:40:03,314 prepare_cql3_query
DEBUG 21:40:03,326 execute_prepared_cql3_query
DEBUG 21:40:03,327 request complete
insrf1ej

insrf1ej1#

请尝试新的datastax cassandra spark驱动程序(此处提供)。它能够直接访问cassandra,而无需hadoopapi。

相关问题