ibm datastage将数据加载到apache cassandra引发transportexception

wvt8vs2t  于 2021-06-14  发布在  Cassandra
关注(0)|答案(1)|浏览(444)

我使用datastage 11.7 cassandra连接器连接2个节点配置的cassandra虚拟机(192.168.3.240和192.168.3.241)。
我的apachecassandra版本是3.11.3,我使用的是datastax驱动程序:dse-java-driver-core-1.8.1.jar和dse-java-driver-extras-1.8.1.jar。
在datastage中从cassandra连接器读取数据是可以的,但是当我尝试将数据插入cassandra表时,它会引发以下异常:

Cassandra_Connector_0,1: com.datastax.driver.core.exceptions.TransportException: [/192.168.3.241:9042] Connection has been closed
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 29)
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 15)
at com.datastax.driver.core.DriverThrowables.propagateCause (DriverThrowables.java: 26)
at com.datastax.driver.core.AbstractSession.prepare (AbstractSession.java: 77)
at com.ibm.is.cc.cassandra.engine.CassandraSession.prepareInsertStatement (CassandraSession.java: 161)
at com.ibm.is.cc.cassandra.engine.InsertMutator.<init> (InsertMutator.java: 25)
at com.ibm.is.cc.cassandra.config.MutationType$1.newMutator (MutationType.java: 21)
at com.ibm.is.cc.cassandra.engine.CassandraSession.getMutator (CassandraSession.java: 156)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepareColumnExtractor (CassandraConsumer.java: 92)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepare (CassandraConsumer.java: 62)

你知道为什么会抛出这样的异常吗?
更新1:现在我使用的是oss驱动程序,“插入数据”工作现在可以正常工作了。但是,如果我使用oss驱动程序,这次“read data”作业将抛出nosuchmethoderror:

Cassandra_Connector_0,1: com.ascential.e2.common.CC_Exception: "Connector encountered an exception: java.lang.NoSuchMethodError: com/datastax/driver/core/SimpleStatement.setRoutingToken(Lcom/datastax/driver/core/Token;)Lcom/datastax/driver/core/RegularStatement; (loaded from file:/home/dsadm/cassandra-client-write/cassandra-driver-core-3.7.1.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75) called from class com.ibm.is.cc.cassandra.engine.CassandraSession (loaded from file:/opt/IBM/InformationServer/Server/DSEngine/../DSComponents/bin/cassandra/CassandraConnectorCore.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75).
    at com.ibm.is.cc.cassandra.engine.CassandraSession.executeSelect(CassandraSession.java:126)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.runProducers(CassandraAdapter.java:106)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.run(CassandraAdapter.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod(Delegator.java:40)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100(Delegator.java:16)
    at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke(Delegator.java:56)
    at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run(CassandraAdapterWrapper.java:169)
"
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.createException (CC_CassandraErrorCode.java: 124)
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.newWrapperFailureException (CC_CassandraErrorCode.java: 42)
at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod (Delegator.java: 44)
at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100 (Delegator.java: 16)
at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke (Delegator.java: 56)
at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run (CassandraAdapterWrapper.java: 169)
ecr0jaav

ecr0jaav1#

问题很可能出在您使用的驱动程序中。您正在使用dse的驱动程序,而对于开源cassandra,您需要使用oss驱动程序,maven坐标如下:

<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-core</artifactId>
  <version>3.7.1</version>
</dependency>
<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-extras</artifactId>
  <version>3.7.1</version>
</dependency>

相关问题