I am using "gridgain-community-8.8.1" package to bring up ignite in one AWS EC2 instance and access that from AWS EMR Spark-3.0.1 job. While trying to get the ignite instance from spark job using "Ignition.getOrStart(igniteConf)", its throwing exceptions. I am using same java openJDK8 and gridgain library 8.8.1 for both EC2 instance and EMR spark job.
Below ignite config I have used to bring up ignite in EC2 instance, ignite was successfully brought up: (using ./gridgain-community-8.8.1/bin/ignite.sh aws-s3-ip-finder.xml)aws-s3-ip-finder.xml内容为：
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation=" http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Enable peer class loading. -->
<!-- <property name="peerClassLoadingEnabled" value="true"/> -->
<!-- Set deployment mode. -->
<!-- <property name="deploymentMode" value="CONTINUOUS"/> -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
<property name="awsCredentialsProvider" ref="aws.creds"/>
<property name="bucketName" value="a206768-la-ignite-s3-ip-finder"/>
</bean>
</property>
</bean>
</property>
</bean>
<!-- Instance Profile based credentials -->
<bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
<constructor-arg value="false" />
</bean>
</beans>下面是我连接到这个ec2 ignite示例的scala代码：
def connectToIgniteClient(): Ignite = {
logInfo(">>> Connecting to ignite-client")
val instanceProfileCreds = new InstanceProfileCredentialsProvider(false)
logInfo(">>> IP finding")
val ipFinder = new TcpDiscoveryS3IpFinder()
ipFinder.setAwsCredentialsProvider(instanceProfileCreds)
ipFinder.setBucketName("a206768-la-ignite-s3-ip-finder")
logInfo(">>> S3 connected")
val discoverySpi = new TcpDiscoverySpi()
discoverySpi.setIpFinder(ipFinder)
logInfo(">>> Spi connected")
val igniteConf = new IgniteConfiguration()
igniteConf.setClientMode(true)
igniteConf.setDiscoverySpi(discoverySpi)
logInfo(">>> Config set")
Ignition.getOrStart(igniteConf) //Exception is thrown from this line
}以下是emr spark cluster的异常日志：
21/03/25 17:32:24 INFO Client: Application report for application_1616693342082_0001 (state: RUNNING)
21/03/25 17:32:24 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 10.150.80.195
ApplicationMaster RPC port: -1
queue: default
start time: 1616693536055
final status: UNDEFINED
tracking URL: http://ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com:20888/proxy/application_1616693342082_0001/
user: hadoop
21/03/25 17:32:24 INFO YarnClientSchedulerBackend: Application application_1616693342082_0001 has started running.
21/03/25 17:32:24 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 45257.
21/03/25 17:32:24 INFO NettyBlockTransferService: Server created on ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com:45257
21/03/25 17:32:24 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
21/03/25 17:32:24 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com, 45257, None)
21/03/25 17:32:24 INFO BlockManagerMasterEndpoint: Registering block manager ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com:45257 with 1028.8 MiB RAM, BlockManagerId(driver, ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com, 45257, None)
21/03/25 17:32:24 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com, 45257, None)
21/03/25 17:32:24 INFO BlockManager: external shuffle service port = 7337
21/03/25 17:32:24 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, ip-10-150-81-252.xxxx.aws-int.thomsonreuters.com, 45257, None)
21/03/25 17:32:24 INFO ServerInfo: Adding filter to /metrics/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
21/03/25 17:32:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@194224ca{/metrics/json,null,AVAILABLE,@Spark}
21/03/25 17:32:24 INFO SingleEventLogFileWriter: Logging events to hdfs:/var/log/spark/apps/application_1616693342082_0001.inprogress
21/03/25 17:32:25 INFO Utils: Using initial executors = 50, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
21/03/25 17:32:25 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
21/03/25 17:32:25 INFO YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
21/03/25 17:32:25 INFO YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark-client://YarnAM)
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Spark initialized
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Loading Person data to ignite caches
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Connecting to ignite
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Connecting to ignite-client
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> IP finding
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> S3 connected
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Spi connected
21/03/25 17:32:25 INFO IgniteCommunicatorPerson: >>> Config set
21/03/25 17:32:25 WARN : Failed to resolve default logging config file: config/java.util.logging.properties
Console logging handler is not configured.
21/03/25 17:32:25 WARN G: Ignite work directory is not provided, automatically resolved to: /mnt/var/lib/hadoop/steps/s-FV02OVOM7RT4/ignite/work
21/03/25 17:32:26 INFO IgniteKernal:
21/03/25 17:32:26 INFO IgniteKernal: Config URL: n/a
21/03/25 17:32:26 INFO IgniteKernal: IgniteConfiguration [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, dataStreamerPoolSize=8, utilityCachePoolSize=8, utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, buildIdxPoolSize=1, igniteHome=null, igniteWorkDir=/mnt/var/lib/hadoop/steps/s-FV02OVOM7RT4/ignite/work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@4d1f1ff5, nodeId=c2f73ab1-066c-4fc1-b1d2-2ebd6ca79311, marsh=BinaryMarshaller [], marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=5000, netCompressionLevel=1, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, metricsUpdateFreq=2000, metricsExpTime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=org.apache.ignite.spi.communication.tcp.internal.ConnectGateway@23f8036d, ctxInitLatch=java.util.concurrent.CountDownLatch@68f69ca3[Count = 1], stopping=false, clientPool=null, nioSrvWrapper=null, stateProvider=null, connectionRequestor=null], evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@1e3566e, colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [], indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@2842c098, addrRslvr=null, encryptionSpi=org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi@2820b369, tracingSpi=org.apache.ignite.spi.tracing.NoopTracingSpi@46b21632, clientMode=true, rebalanceThreadPoolSize=1, rebalanceTimeout=10000, rebalanceBatchesPrefetchCnt=2, rebalanceThrottle=0, rebalanceBatchSize=524288, txCfg=TransactionConfiguration [txSerEnabled=false, dfltIsolation=REPEATABLE_READ, dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0, txTimeoutOnPartitionMapExchange=0, deadlockTimeout=10000, pessimisticTxLogSize=0, pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null, useJtaSync=false], cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, p2pMissedCacheSize=100, locHost=null, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000, sysWorkerBlockedTimeout=120000, clientFailureDetectionTimeout=30000, metricsLogFreq=60000, connectorCfg=ConnectorConfiguration [jettyPath=null, host=null, port=11211, noDelay=true, directBuf=false, sndBufSize=32768, rcvBufSize=32768, idleQryCurTimeout=600000, idleQryCurCheckFreq=60000, sndQueueLimit=0, selectorCnt=4, idleTimeout=7000, sslEnabled=false, sslClientAuth=false, sslCtxFactory=null, sslFactory=null, portRange=100, threadPoolSize=8, msgInterceptor=null], odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, sslCtxFactory=null, platformCfg=null, binaryCfg=null, memCfg=null, pstCfg=null, dsCfg=null, activeOnStart=null, autoActivation=null, clusterStateOnStart=null, sqlConnCfg=null, cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true, maxOpenCursorsPerConn=128, threadPoolSize=8, selectorCnt=4, idleTimeout=0, handshakeTimeout=10000, jdbcEnabled=true, odbcEnabled=true, thinCliEnabled=true, sslEnabled=false, useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null, thinCliCfg=ThinClientConfiguration [maxActiveTxPerConn=100, maxActiveComputeTasksPerConn=0]], mvccVacuumThreadCnt=2, mvccVacuumFreq=5000, authEnabled=false, failureHnd=null, commFailureRslvr=null, sqlCfg=SqlConfiguration [longQryWarnTimeout=1000, sqlQryHistSize=1000, sqlGlobalMemoryQuota=60%, sqlQueryMemoryQuota=0, sqlOffloadingEnabled=false, dfltQryTimeout=0]]
21/03/25 17:32:26 INFO IgniteKernal: Daemon mode: off
21/03/25 17:32:26 INFO IgniteKernal: OS: Linux 4.14.200-155.322.amzn2.x86_64 amd64
21/03/25 17:32:26 INFO IgniteKernal: OS user: hadoop
21/03/25 17:32:26 INFO IgniteKernal: PID: 14773
21/03/25 17:32:26 INFO IgniteKernal: Language runtime: Java Platform API Specification ver. 1.8
21/03/25 17:32:26 INFO IgniteKernal: VM information: OpenJDK Runtime Environment 1.8.0_272-b10 Amazon.com Inc. OpenJDK 64-Bit Server VM 25.272-b10
21/03/25 17:32:26 INFO IgniteKernal: VM total memory: 2.0GB
21/03/25 17:32:26 INFO IgniteKernal: Remote Management [restart: off, REST: off, JMX (remote: off)]
21/03/25 17:32:26 INFO IgniteKernal: Logger: JavaLogger [quiet=true, config=null]
21/03/25 17:32:26 INFO IgniteKernal: IGNITE_HOME=null
21/03/25 17:32:26 INFO IgniteKernal: VM arguments: [-DEMR_CLUSTER_ID=j-3MH2DITFW5UJE, -DEMR_RELEASE_LABEL=emr-6.2.0, -DEMR_STEP_ID=s-FV02OVOM7RT4, -Xmx2048M, -XX:OnOutOfMemoryError=kill -9 %p, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxHeapFreeRatio=70, -XX:+CMSClassUnloadingEnabled]
21/03/25 17:32:26 INFO IgniteKernal: Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
21/03/25 17:32:26 WARN IgniteKernal: Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
21/03/25 17:32:26 WARN GridDiagnostic: Initial heap size is 236MB (should be no less than 512MB, use -Xms512m -Xmx512m).
21/03/25 17:32:26 INFO IgnitePluginProcessor: Configured plugins:
21/03/25 17:32:26 INFO IgnitePluginProcessor: ^-- None
21/03/25 17:32:26 INFO IgnitePluginProcessor:
21/03/25 17:32:26 INFO FailureProcessor: Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]
21/03/25 17:32:26 INFO TcpCommunicationSpi: Successfully bound communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false]
21/03/25 17:32:26 WARN TcpCommunicationSpi: Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
21/03/25 17:32:26 WARN NoopCheckpointSpi: Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation)
21/03/25 17:32:26 WARN GridCollisionManager: Collision resolution is disabled (all jobs will be activated upon arrival).
21/03/25 17:32:26 INFO IgniteKernal: Security status [authentication=off, tls/ssl=off]
21/03/25 17:32:27 INFO PdsFoldersResolver: Consistent ID used for local node is [c2f73ab1-066c-4fc1-b1d2-2ebd6ca79311] according to persistence data storage folders
21/03/25 17:32:27 WARN IgniteH2Indexing: Serialization of Java objects in H2 was enabled.
21/03/25 17:32:27 INFO QueryMemoryManager: SQL query global quota was set to 1267571097. Current memory tracking parameters: [qryQuota=0, globalQuota=1267571097, offloadingEnabled=false]
21/03/25 17:32:27 INFO QueryMemoryManager: SQL query memory quota was set to 0. Current memory tracking parameters: [qryQuota=0, globalQuota=1267571097, offloadingEnabled=false]
21/03/25 17:32:27 INFO ClientListenerProcessor: Client connector processor has started on TCP port 10800
21/03/25 17:32:27 INFO GridRestProcessor: REST protocols do not start on client node. To start the protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system property.
21/03/25 17:32:28 INFO IgniteKernal: Non-loopback local IPs: 10.150.81.252, fe80:0:0:0:10bf:75ff:fe95:1e9%eth0
21/03/25 17:32:28 INFO IgniteKernal: Enabled local MACs: 12BF759501E9
21/03/25 17:32:28 INFO ClusterProcessor: Cluster ID and tag has been read from metastorage: null
21/03/25 17:32:28 INFO IgniteClusterImpl: Shutdown policy was updated [oldVal=null, newVal=null]
21/03/25 17:32:28 WARN TcpDiscoveryS3IpFinder: Amazon client configuration is not set (will use default).
21/03/25 17:32:28 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 4, script: , vendor: , memory -> name: memory, amount: 4743, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
21/03/25 17:32:28 ERROR IgniteKernal: Failed to start manager: GridManagerAdapter [enabled=true, name=o.a.i.i.managers.discovery.GridDiscoveryManager]
class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@f996445], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:280)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:986)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1931)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1284)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2108)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1771)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1150)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:670)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:611)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:410)
at un.api.aws.client.IgniteCommunicatorPerson$.connectToIgniteClient(IgniteCommunicatorPerson.scala:158)
at un.api.aws.client.IgniteCommunicatorPerson$.loadPersonDataToIgniteCache(IgniteCommunicatorPerson.scala:63)
at un.api.aws.client.IgniteCommunicatorPerson$.connectToIgniteClusterAndLoadData(IgniteCommunicatorPerson.scala:37)
at un.api.aws.client.IgniteCommunicatorPerson$.main(IgniteCommunicatorPerson.scala:21)
at un.api.aws.client.IgniteCommunicatorPerson.main(IgniteCommunicatorPerson.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:936)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1015)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1024)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to deserialize object with given class loader: sun.misc.Launcher$AppClassLoader@70dea4e
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.checkFailedError(TcpDiscoverySpi.java:2019)
at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1996)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at org.apache.ignite.spi.discovery.tcp.ClientImpl$1.body(ClientImpl.java:315)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57)
21/03/25 17:32:28 ERROR IgniteKernal: Got exception while starting (will rollback startup routine).
class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1936)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1284)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2108)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1771)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1150)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:670)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:611)
at org.apache.ignite.Ignition.getOrStart(Ignition.java:410)
at un.api.aws.client.IgniteCommunicatorPerson$.connectToIgniteClient(IgniteCommunicatorPerson.scala:158)
at un.api.aws.client.IgniteCommunicatorPerson$.loadPersonDataToIgniteCache(IgniteCommunicatorPerson.scala:63)
at un.api.aws.client.IgniteCommunicatorPerson$.connectToIgniteClusterAndLoadData(IgniteCommunicatorPerson.scala:37)
at un.api.aws.client.IgniteCommunicatorPerson$.main(IgniteCommunicatorPerson.scala:21)
at un.api.aws.client.IgniteCommunicatorPerson.main(IgniteCommunicatorPerson.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:936)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1015)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1024)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@f996445], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:280)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:986)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1931)
... 24 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to deserialize object with given class loader: sun.misc.Launcher$AppClassLoader@70dea4e
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.checkFailedError(TcpDiscoverySpi.java:2019)
at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1996)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at org.apache.ignite.spi.discovery.tcp.ClientImpl$1.body(ClientImpl.java:315)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57)
21/03/25 17:32:28 INFO IgniteKernal:任何解决这个问题的建议都将不胜感激。我使用scala版本#2.12.12、spark版本#3.0.1和openjdk-8（也适用于ec2示例）来执行emr spark作业。
谢谢，钱丹
暂无答案！
目前还没有任何答案，快来回答吧！