在spring Maven中运行集成测试。创建SparkSession时,给予以下错误
- 我有
<additionalClasspathElement>${project.build.outputDirectory}
- 并有
<argLine>--add-exports java.base/sun.nio.ch=ALL-UNNAMED</argLine>
ERROR!
java.lang.IllegalAccessError: class org.apache.spark.storage.StorageUtils$ (in unnamed module @0x74235045) cannot access class sun.nio.ch.DirectBuffer (in module java.base) because module java.base does not export sun.nio.ch to unnamed module @0x74235045
at org.apache.spark.storage.StorageUtils$.<clinit>(StorageUtils.scala:213)
at org.apache.spark.storage.BlockManagerMasterEndpoint.<init>(BlockManagerMasterEndpoint.scala:110)
at org.apache.spark.SparkEnv$.$anonfun$create$9(SparkEnv.scala:348)
at org.apache.spark.SparkEnv$.registerOrLookupEndpoint$1(SparkEnv.scala:287)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:336)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:191)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:277)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:460)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
at scala.Option.getOrElse(Option.scala:201)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
Maven Profile看起来像这样
<profile>
<id>integration-test</id>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
<resource>
<directory>src/test/resources</directory>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- Setting additionalClasspathElement to solve NoClassDefFoundError error.
See here for more info: https://stackoverflow.com/a/50804520/9914653 -->
<additionalClasspathElements>
<additionalClasspathElement>${project.build.outputDirectory}
</additionalClasspathElement>
</additionalClasspathElements>
<includes>
<include>**/*IntegrationTest.java</include>
<include>**/*IntegrationTest.kt</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
SureFire插件看起来像这样,
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<!-- custom value for `skipTests` allows to skip surefire tests (unit tests)
when running integration tests (failsafe tests) https://stackoverflow.com/a/17932772/9914653
In order to run integration tests: `mvn verify -Pintegration-test -Dskip.surefire.tests`
In order to skip unit tests use -DskipTests as usual: `mvn clean install -DskipTests`.
-->
<skipTests>${skip.surefire.tests}</skipTests>
<excludes>
<exclude>**/*IntegrationTest.java</exclude>
<exclude>**/*IntegrationTest.kt</exclude>
</excludes>
<!-- The arg is needed because of Spark and JDK 17 issue https://stackoverflow.com/a/72724817/9914653 -->
<argLine>--add-exports java.base/sun.nio.ch=ALL-UNNAMED</argLine>
</configuration>
</plugin>
1条答案
按热度按时间eanckbw91#
因此,问题是在配置文件部分缺少一行,与插件
maven-surefire-plugin
相同。<argLine>--add-exports java.base/sun.nio.ch=ALL-UNNAMED</argLine>