pyspark 为什么阅读Excel文件不适用于Crealytics版本spark-excel_2.12-3.5.0_0.20.1

rn0zuynd  于 11个月前  发布在  Spark
关注(0)|答案(1)|浏览(155)

我可以使用Crealytics库spark-excel_2.12-3.4.1_0.19.0读取Excel文件数据,但无法使用最新版本spark-excel_2.12-3.5.0_0.20.1执行相同的代码。
我尝试了下面的代码,但没有一个代码与最新的库spark-excel_2.12-3.5.0_0.20.1。

df=spark.read.format("com.crealytics.spark.excel").option("inferschema",True).option("header", True).option("ignoreLeadingWhiteSpace", "true").option("ignoreTrailingWhiteSpace", "true").option("keepUndefinedRows", True).option("dataAddress","'SHEET_NAME'!").load(path)
   
df=spark.read.format("excel").option("inferschema",True).option("header", True).option("ignoreLeadingWhiteSpace", "true").option("ignoreTrailingWhiteSpace", "true").option("keepUndefinedRows", True).option("dataAddress","'SHEET_NAME'!").load(path)

字符串
当使用最新的Crealytics时,它会抛出以下错误

Py4JJavaError: An error occurred while calling o412.load.
: java.lang.ClassNotFoundException: 
Failed to find data source: excel. Please find packages at
https://spark.apache.org/third-party-projects.html
       
    at org.apache.spark.sql.errors.QueryExecutionErrors$.failedToFindDataSourceError(QueryExecutionErrors.scala:837)
    at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:744)
    at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:794)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:328)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:237)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
    at py4j.Gateway.invoke(Gateway.java:306)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:195)
    at py4j.ClientServerConnection.run(ClientServerConnection.java:115)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: excel.DefaultSource
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
    at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:730)
    at scala.util.Try$.apply(Try.scala:213)
    at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:730)
    at scala.util.Failure.orElse(Try.scala:224)
    at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:730)
    ... 15 more


请注意,在Databrciks Python cell中使用scala 2.x版本执行上述代码
我的问题是,为什么升级到最新版本的crealytics时不工作?

tjjdgumg

tjjdgumg1#

很抱歉浪费你的时间在这里。我发现有一个与图书馆的问题,它被报告。这里是相同的链接
https://github.com/crealytics/spark-excel/issues/789

相关问题