我想在Spark Scala中读取一个名为monthlyPurchaseFile{202205}-May.TXT
的文件
我使用的代码如下:val df = spark.read.text("handel_special_ch/monthlyPurchaseFile{202205}-May.TXT"
但我有以下几点例外:
org.apache.spark.sql.AnalysisException: Path does not exist: file:/home/hdp_batch_datalake_dev/handel_special_ch/monthlyPurchaseFile{202205}-May.TXT
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$checkAndGlobPathIfNecessary$3(DataSource.scala:792)
at org.apache.spark.util.ThreadUtils$.$anonfun$parmap$2(ThreadUtils.scala:372)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
请建议我如何阅读名称中包含字符{
、}
的文件。
2条答案
按热度按时间flvlnr441#
传递给
spark.read.text
方法的path
被视为正则表达式。因为{
和}
是特殊字符,所以Spark会尝试根据该表达式匹配路径。您可以使用?
字符来匹配任何字符,因此以下操作应该可以:gdx19jrr2#
字符
\\
用作转义序列。因此,使用下面的代码可以达到预期效果,并解决问题:val df = spark.read.text("handel_special_ch/monthlyPurchaseFile\\{202205\\}-May.TXT"