Spark 2.2 Structured Streaming Stream -静态左外连接问题

5kgi1eie  于 2023-05-29  发布在  Apache
关注(0)|答案(1)|浏览(192)

我似乎错过了一些关于流的东西-Spark 2.2中的静态连接。
手册上说这样的连接是可能的,但是我不能得到正确的语法。奇怪。没有使用水印。

val joinedDs = salesDs
  .join(customerDs, "customerId", joinType="leftOuter")

错误得到如下,但我很确定我有正确的边:

<console>:81: error: overloaded method value join with alternatives:
  (right: org.apache.spark.sql.Dataset[_],joinExprs: 
org.apache.spark.sql.Column,joinType: String)org.apache.spark.sql.DataFrame <and>
  (right: org.apache.spark.sql.Dataset[_],usingColumns: Seq[String],joinType: String)org.apache.spark.sql.DataFrame
 cannot be applied to (org.apache.spark.sql.Dataset[Customer], String, joinType: String)
         .join(customerDs, "customerId", joinType="left_Outer")
          ^
0tdrvxhp

0tdrvxhp1#

由于某些原因,在添加joinType时,我还需要添加Seq。

.join(customerDs, Seq("customerId"), "left_Outer")

相关问题