您可以对两个 Dataframe 执行左连接，并使用coalesce函数从df1中选择Age、Year（如果不为空），否则从df2中选择
输入：

import spark.implicits._

val data1 = Seq(
        (1, "Alice", 20, 95, "X"),
        (2, "Bob", 30, 96, "Y"),
        (3, "Jack", 10, 98, "Z"),
        (4, "Will", 25, 99, "A")
      )
val df1 = spark.sparkContext.parallelize(data1).toDF("ID", "Name", "Age", "Year", "Area")

val data2 = Seq(
        (10, "Alice", 20, 95, "US"),
        (20, "Bob", 30, 96, "UK"),
        (3, "Jack", 30, 24, "DE"),
        (4, "Will", 40, 25, "ES")
      )
val df2 = spark.sparkContext.parallelize(data2).toDF("ID", "Name", "Age", "Year", "Country")

连接两个 Dataframe ：

df2.join(df1.select("ID", "Age", "Year"), "ID", "left_outer")
    .select(df2("ID"), 
            df2("Name"), 
            coalesce(df1("Age"), df2("Age")).as("Age"), 
            coalesce(df1("Year"), df2("Year")).as("Year"), 
            df2("Country")) 
    .show()

输出：

+---+-----+---+----+-------+
| ID| Name|Age|Year|Country|
+---+-----+---+----+-------+
| 10|Alice| 20|  95|     US|
| 20|  Bob| 30|  96|     UK|
|  3| Jack| 10|  98|     DE|
|  4| Will| 25|  99|     ES|
+---+-----+---+----+-------+