你可以分组 struct(Buyer_name, Buyer_state) 以及 struct(CoBuyer_name, CoBuyer_state) 变成一个 Array 然后使用 explode ，如下图：

import org.apache.spark.sql.functions._
import spark.implicits._

val df = Seq(
  ("Bob", "CA", "Joe", "CA", 20, "010119"),
  ("Stacy", "IL", "Jamie", "IL", 50, "020419")
).toDF("Buyer_name", "Buyer_state", "CoBuyer_name", "CoBuyer_state", "Price", "Date")

df.
  withColumn("Buyers", array(
    struct($"Buyer_name".as("_1"), $"Buyer_state".as("_2")),
    struct($"CoBuyer_name".as("_1"), $"CoBuyer_state".as("_2"))
  )).
  withColumn("Buyer", explode($"Buyers")).
  select(
    $"Buyer._1".as("Buyer_name"), $"Buyer._2".as("Buyer_state"), $"Price", $"Date"
  ).show
// +----------+-----------+-----+------+
// |Buyer_name|Buyer_state|Price|  Date|
// +----------+-----------+-----+------+
// |       Bob|         CA|   20|010119|
// |       Joe|         CA|   20|010119|
// |     Stacy|         IL|   50|020419|
// |     Jamie|         IL|   50|020419|
// +----------+-----------+-----+------+

对我来说，这听起来像是一个可以通过 union scala中的函数：

val df = Seq(
  ("Bob", "CA", "Joe", "CA", 20, "010119"),
  ("Stacy", "IL", "Jamie", "IL", 50, "020419")
).toDF("Buyer_name", "Buyer_state", "CoBuyer_name", "CoBuyer_state", "Price", "Date")

val df_new = df.select("Buyer_name", "Buyer_state", "Price", "Date").union(df.select("CoBuyer_name", "CoBuyer_state", "Price", "Date"))

df_new.show

感谢leo提供了我使用的Dataframe定义。