试试这个-

加载测试数据

val df = Seq(("p1", """{"a.id": 1, "b": 2}"""), ("p2", """{"a.id": 3}"""))
      .toDF("p_id", "p_meta")
    df.show(false)
    df.printSchema()
    /**
      * +----+-------------------+
      * |p_id|p_meta             |
      * +----+-------------------+
      * |p1  |{"a.id": 1, "b": 2}|
      * |p2  |{"a.id": 3}        |
      * +----+-------------------+
      *
      * root
      * |-- p_id: string (nullable = true)
      * |-- p_meta: string (nullable = true)
      */

使用get\u json\u对象，如下所示

df.withColumn("x", get_json_object($"p_meta", "$['a.id']"))
      .show(false)
    /**
      * +----+-------------------+---+
      * |p_id|p_meta             |x  |
      * +----+-------------------+---+
      * |p1  |{"a.id": 1, "b": 2}|1  |
      * |p2  |{"a.id": 3}        |3  |
      * +----+-------------------+---+
      */

您也可以使用json\u元组

df.withColumn("x", json_tuple($"p_meta", "a.id"))
      .show(false)
    /**
      * +----+-------------------+---+
      * |p_id|p_meta             |x  |
      * +----+-------------------+---+
      * |p1  |{"a.id": 1, "b": 2}|1  |
      * |p2  |{"a.id": 3}        |3  |
      * +----+-------------------+---+
      */

而不是 get_json_object() 尝试使用json\u tuple（）函数。 Example: ```
df.show(false)
//+---------------------+
//|c |
//+---------------------+
//|{"id.i":1,"name":"a"}|
//+---------------------+

df.withColumn("gjo",json_tuple(col("c"),"id.i")).show(false)
//+---------------------+---+
//|c |gjo|
//+---------------------+---+
//|{"id.i":1,"name":"a"}|1 |
//+---------------------+---+

//we can keep multiple json keys
sql("""select json_tuple(c,"id.i","name") from tmp""").show(false)
//+---+---+
//|c0 |c1 |
//+---+---+
//|1 |a |
//+---+---+
``` Option2: 使用 from_json() 函数读取json字符串 StructType 架构。

import org.apache.spark.sql.types._
import org.apache.spark.sql.functions._

val sch=new StructType().add("id.i",IntegerType).add("name",StringType)

df.withColumn("fj",from_json(col("c"),sch)).select("fj.*")show()
//+----+----+
//|id.i|name|
//+----+----+
//|   1|   a|
//+----+----+

然后可以使用替换列名 .map , .toDF , .withColumnRenamed() 功能。