如何最有效地将scalaDataframe中的嵌套列字段强制转换/提取到case类?

3j86kqsm  于 2021-05-24  发布在  Spark
关注(0)|答案(0)|浏览(298)

我有一个案例课,比如:
案例类pa(id:long,name:string,level:int,…,path:seq[parentpath])
parentpath类类似于:
case类parentpath(级别:int,id:long,名称:string)
现在,当我从配置单元返回Dataframe并将df转换为pa时,我得到了异常,类转换异常。错误:

java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to com.abc.ParentPath
    at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)*

我正在阅读df如下:

case Success(df) =>
      Some(df.map(row =>
        (row: @unchecked) match {
          case (r: GenericRowWithSchema) => PA(
              r.getAs[Long]("id"),
              r.getAs[String]("name"),
              r.getAs[Int]("level"),
              r.getAs[Seq[(Int, Long, String)]]("path").asInstanceOf[Seq[ParentPath]
            )
          case _ => null

转换df以获取内部case类数据的另一种方法是什么?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题