如何检查模式的字段是否为字符串数组类型

oaxa6hgo  于 2021-05-27  发布在  Spark
关注(0)|答案(1)|浏览(368)

当前代码为:

schema.fields.foreach(f => {
                            // check if schema field is type of array 
                            // if yes then throw array not supported 
                             if (f.dataType.typeName == "array") {
 throw ArrayDataTypeNotSupportedException(s"${f.name} column is ArrayType, " +
                                  "writing arrays to CSV isn't supported. Please convert this column to a different data type.")
                             }
                            })

但现在我想要这样的情况,如果schema.field.typename是字符串数组或字符串数组,那么字符串数组应该转换为逗号分隔的字符串。如果它不是字符串数组,则抛出array not accepted异常。简言之,我只想支持字符串数组,而不支持其他数组。如果字符串数组之外有任何东西,那么我想传递下面的异常

throw ArrayDataTypeNotSupportedException(s"${f.name} column is ArrayType, " +
                                      "writing arrays to CSV isn't supported. Please convert this column to a different data type.")
1dkrff03

1dkrff031#

您可以做的是进一步深入了解数组类型并检查元素类型。以下代码可以执行此操作:

if (f.dataType == ArrayType) {
val arrayType = f.dataType.asInstanceOf[ArrayType]

if (arrayType.elementType == StringType) {
 throw ArrayDataTypeNotSupportedException(s"${f.name} column is ArrayType, " +
                                  "writing arrays to CSV isn't supported. Please convert this column to a different data type.")
                          }
}

相关问题