无法在spark中使用带管道分隔符的str\u to\uMap

wixjitnu 于 2021-07-13 发布在 Spark

关注(0)|答案(2)|浏览(372)

我正在使用

val df5 = spark.sql("select str_to_map('fruits=banana|sports=football','\\|','=') as json_temp")

但产出并不像预期的那样

+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|json_temp                                                                                                                                                                          |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|[f ->, r ->, u ->, i ->, t ->, s ->,  -> , b ->, a ->, n ->, a ->, n ->, a ->, | ->, s ->, p ->, o ->, r ->, t ->, s ->,  -> , f ->, o ->, o ->, t ->, b ->, a ->, l ->, l ->,  ->]|
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

它无法读取分隔符和分隔符。
我希望输出像

fruits -> banana, sports -> football

scala apache-spark apache-spark-sql

来源：https://stackoverflow.com/questions/66274159/not-able-to-use-str-to-map-with-pipe-delimiter-in-spark

2条答案

按热度按时间

x4shl7ld1#

在scala中对正则表达式转义使用四个反斜杠：

spark.sql("select str_to_map('fruits=banana|sports=football','\\\\|','=') as json_temp").show(false)
+--------------------------------------+
|json_temp                             |
+--------------------------------------+
|[fruits -> banana, sports -> football]|
+--------------------------------------+

或者使用带两个反斜杠的三重引号：

spark.sql("""select str_to_map('fruits=banana|sports=football','\\|','=') as json_temp""").show(false)
+--------------------------------------+
|json_temp                             |
+--------------------------------------+
|[fruits -> banana, sports -> football]|
+--------------------------------------+

赞(0）回复(0）举报 2021-07-13

x759pob22#

您需要转义管道角色。你可以把它放进去 [] :

val df5 = spark.sql(
  "select str_to_map('fruits=banana|sports=football','[|]','=') as json_temp"
)

//+--------------------------------------+
//| json_temp                            |
//+--------------------------------------+
//|[fruits -> banana, sports -> football]|
//+--------------------------------------+

赞(0）回复(0）举报 2021-07-13

我来回答

无法在spark中使用带管道分隔符的str\u to\uMap

2条答案

相关问题

热门标签

最新问答