scala—从任何列为null或空的dataframe中删除记录

mbyulnm0  于 2021-05-27  发布在  Spark
关注(0)|答案(2)|浏览(511)

是否有任何方法可以从列值为null或空的Dataframe中删除记录?

+---+-------+--------+-------------------+-----+----------+
|id |zipcode|type    |city               |state|population|
+---+-------+--------+-------------------+-----+----------+
|1  |704    |STANDARD|                   |PR   |30100     |
|2  |704    |        |PASEO COSTA DEL SUR|PR   |          |
|3  |76166  |UNIQUE  |CINGULAR WIRELESS  |TX   |84000     |
+---+-------+--------+-------------------+-----+----------+

我希望输出为:

+---+-------+------+-----------------+-----+----------+
|id |zipcode|type  |city             |state|population|
+---+-------+------+-----------------+-----+----------+
|4  |76166  |UNIQUE|CINGULAR WIRELESS|TX   |84000     |
+---+-------+------+-----------------+-----+----------+
pod7payv

pod7payv1#

试试这个:

df_name.na.drop()
  .show(false)

希望对你有帮助。。。

uemypmqf

uemypmqf2#

试试这个:

df
  .na.replace(df.columns,Map("" -> null)) // convert empty strings with null
  .na.drop() // drop nulls and NaNs
  .show()

相关问题