您可以先使用ROW_NUMBER()窗口函数枚举分区内的行，然后只选择ROW_NUMBER结果为1的行。
输入示例：

df = spark.createDataFrame([('n', {'m': '1'}), ('n', {'m': '1'})], ['name', 'details'])
df.createOrReplaceTempView("table_name")
df.show()
# +----+--------+
# |name| details|
# +----+--------+
# |   n|{m -> 1}|
# |   n|{m -> 1}|
# +----+--------+

仅提取不同的记录：

df_row_num = spark.sql("""
    WITH cte_row_num AS (
        SELECT name
              ,details 
              ,ROW_NUMBER() OVER (
                  PARTITION BY name
                              ,sort_array(map_keys(details))
                              ,sort_array(map_values(details))
                  ORDER BY name) as row_num
        FROM table_name)
    SELECT name
          ,details 
    FROM cte_row_num
    WHERE row_num = 1
""")

df_row_num.show()
# +----+--------+
# |name| details|
# +----+--------+
# |   n|{m -> 1}|
# +----+--------+

看起来你可以用map_entries把map列转换成结构体数组，然后取一个distinct，再把它转换回map列。
下面是一个工作示例

data_sdf.show()

# +----+--------+
# |name| details|
# +----+--------+
# |   n|{m -> 1}|
# |   n|{m -> 1}|
# +----+--------+

data_sdf.createOrReplaceTempView('data_tbl')

spark.sql('''
    select name, map_from_entries(details_entries) as details
    from (
        select distinct name, sort_array(map_entries(details)) as details_entries
        from data_tbl)
    '''). \
    show()

# +----+--------+
# |name| details|
# +----+--------+
# |   n|{m -> 1}|
# +----+--------+