您可以从您提供的 Dataframe 中生成预期的输出，假设只有两个样本，以防数字重复。

data = {'number': ['1234', '1234', '4567', '6789', '2423', '2423'],
    'reason': ['del', 'undel', 'del', 'undel', 'del', 'undel']} 
df = pd.DataFrame(data, columns=['number', 'reason'])

unique_vals = df.number.value_counts().to_dict()

for index, item in enumerate(df.iterrows()):
    nbr = item[1]['number']
    filter_check = unique_vals[nbr]
    indexes = df[ df['number'] == nbr ]['reason'].index.to_list()
    if filter_check == 2:
        if 'to_filter' not in df.columns:
             df.insert(2, "to_filter", pd.Series([1, 0], index=[idx for idx in indexes]), allow_duplicates= False)
        else:
            df['to_filter'].iloc[indexes[0]] = 1
            df['to_filter'].iloc[indexes[1]] = 0
        
    elif filter_check == 1:
        df['to_filter'].iloc[indexes[0]] = 1

其输出：

number  reason  to_filter
0   1234    del 1.0
1   1234    undel   0.0
2   4567    del 1.0
3   6789    undel   1.0
4   2423    del 1.0
5   2423    undel   0.0

下面是我对窗口函数的尝试
当你最多有2条记录，一条带del，第二条带undel时，它只适用于描述的场景。如果有重复的记录，它需要调整

import datetime
from pyspark.sql import Window
import pyspark.sql.functions as F

x = [
    (1234, "del"),
    (1234, "undel"),
    (4567, "del"),
    (6789, "undel"),
    (2423, "del"),
    (2423, "undel"),
]
df = spark.createDataFrame(x, schema=["number", "reason"])

window = Window.partitionBy("number").orderBy(F.col("reason").asc())
dfWithRowNumber = df.withColumn("row_number", F.row_number().over(window))
dfWithToFilterColumn = dfWithRowNumber.withColumn(
    "toFilter", F.when(F.col("row_number") == F.lit(1), F.lit(1)).otherwise(F.lit(0))
).drop("row_number")

dfWithToFilterColumn.show()

输出为：

+------+------+--------+
|number|reason|toFilter|
+------+------+--------+
|  1234|   del|       1|
|  1234| undel|       0|
|  2423|   del|       1|
|  2423| undel|       0|
|  4567|   del|       1|
|  6789| undel|       1|
+------+------+--------+

窗口功能是你的朋友，如果你想过滤做以下

(df.withColumn('x', dense_rank().over(Window.partitionBy('number').orderBy(monotonically_increasing_id())))#Create colum that numbers the reasons in each group
 .where((col('x')==1))#Filter out duplicated reasons in each group
 .drop('x')#Drop the filter column
).show()

+------+------+
|number|reason|
+------+------+
|  1234|   del|
|  2423|   del|
|  4567|   del|
|  6789| undel|
+------+------+

如果要显示0和1，请执行以下操作

(df.withColumn('x', dense_rank().over(Window.partitionBy('number').orderBy(monotonically_increasing_id())))#Create colum that numbers the reasons in each group
 .withColumn('x', when(col('x')==1,1).otherwise(0))#Filter out duplicated reasons in each group
 
).show()

+------+------+---+
|number|reason|  x|
+------+------+---+
|  1234|   del|  1|
|  1234| undel|  0|
|  2423|   del|  1|
|  2423| undel|  0|
|  4567|   del|  1|
|  6789| undel|  1|
+------+------+---+