从pandas_udf返回一个pandas系列：

@F.pandas_udf(T.FloatType())
def foo(control_mean: pd.Series, control_sd: pd.Series, pooled_se: pd.Series) -> pd.Series:
    mu_null = 0
    ##Calculate Likelihood of Null
    pdf = stats.norm.pdf(control_mean, mu_null, pooled_se)
    return pd.Series(pdf)

df.withColumn(
    "pdf", 
    foo(
        F.col("control_mean"), 
        F.col("control_sd"), 
        F.col("pooled_se")
    )
).show()

+------------+----------+---------+---+
|control_mean|control_sd|pooled_se|pdf|
+------------+----------+---------+---+
|          25|        20|     0.25|0.0|
|          20|        20|     0.22|0.0|
|          35|        20|     0.67|0.0|
+------------+----------+---------+---+

调试panda_udf的提示

将pandas_udf的返回类型更改为StringType，并返回长度等于行数的序列。对于返回序列的每个元素，可以使用文本格式的调试条目，也可以将其保留为空字符串。
例如：在下面的输出中，我们将打印

• pdf的类型
• pdf的形状
• pdf的字符串表示

@F.pandas_udf(T.StringType())
def foo(control_mean: pd.Series, control_sd: pd.Series, pooled_se: pd.Series) -> pd.Series:
    mu_null = 0
    ##Calculate Likelihood of Null
    pdf = stats.norm.pdf(control_mean, mu_null, pooled_se)
    return pd.Series([str(type(pdf)), str(pdf.shape), str(pdf)])

df.withColumn(
    "pdf", 
    foo(
        F.col("control_mean"), 
        F.col("control_sd"), 
        F.col("pooled_se")
    )
).show(truncate=False)

+------------+----------+---------+-----------------------+
|control_mean|control_sd|pooled_se|pdf                    |
+------------+----------+---------+-----------------------+
|25          |20        |0.25     |<class 'numpy.ndarray'>|
|20          |20        |0.22     |(3,)                   |
|35          |20        |0.67     |[0. 0. 0.]             |
+------------+----------+---------+-----------------------+