逻辑有点复杂。
您可以在中执行以下操作 pyspark Pypark公司

from pyspark.sql import functions as F
from pyspark.sql import Window
import sys
windowSpec = Window.partitionBy("type").orderBy("type")
df = df.withColumn('result', F.lag(df.col2, 1).over(windowSpec) - df.col1 + df.col3)
df = df.withColumn('result', F.when(df.result.isNull(), F.lit(0)).otherwise(df.result))
df = df.withColumn('result', F.sum(df.result).over(windowSpec.rowsBetween(-sys.maxsize, -1)) + df.result)
df = df.withColumn('result', F.when(df.result.isNull(), F.lit(0)).otherwise(df.result))

斯卡拉

import org.apache.spark.sql.expressions._
import org.apache.spark.sql.functions._
val windowSpec = Window.partitionBy("type").orderBy("type")
df.withColumn("result", lag("col2", 1).over(windowSpec) - $"col1"+$"col3")
  .withColumn("result", when($"result".isNull, lit(0)).otherwise($"result"))
  .withColumn("result", sum("result").over(windowSpec.rowsBetween(Long.MinValue, -1)) +$"result")
  .withColumn("result", when($"result".isNull, lit(0)).otherwise($"result"))

你应该得到以下结果。

+----+----+----+----+------+
|type|col1|col2|col3|result|
+----+----+----+----+------+
|1   |0   |41  |0   |0.0   |
|1   |27  |0   |0   |14.0  |
|1   |1   |0   |0   |13.0  |
|1   |183 |0   |2   |-168.0|
|3   |0   |126 |0   |0.0   |
|3   |2   |0   |1   |125.0 |
|3   |4   |0   |0   |121.0 |
|3   |5   |0   |0   |116.0 |
|2   |null|0   |0   |0.0   |
|2   |null|10  |0   |0.0   |
+----+----+----+----+------+

编辑
第一个 withColumn 应用公式 prev(col2) - col1 + col3 . 第二个 withColumn 将的null更改为0 result 列。第三个 withColumn 用于累计和，即将所有值相加，直到结果列的当前行为止。所以三个 withColumn 相当于 prev(col2) + prev(results) 1 col1 + col3 . 最后 withColumn 正在将中的空值更改为0 result 列。