pandas 用连续数字填充非连续缺失

sg3maiej  于 2023-03-11  发布在  其他
关注(0)|答案(2)|浏览(134)

对于给定的 Dataframe ...

data = pd.DataFrame([[1., 6.5], [1., np.nan],[5, 3], [6.5, 3.], [2, np.nan]])

看起来像这样。

0       1
0   1.0     6.5
1   1.0     NaN
2   5.0     3.0
3   6.5     3.0
4   2.0     NaN

...我想创建第三列,其中第二列的所有缺失值都用连续数字替换。因此,结果应该如下所示:

0       1     2
0   1.0     6.5   NaN
1   1.0     NaN   1
2   5.0     3.0   NaN
3   6.5     3.0   NaN
4   2.0     NaN   2

(my Dataframe 具有更多的行,因此设想第二列中有70个缺失,从而第三列中的最后一个数字将是70)

如何创建第三列?

oknwwptz

oknwwptz1#

您可以这样做,我冒昧地重命名了列,以避免混淆我选择的内容,您可以使用以下命令对 Dataframe 执行相同操作:

data = data.rename(columns={0:'a',1:'b'})

In [41]:

data.merge(pd.DataFrame({'c':range(1,len(data[data.b.isnull()]) + 1)}, index=data[data.b.isnull()].index),how='left', left_index=True, right_index=True)
Out[41]:
     a    b   c
0  1.0  6.5 NaN
1  1.0  NaN   1
2  5.0  3.0 NaN
3  6.5  3.0 NaN
4  2.0  NaN   2

[5 rows x 3 columns]

下面是对这一行的一些解释:

# we want just the rows where column 'b' is null:
data[data.b.isnull()]

# now construct a dataset of the length of this dataframe starting from 1:
range(1,len(data[data.b.isnull()]) + 1) # note we have to add a 1 at the end

# construct a new dataframe from this and crucially use the index of the null values:
pd.DataFrame({'c':range(1,len(data[data.b.isnull()]) + 1)}, index=data[data.b.isnull()].index)

# now perform a merge and tell it we want to perform a left merge and use both sides indices, I've removed the verbose dataframe construction and replaced with new_df here but you get the point
data.merge(new_df,how='left', left_index=True, right_index=True)

编辑

你也可以用@Karl.D的建议换一种方式:

In [56]:

data['c'] = data['b'].isnull().cumsum().where(data['b'].isnull())
data
Out[56]:
     a    b   c
0  1.0  6.5 NaN
1  1.0  NaN   1
2  5.0  3.0 NaN
3  6.5  3.0 NaN
4  2.0  NaN   2

[5 rows x 3 columns]

计时还表明,卡尔的方法对于较大的数据集会更快,但我会对此进行描述:

In [57]:

%timeit data.merge(pd.DataFrame({'c':range(1,len(data[data.b.isnull()]) + 1)}, index=data[data.b.isnull()].index),how='left', left_index=True, right_index=True)
%timeit data['c'] = data['b'].isnull().cumsum().where(data['b'].isnull())
1000 loops, best of 3: 1.31 ms per loop
1000 loops, best of 3: 501 µs per loop
3ks5zfa0

3ks5zfa02#

def function1(dd:pd.DataFrame):
    return dd.assign(col2=(dd['1'].isna()).cumsum()) if dd.name else dd
df1.groupby(df1['1'].isna()).apply(function1)

输出:

0    1  col2
0  1.0  6.5   NaN
1  1.0  NaN   1.0
2  5.0  3.0   NaN
3  6.5  3.0   NaN
4  2.0  NaN   2.0

相关问题