Pytorch的nn.BCEWithLogitsLoss()的行为与nn.BCELoss()完全不同

pnwntuvh  于 2023-04-12  发布在  Git
关注(0)|答案(1)|浏览(137)

我对pytorch完全陌生。我正在参加一个电子课程,并正在尝试使用pytorch。所以我遇到了两个损失函数(使用这两个损失的假设是logits的数值稳定性):
nn.BCEWithLogitsLoss()

nn.BCELoss()
为了对代码和这两个损失函数进行适当的调整,我使用了完全不同的精度曲线!例如,使用**nn.BCELoss()**作为以下代码片段:

model = nn.Sequential(
nn.Linear(D, 1),
nn.Sigmoid()
)

criterion = nn.BCELoss()

准确度图为:enter image description here
对于nn.BCEWithLogitsLoss(),如下所示:

model = nn.Linear(D, 1)
criterion = nn.BCEWithLogitsLoss()

准确度图为:enter image description here
这两个例子的其余代码是相同的。(注意,损失曲线是相似的和体面的)两个片段的倾斜曲线是这样的:enter image description here我不知道是什么原因导致了这个问题(如果我的代码中有bug或者我的pytorch有什么问题。谢谢你的时间,并提前帮助。

dy2hfwbg

dy2hfwbg1#

BCELoss()期望您的输出是概率,即使用sigmoid激活。
BCEWithLogitsLoss()期望您的输出是logits,即没有sigmoid激活。
我想你可能计算错了(比如精度)。这里我给予你一个基于你的代码的简单例子:
概率:

dummy_x = torch.randn(1000,1)
dummy_y = (dummy_x > 0).type(torch.float)

model1 = nn.Sequential(
    nn.Linear(1, 1),
    nn.Sigmoid()
)
criterion1 = nn.BCELoss()
optimizer = torch.optim.Adam(model1.parameters(), 0.001)

def binary_accuracy(preds, y, logits=False):
    if logits:
        rounded_preds = torch.round(torch.sigmoid(preds))
    else:
        rounded_preds = torch.round(preds)
    correct = (rounded_preds == y).float()
    accuracy = correct.sum() / len(y)
    return accuracy

for e in range(2000):
    y_hat = model1(dummy_x)
    loss = criterion1(y_hat, dummy_y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if e != 0 and e % 100==0:
        print(f"Epoch: {e}, Loss: {loss:.4f}")
        print(f"Epoch: {e}, Acc: {binary_accuracy(y_hat, dummy_y)}")

#Result:
Epoch: 100, Loss: 0.5840
Epoch: 100, Acc: 0.5839999914169312
Epoch: 200, Loss: 0.5423
Epoch: 200, Acc: 0.6499999761581421
...
Epoch: 1800, Loss: 0.2862
Epoch: 1800, Acc: 0.9950000047683716
Epoch: 1900, Loss: 0.2793
Epoch: 1900, Acc: 0.9929999709129333

现在用logits

model2 = nn.Linear(1, 1)
criterion2 = nn.BCEWithLogitsLoss()
optimizer2 = torch.optim.Adam(model2.parameters(), 0.001)
for e in range(2000):
    y_hat = model2(dummy_x)
    loss = criterion2(y_hat, dummy_y)
    optimizer2.zero_grad()
    loss.backward()
    optimizer2.step()

    if e != 0 and e % 100==0:
        print(f"Epoch: {e}, Loss: {loss:.4f}")
        print(f"Epoch: {e}, Acc: {binary_accuracy(y_hat, dummy_y, logits=True)}")

#Results: 
Epoch: 100, Loss: 1.1042
Epoch: 100, Acc: 0.007000000216066837
Epoch: 200, Loss: 1.0484
Epoch: 200, Acc: 0.01899999938905239
...
Epoch: 1800, Loss: 0.5019
Epoch: 1800, Acc: 0.9879999756813049
Epoch: 1900, Loss: 0.4844
Epoch: 1900, Acc: 0.9879999756813049

相关问题