我试图根据这个schema用PyTorch实现梯度下降，但不知道如何正确更新权重。这只是一个玩具例子，有两个线性层，隐藏层有两个节点，一个输出。

Learning rate = 0.05;
target output = 1

https://hmkcode.github.io/ai/backpropagation-step-by-step/
Forward
Backward
代码如下：

import torch
    import torch.nn as nn
    import torch.nn.functional as F
    import torch.optim as optim

    class MyNet(nn.Module):

    def __init__(self):
         super(MyNet, self).__init__()
         self.linear1 = nn.Linear(2, 2,  bias=None)
         self.linear1.weight = torch.nn.Parameter(torch.tensor([[0.11, 0.21], [0.12, 0.08]]))
         self.linear2 = nn.Linear(2, 1,  bias=None)
         self.linear2.weight = torch.nn.Parameter(torch.tensor([[0.14, 0.15]]))

    def forward(self, inputs):
         out = self.linear1(inputs)
         out = self.linear2(out)
         return out

    losses = []
    loss_function = nn.L1Loss()
    model = MyNet()
    optimizer = optim.SGD(model.parameters(), lr=0.05)
    input = torch.tensor([2.0,3.0])
    print('weights before backpropagation = ',   list(model.parameters()))
    for epoch in range(1):
       result = model(input )
       loss = loss_function(result , torch.tensor([1.00],dtype=torch.float))
       print('result = ', result)
       print("loss = ",   loss)
       model.zero_grad()
       loss.backward()
       print('gradients =', [x.grad.data  for x in model.parameters()] )
       optimizer.step()
       print('weights after backpropagation = ',   list(model.parameters()))

结果如下：

weights before backpropagation =  [Parameter containing:
    tensor([[0.1100, 0.2100],
            [0.1200, 0.0800]], requires_grad=True), Parameter containing:
    tensor([[0.1400, 0.1500]], requires_grad=True)]

    result =  tensor([0.1910], grad_fn=<SqueezeBackward3>)
    loss =  tensor(0.8090, grad_fn=<L1LossBackward>)

    gradients = [tensor([[-0.2800, -0.4200], [-0.3000, -0.4500]]), 
                 tensor([[-0.8500, -0.4800]])]

    weights after backpropagation =  [Parameter containing:
    tensor([[0.1240, 0.2310],
            [0.1350, 0.1025]], requires_grad=True), Parameter containing:
    tensor([[0.1825, 0.1740]], requires_grad=True)]

正向传递值：

2x0.11 + 3*0.21=0.85 ->  
2x0.12 + 3*0.08=0.48 -> 0.85x0.14 + 0.48*0.15=0.191 -> loss =0.191-1 = -0.809

向后传球：让我们计算w5和w 6（输出节点权重）

w = w - (prediction-target)x(gradient)x(output of previous node)x(learning rate)  
w5= 0.14 -(0.191-1)*1*0.85*0.05= 0.14 + 0.034= 0.174  
w6= 0.15 -(0.191-1)*1*0.48*0.05= 0.15 + 0.019= 0.169

在我的例子中，Torch没有将损失乘以导数，所以我们在更新后得到了错误的权重。对于输出节点，我们得到了新的权重w5，w 6 [0.1825，0.1740]，而它应该是[0.174，0.169]
向后移动以更新输出节点的第一个权重（w5），我们需要计算：(prediction-target)x(gradient)x(output of previous node)x(learning rate)=-0.809*1*0.85*0.05=-0.034。更新了权重w5 = 0.14-(-0.034)=0.174。但pytorch计算的是new weight = 0.1825。它忘了乘以(prediction-target)=-0.809。对于输出节点，我们得到梯度-0.8500和-0.4800。但是我们仍然需要将它们乘以损失0.809和学习率0.05，然后才能更新权重。
做这件事的正确方法是什么？我们是否应该将'loss'作为参数传递给backward()，如下所示：loss.backward(loss)。
这似乎解决了它。但我在文档中找不到任何关于这方面的例子。