pytorch 由原地操作修改的变量之一

hl0ma9xz  于 12个月前  发布在  其他
关注(0)|答案(4)|浏览(136)

我是Pytorch的新手,这里我想用这个模型来生成一些图像,但是因为这是在Pytorch 1.5之前写的,因为梯度计算已经被修复了,所以这是错误消息。

RuntimeError: one of the variables needed for gradient computation has been 
modified by an inplace operation: [torch.cuda.FloatTensor [1, 512, 4, 4]] 
is at version 2; expected version 1 instead. 
Hint: enable anomaly detection to find the operation that 
failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

字符串
我看了过去的例子,不知道这里的问题是什么,我相信它是发生在这个地区,但我不知道在哪里!任何帮助将不胜感激!

def process(self, images, edges, masks):
self.iteration += 1

    # zero optimizers
    self.gen_optimizer.zero_grad()
    self.dis_optimizer.zero_grad()

    # process outputs
    outputs = self(images, edges, masks)
    gen_loss = 0
    dis_loss = 0

    # discriminator loss
    dis_input_real = torch.cat((images, edges), dim=1)
    dis_input_fake = torch.cat((images, outputs.detach()), dim=1)
    dis_real, dis_real_feat = self.discriminator(dis_input_real)        # in: (grayscale(1) + edge(1))
    dis_fake, dis_fake_feat = self.discriminator(dis_input_fake)        # in: (grayscale(1) + edge(1))
    dis_real_loss = self.adversarial_loss(dis_real, True, True)
    dis_fake_loss = self.adversarial_loss(dis_fake, False, True)
    dis_loss += (dis_real_loss + dis_fake_loss) / 2

    # generator adversarial loss
    gen_input_fake = torch.cat((images, outputs), dim=1)
    gen_fake, gen_fake_feat = self.discriminator(gen_input_fake)        # in: (grayscale(1) + edge(1))
    gen_gan_loss = self.adversarial_loss(gen_fake, True, False)
    gen_loss += gen_gan_loss

    # generator feature matching loss
    gen_fm_loss = 0
    for i in range(len(dis_real_feat)):
        gen_fm_loss += self.l1_loss(gen_fake_feat[i], dis_real_feat[i].detach())
    gen_fm_loss = gen_fm_loss * self.config.FM_LOSS_WEIGHT
    gen_loss += gen_fm_loss

    # create logs
    logs = [
        ("l_d1", dis_loss.item()),
        ("l_g1", gen_gan_loss.item()),
        ("l_fm", gen_fm_loss.item()),
    ]

    return outputs, gen_loss, dis_loss, logs

def forward(self, images, edges, masks):
    edges_masked = (edges * (1 - masks))
    images_masked = (images * (1 - masks)) + masks
    inputs = torch.cat((images_masked, edges_masked, masks), dim=1)
    outputs = self.generator(inputs)                                    # in: [grayscale(1) + edge(1) + mask(1)]
    return outputs

def backward(self, gen_loss=None, dis_loss=None):
    if dis_loss is not None:
        dis_loss.backward()
    self.dis_optimizer.step()

    if gen_loss is not None:
        gen_loss.backward()
    self.gen_optimizer.step()


谢谢你,谢谢

yacmzcpb

yacmzcpb1#

你不能一次计算出两个节点和发电机的损失,并像这样背靠背地进行反向传播:

if dis_loss is not None:
    dis_loss.backward()
self.dis_optimizer.step()

if gen_loss is not None:
    gen_loss.backward()
self.gen_optimizer.step()

字符串
原因如下:当你调用self.dis_optimizer.step()的时候,你实际上是在原地修改了gen_loss的参数,这和你试图反向传播的gen_loss的参数是一样的。这是不可能的。
你必须计算dis_loss的反向传播,更新权重,并清除梯度。只有这样,你才能用新更新的权重计算gen_loss。最后,在生成器上反向传播。
这个tutorial是典型GAN培训的一个很好的演练。

pgvzfuti

pgvzfuti2#

这可能不是你问题的确切答案,但我在尝试使用“自定义”分布式优化器时遇到了这个问题。例如,我使用Cherry的优化器,同时意外地将模型移动到DDP模型。一旦我根据Cherry的工作方式将模型移动到设备,我就不再遇到这个问题。
上下文:https://github.com/learnables/learn2learn/issues/263

dzhpxtsq

dzhpxtsq3#

这对我很有效。更多细节请看here

def backward(self, gen_loss=None, dis_loss=None):
    if dis_loss is not None:
        dis_loss.backward(retain_graph=True)  # modified here
    self.dis_optimizer.step()

    if gen_loss is not None:
        gen_loss.backward()
    self.gen_optimizer.step()

字符串

tjjdgumg

tjjdgumg4#

我可能会迟到一点,但以下是我的工作:

def backward(self, gen_loss=None, dis_loss=None):
    if dis_loss is not None:
       dis_loss.backward()

    if gen_loss is not None:
       gen_loss.backward(retain_graph=True)
    self.dis_optimizer.step()
    self.gen_optimizer.step()

字符串

相关问题