Paddle 用CrossEntropyLoss计算loss不下降

avkwfej4  于 2021-11-30  发布在  Java
关注(0)|答案(6)|浏览(383)

PaddlePaddle版本:2.1.2

import sys
sys.path.append('/home/aistudio/external-libraries')
sys.path.append('/home/aistudio/vc')
import paddle
from model import DomainClassifier
from utility import speakers
from data_loader import data_loader
import paddle.nn as nn

dloader = data_loader('/home/aistudio/vc/data/processed', batch_size=1, mode='train',num_workers=1)
C = DomainClassifier(len(speakers))
c_optimizer = paddle.optimizer.Adam(learning_rate=0.001,parameters=C.parameters(),beta1=0.5,beta2=0.999)

data_iter = iter(dloader)
x_real, speaker_idx_org, label_org = next(data_iter)
speaker_idx_org = paddle.reshape(speaker_idx_org,[-1])

for i in range(100):
    CELoss = nn.CrossEntropyLoss()
    cls_real = C(x_real)
    cls_loss_real = CELoss(input=cls_real, label=speaker_idx_org)

    cls_loss_real.backward()
    c_optimizer.step()
    c_optimizer.clear_grad()

    loss = cls_loss_real.item()
    print(loss)

loss一直是5.308267593383789

fnatzsnv

fnatzsnv1#

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

dfuffjeb

dfuffjeb2#

您好,您可以参考这个链接中 https://aistudio.baidu.com/aistudio/education/group/info/1297/content 课节13 一个案例吃透深度学习(中)项目5. 【手写数字识别】之损失函数案例,里面使用了paddle.nn.functional.cross_entropy api 进行cross entropy损失计算。您可以尝试使用这个api计算loss或者将api替换为CrossEntropyLoss进行尝试

hwazgwia

hwazgwia3#

更换paddle.nn.functional.cross_entropy后loss一直还是5.308267593383789

7jmck4yq

7jmck4yq4#

您好,我阅读代码时发现 x_real, speaker_idx_org, label_org = next(data_iter) 数据读取代码没有放在训练循环中,会导致训练过程中一直使用同样的数据训练,但在这种情况下,loss也应该是会更新的。目前只看贴出来的代码不能找到是什么问题,如果问题还没有解决,麻烦把代码和数据都同步给我吧

33qvvth1

33qvvth15#

测试的时候使用paddle.ones生成的来代替数据集的数据
test.py

import paddle
import paddle.nn as nn
import paddle.nn.functional as F

class Down2d(nn.Layer):
    def __init__(self,in_channel ,out_channel, kernel, stride, padding):
        super(Down2d, self).__init__()
        self.c1 = nn.Conv2D(in_channel, out_channel, kernel_size=kernel, stride=stride, padding=padding)
        self.n1 = nn.InstanceNorm2D(out_channel)
        self.c2 = nn.Conv2D(in_channel, out_channel, kernel_size=kernel, stride=stride, padding=padding)
        self.n2 = nn.InstanceNorm2D(out_channel)

    def forward(self, x):
        x1 = self.c1(x)
        x1 = self.n1(x1)

        x2 = self.c2(x)
        x2 = self.n2(x2)

        x3 = x1 * F.sigmoid(x2)
        return x3

class DomainClassifier(nn.Layer):
    def __init__(self,num):
        super(DomainClassifier, self).__init__()
        self.main = nn.Sequential(
            Down2d(1, 8, (4,4), (2,2), (5,1)),
            Down2d(8, 16, (4,4), (2,2), (1,1)),
            Down2d(16, 32, (4,4), (2,2), (0,1)),
            Down2d(32, 16, (3,4), (1,2), (1,1)),
            nn.Conv2D(16, num, (1,4), (1,2), (0,1)),
            nn.AvgPool2D((1,16)),
            nn.LogSoftmax()
        )

    def forward(self, x):
        x = x[:, :, 0:8, :]
        x = self.main(x)
        x = paddle.reshape(x,[x.shape[0], x.shape[1]])
        return x

C = DomainClassifier(202)
c_optimizer = paddle.optimizer.Adam(learning_rate=0.001,parameters=C.parameters(),beta1=0.5,beta2=0.999)
x_real = paddle.ones([1,1,36,512])
speaker_idx_org = paddle.to_tensor([1],dtype='int64')

for i in range(10):
    CELoss = nn.CrossEntropyLoss()
    cls_real = C(x_real)
    cls_loss_real = CELoss(input=cls_real, label=speaker_idx_org)
    #cls_loss_real=paddle.nn.functional.cross_entropy(input=cls_real, label=speaker_idx_org)

    cls_loss_real.backward()
    c_optimizer.step()
    c_optimizer.clear_grad()

    loss = cls_loss_real.item()
    print(loss)

输出

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
5.308267593383789
lo8azlld

lo8azlld6#

DomainClassifier这个class,网络定义时使用了nn.LogSoftmax(),CrossEntropyLoss() 这个api的介绍中写了“该OP计算输入input和标签label间的交叉熵损失 ,它结合了 LogSoftmax 和 NLLLoss 的OP计算,可用于训练一个 n 类分类器。”,所以LogSoftmax使用两遍,导致DomainClassifier的输出全0,去掉网络定义中的nn.LogSoftmax(),loss就可以正常变化了

相关问题