Paddle embedding id使用顺序编码可正确训练,而使用稀疏签名 f64模型基本不变(auc 差+不动)

v2g6jxz6  于 2021-11-30  发布在  Java
关注(0)|答案(1)|浏览(255)

代码

class LRLayer(nn.Layer):
    def __init__(self, params):
    ¦   super(LRLayer, self).__init__()
    ¦   self.init_value_ = 0.0
    ¦   self.reg = 0.001

    def forward(self, inputs, is_infer=False):
    ¦   input_idxs, input_weights = inputs

    ¦   embs = []
    ¦   for idx in input_idxs:
    ¦   ¦   emb = fluid.layers.embedding(idx,
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   [10240000, 1],
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   is_sparse=True,
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   is_distributed=True,
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   param_attr=fluid.ParamAttr(
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   name="emb",
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   # initializer=paddle.nn.initializer.Constant(value=0)
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   # initializer=paddle.nn.initializer.TruncatedNormal(mean=0.0, std=self.init_value_),
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   # regularizer=paddle.regularizer.L1Decay(self.reg)
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ))
    ¦   ¦   #paddle.static.Print(idx)
    ¦   ¦   embs.append(emb)

    ¦   #paddle.static.Print(input_weights)

    ¦   embs = paddle.concat(x=embs, axis=1)

    ¦   self.b_linear = paddle.create_parameter(shape=[1],
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   dtype='float32',
    ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   ¦   default_initializer=paddle.nn.initializer.Constant(value=0))
    ¦   y_first_order = paddle.sum(embs * input_weights, 1, keepdim=True)

    ¦   predict = paddle.nn.functional.sigmoid(y_first_order + self.b_linear)
    ¦   return predict, y_first_order, embs
  • 训练信息

   1)CPU 多机

idx 生成方式

  1. OK
feasign = sign_dict.get(txt, 0)
  1. 不 ok
feasign = sign.sign(txt)

运行方式

fleetrun --worker_num=5 --server_num=1 trainer.py ps -c wd_config.yaml
qjp7pelc

qjp7pelc1#

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

相关问题