• 标题：计算误差问题
• 版本、环境信息：

   1）PaddlePaddle版本：2.21
   3）GPU：RTX3090 24G cudnn8.2 cuda 11.6
   4）系统环境：win11，Python3.7

• 训练信息

   1）单机，单卡
   2）24G
   3）Operator信息

• 复现信息：误差变大
• 问题描述：

pytorch的self.query_proj.weight与paddle的self.query_proj.weight比较

from reprod_log import ReprodLogger
  reprod_logger = ReprodLogger()
  reprod_logger.add("logits", self.query_proj.weight.cpu().detach().numpy())
  reprod_logger.save("forward_torch.npy")

  from reprod_log import ReprodLogger
  reprod_logger = ReprodLogger()
  reprod_logger.add("logits", self.query_proj.weight.T.cpu().detach().numpy())
  reprod_logger.save("forward_paddle.npy")

  [2022/01/28 16:51:33] root INFO: logits: 
  [2022/01/28 16:51:33] root INFO: 	mean diff: check passed: True, value: 0.0
  [2022/01/28 16:51:33] root INFO: diff check passed

误差为0，说明权重没有问题

pytorch的self.query_proj.bias与paddle的self.query_proj.bias比较

from reprod_log import ReprodLogger
  reprod_logger = ReprodLogger()
  reprod_logger.add("logits", self.query_proj.bias.cpu().detach().numpy())
  reprod_logger.save("forward_torch.npy")

  from reprod_log import ReprodLogger
  reprod_logger = ReprodLogger()
  reprod_logger.add("logits", self.query_proj.bias.cpu().detach().numpy())
  reprod_logger.save("forward_paddle.npy")

  [2022/01/28 16:58:35] root INFO: logits: 
  [2022/01/28 16:58:35] root INFO: 	mean diff: check passed: True, value: 0.0
  [2022/01/28 16:58:35] root INFO: diff check passed

误差为0，说明权重没有问题

pytorch输入self.query_proj的参数与paddle输入self.query_proj的参数比较
from reprod_log import ReprodLogger
reprod_logger = ReprodLogger()
reprod_logger.add("logits", query_states.cpu().detach().numpy())
reprod_logger.save("forward_paddle.npy")

from reprod_log import ReprodLogger
  reprod_logger = ReprodLogger()
  reprod_logger.add("logits", query_states.cpu().detach().numpy())
  reprod_logger.save("forward_torch.npy")

[2022/01/28 16:55:24] root INFO: logits:
[2022/01/28 16:55:24] root INFO: mean diff: check passed: True, value: 2.6821624032891123e-08
[2022/01/28 16:55:24] root INFO: diff check passed
误差只有2.6821624032891123e-08， 应该没问题

pytorch self.query_proj(query_states)计算后与paddle的self.query_proj(query_states)计算后结果做比较

from reprod_log import ReprodLogger
    reprod_logger = ReprodLogger()
    reprod_logger.add("logits", self.query_proj(query_states).cpu().detach().numpy())
    reprod_logger.save("forward_paddle.npy")

    from reprod_log import ReprodLogger
    reprod_logger = ReprodLogger()
    reprod_logger.add("logits", self.query_proj(query_states).cpu().detach().numpy())
    reprod_logger.save("forward_torch.npy")

    [2022/01/28 17:02:25] root INFO: logits: 
    [2022/01/28 17:02:25] root INFO: 	mean diff: check passed: False, value: 0.00011985769378952682
    [2022/01/28 17:02:25] root INFO: diff check failed

误差直接上0.0001了，这我怎么办

权重误差是0，输入误差是2.6821624032891123e-08，一计算误差就是0.0001多
越是训练误差越大

pytorch 
self.query_proj.weight：
tensor([[ 0.0657, -0.0202,  0.0140,  ...,  0.0427,  0.1542, -0.0205],
        [ 0.0264,  0.0539,  0.0116,  ...,  0.0186, -0.0568,  0.0083],
        [-0.0955, -0.1249,  0.0155,  ..., -0.0858, -0.0699, -0.0850],
        ...,
        [-0.0136,  0.0438,  0.0282,  ..., -0.0606,  0.0027, -0.0060],
        [ 0.0503, -0.0868, -0.0462,  ..., -0.0325,  0.0751, -0.0699],
        [ 0.0268, -0.0392,  0.0881,  ...,  0.0198, -0.0481, -0.0623]],
       requires_grad=True)

self.query_proj.weight.dtype
torch.float32

self.query_proj.bias:
tensor([ 0.0553,  0.0917,  0.2020,  ..., -0.0589,  0.0430, -0.1407],
       requires_grad=True)

self.query_proj.bias.dtype
torch.float32

paddle
self.query_proj.weight：
Tensor(shape=[1536, 1536], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
       [[ 0.06573486,  0.02635193, -0.09545898, ..., -0.01362610,
          0.05026245,  0.02679443],
        [-0.02015686,  0.05389404, -0.12487793, ...,  0.04376221,
         -0.08679199, -0.03921509],
        [ 0.01401520,  0.01164246,  0.01550293, ...,  0.02821350,
         -0.04620361,  0.08813477],
        ...,
        [ 0.04272461,  0.01856995, -0.08581543, ..., -0.06060791,
         -0.03247070,  0.01977539],
        [ 0.15417480, -0.05679321, -0.06988525, ...,  0.00271034,
          0.07507324, -0.04806519],
        [-0.02049255,  0.00827789, -0.08502197, ..., -0.00597382,
         -0.06988525, -0.06231689]])

self.query_proj.bias:
Tensor(shape=[1536], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
       [ 0.05532837,  0.09167480,  0.20202637, ..., -0.05889893,
         0.04296875, -0.14074707])

权重误差是0，输入误差是2.6821624032891123e-08，一计算误差就是0.0001多
越是训练误差越大
这该怎么办，没办法了，训练也不收敛， 直接读取模型预测准确率也比pytorch的少0.07左右