我正在阅读他的论文[1]，我有一个来自here的实现。在代码的某个点上，Hessian矩阵的对角线近似为函数set_hessian，您可以在下面找到。在set_hessian()的结尾，提到了# approximate the expected values of z*(H@z)。但是，当我打印p.hess时，我得到

tensor([[[[ 2.3836e+01,  1.4929e+01,  4.1799e+00],
          [-1.6726e+01,  6.3954e+00, -5.1418e+00],
          [ 2.2580e+01, -1.1916e+01, -2.5049e+00]],
         [[-1.8261e+01,  8.7626e+00,  1.8244e+00],
          [-1.0819e+01, -2.9184e-01,  1.1601e+01],
          [-1.6267e+01,  5.6232e+00,  3.4282e+00]],
         ....
         [[-3.1088e+01,  4.3013e+01, -4.2021e+01],
          [ 1.5338e+01, -2.9806e+01, -3.0049e+01],
          [-9.8979e+00, -2.2835e+00, -6.0549e+00]]]], device='cuda:0')

p.hess如何被认为是Hessian的对角近似？我试图理解这个结构的原因是因为我想得到最小的特征值，对角矩阵的逆，以及Hessian和梯度之间的乘积，这是一个向量。我们知道对角矩阵的最小特征值是对角线的最小元素，而对角矩阵的逆可以通过将对角线的元素求逆来计算。你能不能请谁来解释一下p.hess的结构？

@torch.no_grad()
    def set_hessian(self):
        """
        Computes the Hutchinson approximation of the hessian trace and accumulates it for each trainable parameter.
        """

        params = []
        for p in filter(lambda p: p.grad is not None, self.get_params()):
            if self.state[p]["hessian step"] % self.update_each == 0:  # compute the trace only each `update_each` step
                params.append(p)
            self.state[p]["hessian step"] += 1

        if len(params) == 0:
            return

        if self.generator.device != params[0].device:  # hackish way of casting the generator to the right device
            self.generator = torch.Generator(params[0].device).manual_seed(2147483647)

        grads = [p.grad for p in params]

        for i in range(self.n_samples):
            zs = [torch.randint(0, 2, p.size(), generator=self.generator, device=p.device,
                                dtype=torch.float32) * 2.0 - 1.0 for p in params]  # Rademacher distribution {-1.0, 1.0}
            h_zs = torch.autograd.grad(grads, params, grad_outputs=zs, only_inputs=True,
                                       retain_graph=i < self.n_samples - 1)
            for h_z, z, p in zip(h_zs, zs, params):
                p.hess += h_z * z / self.n_samples  # approximate the expected values of z*(H@z)

[1]- 你好一种用于机器学习的自适应二阶优化器