ncnn Sigmoid layer produces NaN

kadbb459  于 2022-12-31  发布在  其他
关注(0)|答案(2)|浏览(226)

On OSX (x86_64), the following sample code will output correct values for top[0] and top[1] and NaN for top[2]. All values smaller than -90.0f will be NaN.

`
auto sigmoid = ncnn::create_layer("Sigmoid");

auto bottom = ncnn::Mat(16);
    auto top = ncnn::Mat(16);
    
    bottom[0] = 0.0f;
    bottom[1] = 10.0f;
    bottom[2] = -90.0f;

    sigmoid->forward(bottom, top);
    
    std::cout << top[0] << " " << top[1] << " " << top[2] << '\n';

`

7tofc5zh

7tofc5zh1#

ncnn/src/layer/sigmoid.cpp

Line 42 in 624291e

| | ptr[i] = 1.f / (1.f + exp(-ptr[i])); |

reciprocal is optimized to an approx sse version by compiler like

__m128 recip_float4_single(__m128 x)
{
  __m128 res = _mm_rcp_ps(x);
  __m128 muls = _mm_mul_ps(x, _mm_mul_ps(res, res));
  return res =  _mm_sub_ps(_mm_add_ps(res, res), muls);
}

1+exp(90) returns inf
then 1/inf returns -NaN with this approx version

nue99wik

nue99wik2#

Maybe the code should check the input and return 0 directly if the input value is smaller than -90. Will be slower but will remove the NaNs.

相关问题