ncnn Sigmoid layer produces NaN

kadbb459 于 2022-12-31 发布在其他

关注(0)|答案(2)|浏览(230)

On OSX (x86_64), the following sample code will output correct values for top[0] and top[1] and NaN for top[2]. All values smaller than -90.0f will be NaN.

`
auto sigmoid = ncnn::create_layer("Sigmoid");

auto bottom = ncnn::Mat(16);
    auto top = ncnn::Mat(16);
    
    bottom[0] = 0.0f;
    bottom[1] = 10.0f;
    bottom[2] = -90.0f;

    sigmoid->forward(bottom, top);
    
    std::cout << top[0] << " " << top[1] << " " << top[2] << '\n';

ncnn

来源：https://github.com/Tencent/ncnn/issues/1284

2条答案

按热度按时间

7tofc5zh1#

ncnn/src/layer/sigmoid.cpp

Line 42 in 624291e

| | ptr[i] = 1.f / (1.f + exp(-ptr[i])); |

reciprocal is optimized to an approx sse version by compiler like

__m128 recip_float4_single(__m128 x)
{
  __m128 res = _mm_rcp_ps(x);
  __m128 muls = _mm_mul_ps(x, _mm_mul_ps(res, res));
  return res =  _mm_sub_ps(_mm_add_ps(res, res), muls);
}

1+exp(90) returns inf
then 1/inf returns -NaN with this approx version

赞(0）回复(0）举报 2022-12-31

nue99wik2#

Maybe the code should check the input and return 0 directly if the input value is smaller than -90. Will be slower but will remove the NaNs.

赞(0）回复(0）举报 2022-12-31