detail | 详细描述 | 詳細な説明
Hi,
I'm experiencing some really bad memory usage issues on running ncnn on ARM devices (Android and iOS).
The model is only 65.4MB big, but right after loading params and bin on mobile, the memory usage grows up to 400-500MB (cannot give you exact number because Android Studio profiler is shitty).
In the past I used the same model with TFLite and MLCore and I never noticed this memory usage.
Is there any workaround to decrease memory usage on ARM? I tried messing with opt.use_packing_layout and opt.use_int8_packed options, but nothing helped so far.
Please let me know if I'm missing something, is this memory usage expected?
1条答案
按热度按时间nbewdwxp1#
after a lot of digging I managed to solve my issue setting to false
opt.use_winograd_convolution.After this change, the memory usage decreased by ~350MB, but the funniest thing is that the inference is FASTER. So basically having that option enabled makes my network slower and increases the memory usage by A LOT (making it unusable on old devices).
I have to say that without this option my model eats less memory than Tensorflow Lite on Android and MLCore on iOS, while having more or less the same speed running on CPU with OpenMp enabled (so, good job NCNN).
Now the question is, why is this option enabled by default? Probably there is some case where this kind of convolutional layers bring some advantage, but I cannot find any, surely not on ARM platforms (both on iOS and Android). @nihui is this a known issue? Do you have any indication where or when these layers are useful?