Paddle paddle多线程报错,MemoryError: (ResourceExhausted) Another thread is waitting

pjngdqdw 于 2022-10-22 发布在其他

关注(0)|答案(2)|浏览(520)

bug描述 Describe the Bug

多线程使用jieba+paddle时，会报各类错误/段错误，MemoryError是其中一种
多进程即multiprocessing下运行是OK的，稳定不报错。

MemoryError: (ResourceExhausted) Another thread is waitting. (at /paddle/paddle/fluid/framework/new_executor/workqueue/events_waiter.cc:104)

import _thread as thread
import jieba
import jieba.posseg as pseg
import paddle
paddle.enable_static()
jieba.enable_paddle()

def call_pseg():
    while True:
        text = "你好"
        s = pseg.lcut(text, use_paddle=True)

def main():
    t_num = 6
    for i in range(t_num):
        thread.start_new_thread(call_pseg, ())
    while True:
        pass

main()

系统为Ubuntu 20.04（docker镜像，cpu下运行）
python版本为3.8.10

PIP版本信息：
paddlepaddle 2.3.1
jieba 0.42.1

其他补充信息 Additional Supplementary Information

No response*

Paddle

来源：https://github.com/PaddlePaddle/Paddle/issues/47225

2条答案

按热度按时间

s6fujrry1#

您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue 、 AI社区来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API ， FAQ ， Github Issue and AI community to get the answer.Have a nice day!

赞(0）回复(0）举报 2022-10-22

5q4ezhmt2#

你好，经过分析，初步判断是新执行器问题，目前尚未定位到具体原因，可先通过设置FLAGS_USE_STANDALONE_EXECUTOR=0使用旧执行器来避免该问题，修复该问题后，我们会尽快回复您。

赞(0）回复(0）举报 2022-10-22