使用多处理时python追加不起作用

x8diyxa7  于 2022-12-17  发布在  Python
关注(0)|答案(2)|浏览(120)

所以问题是把“e”附加到我的列表“ok = []”中没有效果,但是考虑到当在ok.append(e)上面一行执行print(e)时,“e”的值被打印出来,这就很奇怪了。
不需要理解程序和它做什么,这里主要的问题是给我的列表附加一些值没有效果,即使这个值是真实的。
我尝试在if __name__=='__main__':中使用ok = [],但得到错误NameError: name 'ok' is not defined,因此我尝试在“some_function”中使用“global ok”,但得到相同的结果

import time
import multiprocessing as mp

ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]

x = 283
y = 436

ok = []

def some_function(x_, y_):

    list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
    for e in list_:
        if not e[0] in [h[0] for h in ok]:
            if not e[1] in [u[1] for u in ok]:
                print(e)
                ok.append(e)

if __name__=='__main__':

    processes = []

    if x / y in ratios1 or x / y in ratios2:
        some_function(x_=x, y_=y)
    else:

        for X_, Y_ in [

            [x, y],
            [x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
            [x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
            [x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]

        ]:

            p = mp.Process(target=some_function, args=(X_,Y_))
            processes.append(p)

    start = time.time()

    for p_ in processes:
        p_.start()

    for p_ in processes:
        p_.join()

    end = time.time()

    print(f"finished in {end - start} sec")
    print(ok)

运行时,输出如下:

[...]                                   # other values of "e"
[283, 433]                              # some random "e" value
[566, 866]                              # some random "e" value
[849, 1299]                             # some random "e" value
[1132, 1732]                            # some random "e" value
finished in 0.8476874828338623 sec      # execution time
[]                                      # the "ok" list being printed out at the end

在“some_function”和end中添加print(id(ok))之后,它会给出以下输出:
OBS:我删除了此输出的打印内容(e)

2489040444480
3014871358528
2324227431488
2471301880896
1803966487616
2531583073344
1665411652672
2149818113088
2330038901824
1283883998272
2498472320064
2147028311104
2509405887552
finished in 0.8341867923736572 sec
2589544128640
[]
chhqkbe1

chhqkbe11#

你需要一个可以从多个进程访问的列表,这是用multiprocessing.Manager.list创建的,你必须把它作为一个参数传递,你不能把它作为一个全局变量,因为继承全局变量是特定于操作系统的。
使用托管列表比普通列表慢,所以如果你发现性能不能接受,你真的应该试着只使用局部变量,忘记使用全局变量,因为IPC是一个昂贵的过程。

import time
import multiprocessing as mp

ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]

x = 283
y = 436

def some_function(x_, y_, ok_list):

    list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
    for e in list_:
        if not e[0] in [h[0] for h in ok_list]:
            if not e[1] in [u[1] for u in ok_list]:
                print(e)
                ok_list.append(e)

if __name__=='__main__':
    manager = mp.Manager()
    ok_list = manager.list()
    processes = []

    if x / y in ratios1 or x / y in ratios2:
        some_function(x_=x, y_=y)
    else:

        for X_, Y_ in [

            [x, y],
            [x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
            [x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
            [x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]

        ]:

            p = mp.Process(target=some_function, args=(X_,Y_,ok_list))
            processes.append(p)

    start = time.time()

    for p_ in processes:
        p_.start()

    for p_ in processes:
        p_.join()

    end = time.time()

    print(f"finished in {end - start} sec")
    print(ok_list)
j2qf4p5b

j2qf4p5b2#

这应该可以实现,问题是当你启动进程的时候,它使用的对象并没有像它们被克隆的那样被传递给它。使用muliprocessing.Pool.starmap允许我们从进程返回值,从而避免了这个问题。我们使用starmap,而不仅仅是map,这样我们就可以向some_function传递多个参数。另外,Pool允许您替换for X_,Y_ in ...循环并以多处理方式运行它。

import time
import multiprocessing as mp
from multiprocessing import Pool

ratios1 = [1/x for x in range(1,11)]
ratios2 = [y/1 for y in range(1,11)]

x = 283
y = 436

ok = []

def some_function(x_, y_):

    list_ = [[a, b] for a in range(1, 1980 + 1) for b in range(1, 1980 + 1) if a / b == x_ / y_]
    for e in list_:
        if not e[0] in [h[0] for h in ok]:
            if not e[1] in [u[1] for u in ok]:
                print(e)
                ok.append(e)
    return ok

if __name__=='__main__':

    processes = []
    res=[]

    if x / y in ratios1 or x / y in ratios2:
        some_function(x_=x, y_=y)
    else:
            start = time.time()
            with Pool(13) as p:
                res = p.starmap(some_function, [[x, y],
                    [x - 1, y], [x, y - 1], [x + 1, y], [x, y + 1],
                    [x - 2, y], [x, y - 2], [x + 2, y], [x, y + 2],
                    [x - 3, y], [x, y - 3], [x + 3, y], [x, y + 3]])
            ok = res
            end = time.time()

    print(f"finished in {end - start} sec")
    print(ok)

相关问题