并行填充numpy数组?

fae0ux8s  于 2023-04-30  发布在  其他
关注(0)|答案(2)|浏览(89)

我有一段代码,看起来像这样

import numpy as np
A = np.zeros((10000, 10))

for i in range(10000):

    # Some time-consuming calculations which result in a 10 element 1D array 'a'

    A[i, :] = a

如何并行化for循环,以便并行填充数组A?我的理解是,多个进程通常不应该写入同一个变量,所以我不清楚什么是正确的方法。

ha5z0ras

ha5z0ras1#

下面的代码为数组的每一行创建一个线程,但不确定它的效率如何。

import numpy as np
import threading

def thread_function(index, array):
  # aforementioned time-consuming calculation, resulting in 'a'
  a = np.ones(10)    # placeholder for calculation

  array[index, :] = a

if __name__ == "__main__":
  A = np.zeros((10000, 10))
  threads = []

  for i in range(10000):
    threads.append(threading.Thread(target=thread_function, args=(i, A)))
    threads[i].start()

  for i in range(10000):
    threads[i].join()

  print(A)

https://repl.it/@RobertClarke64/Python-Multithreading
实际上,它不是特别快,但当计算需要很长时间时,希望比连续运行每个计算要快得多。

dsf9zpds

dsf9zpds2#

为什么不利用GPU的并行能力来实现:

import numpy as np
from numba import vectorize

@vectorize(["float32(float32, float32)"], target='cuda')
def f(a,b):

  for i in range(10000):
    a=4*b
  return a

a = np.ones((10000,10), dtype=np.float32)
b = np.random.rand(10000,10).astype('f')

f(a,b)

相关问题