Cython:创建没有NumPy数组的memoryview?

tuwxkamq  于 2023-06-23  发布在  其他
关注(0)|答案(4)|浏览(107)

因为我发现内存视图方便快捷,所以我尽量避免在cython中创建NumPy数组,而是使用给定数组的视图。然而,有时无法避免的是,不是更改现有数组,而是创建一个新数组。在上层函数中,这并不明显,但在通常称为子例程中,这是很明显的。考虑以下函数

#@cython.profile(False)
@cython.boundscheck(False)
@cython.wraparound(False)
@cython.nonecheck(False)
cdef double [:] vec_eq(double [:] v1, int [:] v2, int cond):
    ''' Function output corresponds to v1[v2 == cond]'''
    cdef unsigned int n = v1.shape[0]
    cdef unsigned int n_ = 0
    # Size of array to create
    cdef size_t i
    for i in range(n):
        if v2[i] == cond:
            n_ += 1
    # Create array for selection
    cdef double [:] s = np.empty(n_, dtype=np_float) # Slow line
    # Copy selection to new array
    n_ = 0
    for i in range(n):
        if v2[i] == cond:
            s[n_] = v1[i]
            n_ += 1
    return s

侧写告诉我,这里有一些速度

我能做的就是调整函数,因为有时候,比如这个向量的平均值被计算出来,有时候是求和。所以我可以重写它,用来求和或取平均值。但是,有没有一种方法可以直接创建内存视图,并动态定义大小。类似于首先创建一个c缓冲区使用malloc等,并在函数结束时将缓冲区转换为视图,传递指针和步幅等等。

    • 编辑1:**也许对于简单的情况,调整函数e。这是一个可以接受的方法。我只添加了一个参数和求和/取平均值。这样我就不必创建一个数组,并且可以在函数malloc中轻松处理。不会再快了吧?
# ...
cdef double vec_eq(double [:] v1, int [:] v2, int cond, opt=0):
    # additional option argument
    ''' Function output corresponds to v1[v2 == cond].sum() / .mean()'''
    cdef unsigned int n = v1.shape[0]
    cdef int n_ = 0
    # Size of array to create
    cdef Py_ssize_t i
    for i in prange(n, nogil=True):
        if v2[i] == cond:
            n_ += 1
    # Create array for selection
    cdef double s = 0
    cdef double * v3 = <double *> malloc(sizeof(double) * n_)
    if v3 == NULL:
        abort()
    # Copy selection to new array
    n_ = 0
    for i in range(n):
        if v2[i] == cond:
            v3[n_] = v1[i]
            n_ += 1
    # Do further computation here, according to option
    # Option 0 for the sum
    if opt == 0:
        for i in prange(n_, nogil=True):
            s += v3[i]
        free(v3)
        return s
    # Option 1 for the mean
    else:
        for i in prange(n_, nogil=True):
            s += v3[i]
        free(v3)
        return s / n_
    # Since in the end there is always only a single double value, 
    # the memory can be freed right here
sycxhyv7

sycxhyv71#

不知道如何处理cpython数组,所以我最终通过自制的“内存视图”as proposed by fabrizioM解决了这个问题。没想到这会有用在紧密循环中创建一个新的np.array是非常昂贵的,所以这给了我一个显著的加速。因为我只需要一个一维数组,我甚至不需要费心去大步。但即使对于更高维的数组,我认为这也会很好。

cdef class Vector:
    cdef double *data
    cdef public int n_ax0

    def __init__(Vector self, int n_ax0):
        self.data = <double*> malloc (sizeof(double) * n_ax0)
        self.n_ax0 = n_ax0

    def __dealloc__(Vector self):
        free(self.data)

...
#@cython.profile(False)
@cython.boundscheck(False)
cdef Vector my_vec_func(double [:, ::1] a, int [:] v, int cond, int opt):
    # function returning a Vector, which can be hopefully freed by del Vector
    cdef int vecsize
    cdef size_t i
    # defs..
    # more stuff...
    vecsize = n
    cdef Vector v = Vector(vecsize)

    for i in range(vecsize):
        # computation
        v[i] = ...

    return v

...
vec = my_vec_func(...
ptr_to_data = vec.data
length_of_vec = vec.n_ax0
3okqufwl

3okqufwl2#

Cython邮件列表中的以下主题可能会引起您的兴趣:
https://groups.google.com/forum/#!topic/cython-users/CwtU_jYADgM
看起来有一些不错的选择,如果你可以从你的函数中返回一个内存视图,这个视图在性能不是问题的不同级别上被强制。

o2g1uqev

o2g1uqev3#

http://docs.cython.org/src/userguide/memoryviews.html可以看出,cython内存视图的内存可以通过以下方式分配:

cimport cython
cdef type [:] cview = cython.view.array(size = size, 
              itemsize = sizeof(type), format = "type", allocate_buffer = True)

或由

from libc.stdlib import malloc, free
cdef type [:] cview = <type[:size]> malloc(sizeof(type)*size)

这两种情况都可以,但首先我有一个问题,如果引入自己的类型(ctypedef一些mytype),因为没有合适的格式。在第二种情况下,内存的释放存在问题。
根据手册,其工作原理如下:

cview.callback_memory_free = free

它将释放内存的函数绑定到memoryview,然而该代码不编译。

66bbxpm5

66bbxpm54#

受@embert的“内存视图”Vector类型的启发,这里有一个类似的Matrix“内存视图”类型,用于存储2D数组数据。

# Importing necessary libraries
from libc.stdlib cimport malloc, free  # Importing memory allocation and deallocation functions

# Defining a Cython class called Matrix
cdef class Matrix:
    cdef double** data  # A 2D array to store the matrix data
    cdef int n, m  # Variables to store the dimensions of the matrix

    # Constructor for the Matrix class
    def __init__(Matrix self, int n, int m):
        self.data = <double **> malloc(n * m * sizeof(double))  # Allocating memory for the matrix data
        self.n = n  # Storing the number of rows
        self.m = m  # Storing the number of columns

    # Destructor for the Matrix class
    def __dealloc__(Matrix self):
        free(self.data)  # Deallocating the memory for the matrix data

    # Method to retrieve a value from the matrix
    def __getitem__(Matrix self, (int, int) ind):
        return self.data[ind[0]][ind[1]]

    # Method to set a value in the matrix
    def __setitem__(Matrix self, (int, int) ind, double value):
        self.data[ind[0]][ind[1]] = value

# Example usage

# Creating a Matrix object with dimensions 2x3
mat = Matrix(2, 3)

# Setting values in the matrix
mat[0, 1] = 4.885
mat[2, 1] = 75.74

# Retrieving a value from the matrix and printing it
print(mat[0, 1])

相关问题