使用numpy阅读csv时会有额外的一行1

plupiseo  于 2023-03-23  发布在  其他
关注(0)|答案(1)|浏览(275)

我正在使用np.genfromtxt读取一个csv文件,并尝试使用converters参数来预处理每一列。
CSV:

"","Col1","Col2","Col3"
"1","Cell.1",NA,1
"2","Cell.2",NA,NA
"3","Cell.3",1,NA
"4","Cell.4",NA,NA
"5","Cell.5",NA,NA
"6","Cell.6",1,NA

代码:

import numpy as np

filename = 'b.csv'
h = ("", "Col1", "Col2", "Col3")

def col1_converter(v):
    print(f'col1_converter {v = }')
    return v

def col2_converter(v):
    print(f'col2_converter {v = }')
    return v

def col3_converter(v):
    print(f'col3_converter {v = }')
    return v

a = np.genfromtxt(
    filename,
    delimiter=',',
    names=True,
    dtype=[None, np.dtype('U8'), np.dtype('U2'), np.dtype('U2')],
    usecols=range(1, len(h)),
    converters={1: col1_converter, 2: col2_converter, 3: col3_converter},
    deletechars='',
)
print()
print(a)

当我在转换器中输入print语句时,我看到在开始时打印了一个无关的1行,它实际上并没有出现在输出的矩阵中。为什么我会看到这一行1?

col1_converter v = b'1'
col2_converter v = b'1'
col3_converter v = b'1'
col1_converter v = b'"Cell.1"'
col1_converter v = b'"Cell.2"'
col1_converter v = b'"Cell.3"'
col1_converter v = b'"Cell.4"'
col1_converter v = b'"Cell.5"'
col1_converter v = b'"Cell.6"'
col2_converter v = b'NA'
col2_converter v = b'NA'
col2_converter v = b'1'
col2_converter v = b'NA'
col2_converter v = b'NA'
col2_converter v = b'1'
col3_converter v = b'1'
col3_converter v = b'NA'
col3_converter v = b'NA'
col3_converter v = b'NA'
col3_converter v = b'NA'
col3_converter v = b'NA'

[('"Cell.1"', 'NA', '1') ('"Cell.2"', 'NA', 'NA') ('"Cell.3"', '1', 'NA')
 ('"Cell.4"', 'NA', 'NA') ('"Cell.5"', 'NA', 'NA') ('"Cell.6"', '1', 'NA')]
l7wslrjt

l7wslrjt1#

**TL;DR:**在执行任何实际转换之前,numpy通过使用参数'1'调用每个转换器函数来“测试”它,以找到该列的合理默认值。这不会影响输出,除非可能更改给定列的默认值。

解释

我觉得很奇怪,为什么每个转换器只被调用一次,然后每一行都调用第1列转换器,然后第2列转换器,等等,这说明这些调用来自代码的不同区域,我使用python的traceback模块来确认:

def col1_converter(v):
    print(f'col1_converter {v = }')
    traceback.print_stack()
    return v

果然,所有对col1_converter的调用都有相同的堆栈跟踪,除了第一个。我查看了堆栈跟踪,发现了以下有趣的代码:

File "/Users/rpmccarter/Library/Python/3.8/lib/python/site-packages/numpy/lib/_iotools.py", line 804, in update
    tester = func(testing_value or '1')

因为numpy是开源的,所以我直接去了GitHub repo,找到了_iotools.py文件,我在这里找到了一个简短的解释,解释了为什么要调用这个转换器,以及这里的转换器调用:

testing_value : str, optional
        A string representing a standard input value of the converter.
        This string is used to help defining a reasonable default
        value.

...

    try:
        tester = func(testing_value or '1')
    except (TypeError, ValueError):
        tester = None

相关问题