python-3.x 读取json文件作为panda Dataframe ?

lokaqttq  于 2023-02-06  发布在  Python
关注(0)|答案(6)|浏览(147)

我正在使用python 3.6,并尝试使用下面的代码下载json文件(350 MB)作为panda Dataframe 。但是,我得到了以下错误:

data_json_str = "[" + ",".join(data) + "]
"TypeError: sequence item 0: expected str instance, bytes found

如何修复错误?

import pandas as pd

# read the entire file into a python array
with open('C:/Users/Alberto/nutrients.json', 'rb') as f:
   data = f.readlines()

# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)

# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ",".join(data) + "]"

# now, load it into pandas
data_df = pd.read_json(data_json_str)
xqnpmsa8

xqnpmsa81#

从您的代码看,您似乎正在加载一个JSON文件,其中每行都有JSON数据。read_json支持lines参数,用于如下数据:

data_df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)
    • 注**

如果每行只有一个JSON对象,而不是单独的JSON对象,请删除lines=True

q9rjltbz

q9rjltbz2#

使用json模块,你可以把json解析成python对象,然后创建一个 Dataframe :

import json
import pandas as pd
with open('C:/Users/Alberto/nutrients.json', 'r') as f:
    data = json.load(f)
df = pd.DataFrame(data)
pu82cl6c

pu82cl6c3#

如果你打开二进制文件('rb'),你会得到字节。

with open('C:/Users/Alberto/nutrients.json', 'rU') as f:

同样如this答案中所述,您也可以直接使用panda,如:

df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)
qmelpv7a

qmelpv7a4#

如果您想将其转换为JSON对象的 * 数组 *,我认为这个方法可以满足您的需要

import json
data = []
with open('nutrients.json', errors='ignore') as f:
    for line in f:
        data.append(json.loads(line))
print(data[0])
at0kjp5o

at0kjp5o5#

请输入以下代码

#call the pandas library
import pandas as pd
#set the file location as URL or filepath of the json file
url = 'https://www.something.com/data.json'
#load the json data from the file to a pandas dataframe
df = pd.read_json(url, orient='columns')
#display the top 10 rows from the dataframe (this is to test only)
df.head(10)

请检查代码并根据您的需要进行修改。我添加了注解来解释每一行代码。希望这对您有所帮助!

6ovsh4lw

6ovsh4lw6#

使用panda读取json文件的最简单方法是:

pd.read_json("sample.json",lines=True,orient='columns')

像这样处理嵌套json

[[{值1:1},{值2:2}],[{值3:3},{值4:4}],.....]

使用Python基础知识

value1 = df['column_name'][0][0].get(Value1)

相关问题