如何将JSON文件读取为pandas DataFrame?

zzoitvuj  于 2023-11-15  发布在  其他
关注(0)|答案(6)|浏览(140)

我正在使用python 3.6,并尝试使用下面的代码下载json文件(350 MB)作为panda Dataframe 。但是,我得到以下错误:

data_json_str = "[" + ",".join(data) + "]
"TypeError: sequence item 0: expected str instance, bytes found

字符串
如何修复错误?

import pandas as pd

# read the entire file into a python array
with open('C:/Users/Alberto/nutrients.json', 'rb') as f:
   data = f.readlines()

# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)

# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ",".join(data) + "]"

# now, load it into pandas
data_df = pd.read_json(data_json_str)

nfg76nw0

nfg76nw01#

从你的代码中,看起来你正在加载一个JSON文件,其中每一行都有JSON数据。read_json支持这样的数据lines参数:

data_df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)

字符串

备注

删除lines=True,如果你有一个单一的JSON对象,而不是每一行上的单个JSON对象。

rdrgkggo

rdrgkggo2#

使用json模块,你可以将json解析成一个python对象,然后从中创建一个框架:

import json
import pandas as pd
with open('C:/Users/Alberto/nutrients.json', 'r') as f:
    data = json.load(f)
df = pd.DataFrame(data)

字符串

q8l4jmvw

q8l4jmvw3#

如果你以二进制('rb')打开文件,你将得到字节。如何:

with open('C:/Users/Alberto/nutrients.json', 'rU') as f:

字符串
另外,正如在this回答中所指出的,你也可以直接使用pandas,比如:

df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)

14ifxucb

14ifxucb4#

如果你想把它转换成JSON对象的 * 数组 *,我想这个可以满足你的要求

import json
data = []
with open('nutrients.json', errors='ignore') as f:
    for line in f:
        data.append(json.loads(line))
print(data[0])

字符串

ghg1uchk

ghg1uchk5#

使用pandas读取json文件最简单的方法是:

pd.read_json("sample.json",lines=True,orient='columns')

字符串

处理这样的嵌套json

[[{Value1:1},{value2:2}],[{value3:3},{value4:4}],.....]

使用Python基础知识

value1 = df['column_name'][0][0].get(Value1)

tvz2xvvm

tvz2xvvm6#

请输入下面的代码

#call the pandas library
import pandas as pd
#set the file location as URL or filepath of the json file
url = 'https://www.something.com/data.json'
#load the json data from the file to a pandas dataframe
df = pd.read_json(url, orient='columns')
#display the top 10 rows from the dataframe (this is to test only)
df.head(10)

字符串
请检查代码并根据您的需要进行修改。我添加了注解来解释每行代码。希望这对您有帮助!

相关问题