Pandas字典到json附加到文件

wmomyfyw  于 2022-10-23  发布在  其他
关注(0)|答案(3)|浏览(183)

我正在尝试将字典中的3个条目分别写入一个json文件中,最终我将与Pandas一起使用。
每本词典都是这样的:

xyz_dictionary = {'x': 1, 'y': 2, 'z':3}

我正在执行以下操作,将其转换为字符串,然后将其添加到.json文件中:

with open('jsonfile.json', 'a') as json_file:
    json_file.write(json.dumps(xyz_dictionary, indent=4))

我的处境是不断创建新的“xyz”字典,因此我必须将每个字典转换为json格式,然后将其附加到json文件中。问题是,在我完成之后,我的json文件最终看起来像这样:

{
    "x": -0.03564453125,
    "y": -0.00830078125,
    "z": 1.0244140625
}{
    "x": -0.0361328125,
    "y": -0.0087890625,
    "z": 1.0244140625
}{
    "x": -0.0390625,
    "y": -0.0087890625,
    "z": 1.025390625
}{
    "x": -0.03662109375,
    "y": -0.0087890625,
    "z": 1.0263671875
}

其中json对象不是逗号分隔的。当我尝试用Pandas加载时,我得到一个trailing Data ValueError
正如您所看到的,它不是一个包含大量json对象的大数组,它只是一堆非逗号分隔的json对象
总之,问题是“我如何创建逗号分隔的json对象,并将它们写入.json文件,该文件是所有对象的汇编?”
谢谢你

gdrx4gfi

gdrx4gfi1#

**edit:**我建议您创建一个json对象数组

import json

{ 'xyz_data': 
[{
    "x": -0.03564453125,
    "y": -0.00830078125,
    "z": 1.0244140625
},
{
    "x": -0.03564453125,
    "y": -0.00830078125,
    "z": 1.0244140625
}, ...
]}

使用append添加到dic

outfile = "Pathforfile.csv"
list = []
xyz_dictionary = {'x': 1, 'y': 2, 'z':3}
list.append(xyz_dictionary)
.... #append all xyz values
data = {'xyz' : list}
json_data = json.dumps(data) # save the data in Json format
json.dump(data, outfile) # or save it directly to a file

读取文件的步骤

json_data=open(file_directory).read()
data = json.loads(json_data)
wecizke3

wecizke32#

我建议你阅读文件,附加新数据,然后写回来。然后,您可以将其正确加载到Pandas中。

import json, os
import pandas as pd

filepath = 'jsonfile.json'

class get_file:
    def __init__(self, path):
        self.path = path or filename
    def __enter__(self):
        #see if file exists and create if not
        data = {'xyz_data':[]}
        if not os.path.isfile(path):
            file = open(path, 'rw')
            json.dump(data, file)
        else:
            #This is just to ensure that the file is valid json
            #if not it replaces the old datafile with a blank json file
            #This is hacky and you will lose all old data!
            file = open(path, 'rw') as file:
            try:
                data = json.load(file)
            except ValueError:
                json.dump(data, file)

        #this line can be deleted, just shows the data after opening
        print(data)
        self.file = file
        return file

    def __exit__(self):
        self.file.close()

def append_data(data, path: str=None):
    """Appends data to the file at the given path, defaults to filepath"""
    path = path or filepath
    with get_data(path) as json_file:
        d = json.load(json_file)
        d = d['xyz_data']
        if isinstance(d, list):
            d.extend(data)
        elif isinstance(d, dict):
            d.append(data)
        else:
            raise TypeError("Must be list or dict")
        json.dump(d, json_file)

def get_dataframe(path):
    path = path or filepath
    with get_data(path) as json_file:
        data = json.load(json_file)
        df = pd.DataFrame(data['xyz_data'])
    return df

这是未经测试的,因为我不在我的工作站上,但希望它能让概念得到理解。如果有任何错误,请告诉我!
干杯

k97glaaz

k97glaaz3#

如果您想将新数据添加到与Pandas兼容的json中,您将面临这样一个事实:to_csv中没有mode="a"。然而,我们仍然可以轻松且高效地将 Dataframe 附加到JSON。只需在to_json方法中使用orient='records', lines=True

with open(out_path, "a") as out:
    newdf = pd.DataFrame(data)
    newdf.to_json(out, orient='records', lines=True)

然后,您可以以相同的方式读取整个数据集。

newdf = pd.read_json(out_path, orient='records', lines=True)

相关问题