如何将带有嵌套对象的JSON文件读取为pandas DataFrame?

j2cgzkjk  于 2023-11-15  发布在  其他
关注(0)|答案(4)|浏览(108)

我很好奇如何使用pandas读取以下结构的嵌套json:

{
    "number": "",
    "date": "01.10.2016",
    "name": "R 3932",
    "locations": [
        {
            "depTimeDiffMin": "0",
            "name": "Spital am Pyhrn Bahnhof",
            "arrTime": "",
            "depTime": "06:32",
            "platform": "2",
            "stationIdx": "0",
            "arrTimeDiffMin": "",
            "track": "R 3932"
        },
        {
            "depTimeDiffMin": "0",
            "name": "Windischgarsten Bahnhof",
            "arrTime": "06:37",
            "depTime": "06:40",
            "platform": "2",
            "stationIdx": "1",
            "arrTimeDiffMin": "1",
            "track": ""
        },
        {
            "depTimeDiffMin": "",
            "name": "Linz/Donau Hbf",
            "arrTime": "08:24",
            "depTime": "",
            "platform": "1A-B",
            "stationIdx": "22",
            "arrTimeDiffMin": "1",
            "track": ""
        }
    ]
}

字符串
这里的数组保持为json,我更希望它扩展为列。

pd.read_json("/myJson.json", orient='records')

编辑

谢谢你的第一个答案。我应该改进我的问题:数组中嵌套属性的扁平化不是强制性的。只需要[A,B,C]连接df.locations 'name']就可以了。
我的文件包含多个JSON对象(每行1个)我想保留数字,日期,名称和位置列。但是,我需要加入位置。

allLocations = ""
isFirst = True
for location in result.locations:
    if isFirst:
        isFirst = False
        allLocations = location['name']
    else:
        allLocations += "; " + location['name']
allLocations


我这里的方法似乎不是有效的/Pandas风格。

t5zmwmid

t5zmwmid1#

可以使用json_normalize

import json

with open('myJson.json') as data_file:    
    data = json.load(data_file)  

df = pd.json_normalize(data, 'locations', ['date', 'number', 'name'], 
                    record_prefix='locations_')
print (df)
  locations_arrTime locations_arrTimeDiffMin locations_depTime  \
0                                                        06:32   
1             06:37                        1             06:40   
2             08:24                        1                     

  locations_depTimeDiffMin           locations_name locations_platform  \
0                        0  Spital am Pyhrn Bahnhof                  2   
1                        0  Windischgarsten Bahnhof                  2   
2                                    Linz/Donau Hbf               1A-B   

  locations_stationIdx locations_track number    name        date  
0                    0          R 3932         R 3932  01.10.2016  
1                    1                         R 3932  01.10.2016  
2                   22                         R 3932  01.10.2016

字符串
编辑:
您可以使用read_json通过DataFrame构造函数解析name,最后使用groupby应用join

df = pd.read_json("myJson.json")
df.locations = pd.DataFrame(df.locations.values.tolist())['name']
df = df.groupby(['date','name','number'])['locations'].apply(','.join).reset_index()
print (df)
        date    name number                                          locations
0 2016-01-10  R 3932         Spital am Pyhrn Bahnhof,Windischgarsten Bahnho...

y4ekin9u

y4ekin9u2#

另一个选择,如果有人发现这一点,因为我是通过一个笔记本工作。

df = pd.read_json('filename.json')
df2 = pd.DataFrame.from_records(df['nest_level_1']['nest_level_2'])

字符串
快乐编码

sq1bmfud

sq1bmfud3#

pandas.json_normalize的一个可能的替代方案是通过只从嵌套字典中提取选定的键和值来构建自己的嵌套框架。这样做的主要原因是因为json_normalize对于非常大的json文件会变慢(并且可能不总是产生你想要的输出)。
所以,这里有一种使用glom来扁平化pandas中嵌套字典的替代方法。目的是从嵌套字典中提取选定的键和值,并将它们保存在pandas嵌套框架(:
下面是一个分步指南:https://medium.com/@enrico.alemani/flatten-nested-dictionaries-in-pandas-using-glom-7948345c88f5

import pandas as pd
from glom import glom
from ast import literal_eval

target = {
    "number": "",
    "date": "01.10.2016",
    "name": "R 3932",
    "locations":
        {
            "depTimeDiffMin": "0",
            "name": "Spital am Pyhrn Bahnhof",
            "arrTime": "",
            "depTime": "06:32",
            "platform": "2",
            "stationIdx": "0",
            "arrTimeDiffMin": "",
            "track": "R 3932"
        }
}   


# Import data
df = pd.DataFrame([str(target)], columns=['target'])

# Extract id keys and save value into a separate pandas column
df['id'] = df['target'].apply(lambda row: glom(literal_eval(row), 'locations.name'))

字符串

doinxwow

doinxwow4#

我有一个多行JSON,每行有一个JSON对象<$'a':'b','scope':'eid':123213}}<$'a':'d ',' scope ':' eid ':1343213}}
没有逗号分隔。每行都是独立的
我使用以下逻辑读取嵌套结构
threshold = pd.read_json(r”/content/data.json”,lines=True)

threshold = pd.read_json(r"/content/data.json",lines=True)
threshold['entityId'] = pd.DataFrame.from_records(threshold['scope'])['entityId']
threshold.head()

字符串

相关问题