如何使用python删除json文件中的重复条目？

aiazj4mn 于 2022-12-05 发布在 Python

关注(0)|答案(2)|浏览(235)

如何使用python删除JSON文件中的重复条目？
我有一个JSON文件，看起来像这样：
我很感激有人能帮助我解决这个问题

json_data = [
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9099",
                    "aks9098",
                    "aks9100",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9103",
                    "aks9103"
                    
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chinq.n@example.com"
    },
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9098",
                    "aks9098",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9102",
                    "aks9103"
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chinw.d@example.com"
    },
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9098",
                    "aks9100",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9102",
                    "aks9103"
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chins.b@example.com"
    }
]

我想从列表中删除重复的条目，预期结果应如下所示：
感谢您能帮助提供解决方案来修复它

json_data = [
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9098",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9103"
                    
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chinq.n@example.com"
    },
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9098",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9103"
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chinw.d@example.com"
    },
    {
        "authType": "ldap",
        "password": "",
        "permissions": [
            {
                "collections": [
                    "aks9099",
                    "aks9098",
                    "aks9100",
                    "aks9101",
                    "aks9102",
                    "aks9103"
                ],
                "project": "Central Project"
            }
        ],
        "role": "devSecOps",
        "username": "chins.b@example.com"
    }
]

python-3.x

来源：https://stackoverflow.com/questions/74635590/how-to-remove-duplicate-entries-from-json-file-using-python

2条答案

按热度按时间

8qgya5xd1#

以下方法是否能解决您的问题？

new_list=[]
for i in json_data:
    if not i in new_list:
        new_list.append(i)
print(new_list)

赞(0）回复(0）举报 2022-12-05

p8ekf7hl2#

即使OP在Python中要求这样做，在jq中也可以很容易地使用函数unique和一个更新赋值来完成：

$ jq '.[].permissions[].collections |= unique' json.txt 
[
  {
    "authType": "ldap",
    "password": "",
    "permissions": [
      {
        "collections": [
          "aks9098",
          "aks9099",
          "aks9100",
          "aks9101",
          "aks9102",
          "aks9103"
        ],
        "project": "Central Project"
      }
    ],
    "role": "devSecOps",
    "username": "chinq.n@example.com"
  },
  {
    "authType": "ldap",
    "password": "",
    "permissions": [
      {
        "collections": [
          "aks9098",
          "aks9099",
          "aks9100",
          "aks9101",
          "aks9102",
          "aks9103"
        ],
        "project": "Central Project"
      }
    ],
    "role": "devSecOps",
    "username": "chinw.d@example.com"
  },
  {
    "authType": "ldap",
    "password": "",
    "permissions": [
      {
        "collections": [
          "aks9098",
          "aks9099",
          "aks9100",
          "aks9101",
          "aks9102",
          "aks9103"
        ],
        "project": "Central Project"
      }
    ],
    "role": "devSecOps",
    "username": "chins.b@example.com"
  }
]

要在Python中调用它，可以这样做：

import subprocess
return_obj = subprocess.run(["jq", ".[].permissions[].collections |= unique","json.txt"], stdout=subprocess.PIPE)
json_data = return_obj.stdout.decode()

赞(0）回复(0）举报 2022-12-05

我来回答

如何使用python删除json文件中的重复条目？

2条答案

相关问题

热门标签

最新问答