如何使用pandas解析html表数据的特定部分

oknrviil 于 2023-06-04 发布在其他

关注(0)|答案(1)|浏览(102)

我一直在学习如何使用Pandas抓取网页，我遇到了一堵墙，我无法提取内部的特定数据。
下面是Pandas正在解析的html：

<tr data-country="Bulgaria"> 
<td><i aria-hidden="true" class="                                    
    circle-country-flags-22 flags-22-bulgaria     display-inline-block"></i>
<a title="Bulgaria Economic Calendar" href="https://www.myfxbook.com/forex-economic- 
 calendar/bulgaria">Bulgaria</a></td>
<td>BNB</td>
<td> <a title="Bulgaria Interest Rates" href="https://www.myfxbook.com/forex-economic- 
calendar/bulgaria/interest-rate-decision">Bulgarian National Bank</a> </td> 
<td class="green"> 2.17% </td>
<td>1.82%</td>
<td> 35bp </td>
<td data-custom-date="2023-04-28 00:00:00.0">Apr 28, 2023</td>
<td data-custom-date="2023-05-29 10:00:00.0">1 day</td>
</tr>

下面是我的响应数组：

{'Central Bank': 'Bulgarian National Bank',
  'Change': '35bp',
  'Country': 'Bulgaria',
  'Current Rate': '2.17%',
  'Last Meeting': 'Apr 28, 2023',
  'Next Meeting': '1 day',
  'Previous Rate': '1.82%',
  'Unnamed: 1': 'BNB'}

这是我专门看的“1天”这一行
当我试图将“2023-05-29 10：00：00.0”解析为响应而不是“1 day”时
以下是我迄今为止为此创建的代码：

import pandas as pd
import requests
import pprint
from datetime import datetime, timedelta

url = "https://www.myfxbook.com/forex-economic-calendar/interest-rates"

r = requests.get(url)
tables = pd.read_html(r.text) # this parses all the tables in webpages to a list
# Extract the first table from the list of parsed tables
parsed_table = tables[0]

# Convert DataFrame to list of dictionaries
list_of_dicts = parsed_table.to_dict(orient='records')

# Print the list of dictionaries

data = []

for row in list_of_dicts:
    data.append(row)

pp = pprint.PrettyPrinter(depth=4)
pp.pprint(data)

我一直在搜索互联网，但还没有能够找到一个解决方案，到目前为止，我如何做到这一点，所以任何帮助将不胜感激这一点。

pandas

来源：https://stackoverflow.com/questions/76349343/how-to-parse-a-specific-part-of-html-table-data-using-pandas

1条答案

按热度按时间

tsm1rwdh1#

简单的解决方案是使用HTML解析器（如beautifulsoup）并替换<td>标记的文本。然后使用pd.read_html获取 Dataframe ：

import pprint
import requests
import pandas as pd
from bs4 import BeautifulSoup

url = "https://www.myfxbook.com/forex-economic-calendar/interest-rates"
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

# select all tags with data-custom-date= attribute
for tag in soup.select('[data-custom-date]'):
    # replace the text of these tags with value of this attribute
    tag.string.replace_with(tag['data-custom-date'])

parsed_table = pd.read_html(str(soup))[0]
data = parsed_table.to_dict(orient="records")

pp = pprint.PrettyPrinter(depth=4)
pp.pprint(data)

图纸：

[{'Central Bank': 'Bulgarian National Bank',
  'Change': '35bp',
  'Country': 'Bulgaria',
  'Current Rate': '2.17%',
  'Last Meeting': '2023-04-28 00:00:00.0',
  'Next Meeting': '2023-05-29 10:00:00.0',
  'Previous Rate': '1.82%',
  'Unnamed: 1': 'BNB'},
 {'Central Bank': 'Central Bank of Kenya',
  'Change': '75bp',
  'Country': 'Kenya',
  'Current Rate': '9.5%',
  'Last Meeting': '2023-03-29 00:00:00.0',
  'Next Meeting': '2023-05-29 13:30:00.0',
  'Previous Rate': '8.75%',
  'Unnamed: 1': 'CBK'},
 {'Central Bank': 'National Bank of the Kyrgyz Republic',
  'Change': '0bp',
  'Country': 'Kyrgyzstan',
  'Current Rate': '13.0%',

...and so on.

赞(0）回复(0）举报 2023-06-04

我来回答

如何使用pandas解析html表数据的特定部分

1条答案

相关问题

热门标签

最新问答