如果div类标题文本包含“英语”，则Python 3 BeautifulSoup获取URL(href或baseURL)

ffx8fchx 于 2022-12-10 发布在 Python

关注(0)|答案(1)|浏览(117)

<div class="gallery" data-tags="19 16 40193 41706 40476 7921 815 425 900 362 229 154 146 13 65 129 766 25 9 51931 188">
    <a href="/g/987654/" class="cover" style="padding:0 0 142.79999999999998% 0">
    <img is="lazyload-image" class="" width="250" height="357" data-src="https://abc.cloud.xyz/galleries/123456/thumb.jpg" alt="" src="https://abc.cloud.xyz/galleries/123456/thumb.jpg">
    <div class="caption">[User] Text ABCDEFGH [English] </div>
    </a>
</div>

程序没有将URL/hrefs保存到txt文件中。我认为它找不到href
如果带有类标题的div元素包含Word英语，则应将属于元素类封面的href（/g/987654/）保存在txt文件中。

from bs4 import BeautifulSoup
import requests

url = "https://google.com"

response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

base_urls = []
for div in soup.find_all("div", {"class": "caption"}):
    if "English" in div.text:
        a_tag = div.find_previous_sibling("a")
        if a_tag:
            base_urls.append(a_tag["baseURL"])

with open("base_urls.txt", "w") as f:
    for base_url in base_urls:
        f.write(base_url + "\n")

到目前为止我所尝试的这段代码可以工作，但它将所有的href保存到txt文件中...

from bs4 import BeautifulSoup
import requests

url = "https://google.com"

page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")

links = soup.find_all("a")

hrefs = [link["href"] for link in links]

with open("links_test1.txt", "w") as file:
    for href in hrefs:
        file.write(href + "\n")

python

来源：https://stackoverflow.com/questions/74740364/python-3-beautifulsoup-get-url-href-or-baseurl-if-div-class-caption-text-conta

1条答案

按热度按时间

0md85ypi1#

查看HTML代码片段，您应该使用.find_previous而不是.find_previous_sibling。另外，使用a_tag['href']而不是a_tag['baseURL']：

from bs4 import BeautifulSoup

html_doc = """\
<div class="gallery" data-tags="19 16 40193 41706 40476 7921 815 425 900 362 229 154 146 13 65 129 766 25 9 51931 188">
    <a href="/g/987654/" class="cover" style="padding:0 0 142.79999999999998% 0">
    <img is="lazyload-image" class="" width="250" height="357" data-src="https://abc.cloud.xyz/galleries/123456/thumb.jpg" alt="" src="https://abc.cloud.xyz/galleries/123456/thumb.jpg">
    <div class="caption">[User] Text ABCDEFGH [English] </div>
    </a>
</div>"""

soup = BeautifulSoup(html_doc, "html.parser")

base_urls = []
for div in soup.find_all("div", {"class": "caption"}):
    if "English" in div.text:
        a_tag = div.find_previous("a")
        if a_tag:
            base_urls.append(a_tag["href"])

print(base_urls)

印刷品：

['/g/987654/']

赞(0）回复(0）举报 2022-12-10

我来回答

如果div类标题文本包含“英语”，则Python 3 BeautifulSoup获取URL(href或baseURL)

1条答案

相关问题

热门标签

最新问答