获取亚马逊?Python 3的所有点评

wwwo4jvm  于 2023-03-16  发布在  Python
关注(0)|答案(1)|浏览(115)

我正试图阅读Python中一个产品的所有评论。我有一个脚本,但它不起作用。

parser = html.fromstring(page_response)
XPATH_AGGREGATE = '//span[@id="acrCustomerReviewText"]'
XPATH_REVIEW_SECTION_1 = '//div[@data-hook="reviews-content"]'
XPATH_REVIEW_SECTION_2 = '//div[@data-hook="review"]'

XPATH_AGGREGATE_RATING = '//table[@id="histogramTable"]//tr'
XPATH_PRODUCT_NAME = '//h1//span[@id="productTitle"]//text()'
XPATH_PRODUCT_PRICE  = '//span[@id="priceblock_ourprice"]/text()'

raw_product_price = parser.xpath(XPATH_PRODUCT_PRICE)
product_price = ''.join(raw_product_price).replace(',','')

raw_product_name = parser.xpath(XPATH_PRODUCT_NAME)
product_name = ''.join(raw_product_name).strip()
total_ratings  = parser.xpath(XPATH_AGGREGATE_RATING)
reviews = parser.xpath(XPATH_REVIEW_SECTION_1)
if not reviews:
    reviews = parser.xpath(XPATH_REVIEW_SECTION_2)

页面是https://www.amazon.com/productreviews/ '+asin+"/,其中asin是一个ID(例如,B 0718 Y23 CQ)。我在评论中什么也没有得到。感谢任何帮助!

ahy6op9u

ahy6op9u1#

好吧,如果我必须诚实,我不知道你使用的一些路径在哪里,因为我找不到它们。我重新编写了你的代码,试图帮助你:

from lxml import html 
import requests
import json
asin = 'B0718Y23CQ'
page_response = requests.get('https://www.amazon.com/product-reviews/'+ asin)
parser = html.fromstring(page_response.content)
reviews_html = parser.xpath('//div[@class="a-section review"]')
reviews_arr = []
for review in reviews_html:
    review_dic = {}
    review_dic['title'] = review.xpath('.//a[@data-hook="review-title"]/text()')
    review_dic['rating'] = review.xpath('.//a[@class="a-link-normal"]/@title')
    review_dic['author'] = review.xpath('.//a[@data-hook="review-author"]/text()')
    review_dic['date'] = review.xpath('.//span[@data-hook="review-date"]/text()')
    review_dic['purchase'] = review.xpath('.//span[@data-hook="avp-badge"]/text()')
    review_dic['review_text'] = review.xpath('.//span[@data-hook="review-body"]/text()')
    review_dic['helpful_votes'] = review.xpath('.//span[@data-hook="helpful-vote-statement"]/text()')
    reviews_arr.append(review_dic)
print(json.dumps(reviews_arr, indent = 4))

输出方案为:

{
        "title": [
            "I find it very useful, I use for anything I need"
        ],
        "rating": [
            "5.0 out of 5 stars"
        ],
        "author": [
            "Nicoletta Delon"
        ],
        "date": [
            "on January 2, 2018"
        ],
        "purchase": [
            "Verified Purchase"
        ],
        "review_text": [
            "I like this a lot. I use it a lot. It's a medium to small size but it holds a lot."
        ],
        "helpful_votes": [
            "\n      One person found this helpful.\n    "
        ]
    }

现在你必须清理结果,从列表中删除它们,防止元素为空,我想你会得到你需要的。要得到所有的评论,你必须迭代页面,添加?pageNumber=1到链接,并迭代数字。你可以使用代理来防止IP阻塞,以防你要做很多请求。

相关问题