scrapy 从选择器列表返回单个项目

tjrkku2a  于 2022-12-18  发布在  其他
关注(0)|答案(1)|浏览(130)

我试图获得所有亚马逊畅销书的数据,并处理我已经使用了scrappy,我能够获得整个选择器列表的数据,但在迭代数据列表的结果仍然只返回单个数据项。

def parse_page(self, response):

        product_data = response.xpath("//div[@id='gridItemRoot']") #THIS RETURNS A SELECTOR LIST

        for data in product_data:
            product_name = data.xpath("//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
            product_rank = data.xpath("//span[@class='zg-bdg-text']/text()").get()
            
         # It only generates a single result
            yield {
                "name": product_name,
                "rank": product_rank
            }

我尝试不迭代selectorlist,而是将selector直接传递给方法并生成result,但这也返回了单个元素。

def parse_page(self, response):
   
      
   # in previous applications all the results were scraped without iterating over any selectorlist just like following

        product_name = response.xpath("//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
        product_rank = response.xpath("//span[@class='zg-bdg-text']/text()").get()
       
 
        yield {
            "name": product_name,
            "rank": product_rank
        }
c2e8gylq

c2e8gylq1#

需要使用相对XPath表达式。

def parse_page(self, response):

        product_data = response.xpath("//div[@id='gridItemRoot']") #THIS RETURNS A SELECTOR LIST

        for data in product_data:
            product_name = data.xpath(".//div[@class='a-section a-spacing-mini _cDEzb_noop_3Xbw5']//img/@alt").get()
            product_rank = data.xpath(".//span[@class='zg-bdg-text']/text()").get()
            
         # It only generates a single result
            yield {
                "name": product_name,
                "rank": product_rank
            }

如果xpath表达式的开头没有.,它将总是获取相对于根元素的第一个匹配项,对于每次迭代,根元素总是相同的元素。

相关问题