scrapy 生成csv文件时输出错误

5lhxktic  于 2022-11-09  发布在  其他
关注(0)|答案(1)|浏览(169)
import scrapy
from scrapy.http import Request

class PushpaSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://adv-tver.ru/lawyer/Pokrovskij184']

    def parse(self, response):
        rows = response.xpath("//table[@class='table']")[1:]
        for row in rows:
            a=row.xpath("//tr[1]//td[2]//text()").get()
            b=row.xpath("//tr[2]//td[2]//text()").get()
            c=row.xpath("//tr[3]//td[2]//text()").get()
            d=row.xpath("//tr[4]//td[2]//text()").get()
            e=row.xpath("//tr[5]//td[2]//text()").get()
            f=row.xpath("//tr[6]//td[2]//text()").get()
            g=row.xpath("//tr[7]//td[2]//text()").get()
            yield{
                'Ф.И.О.:':a,
                }

当我在终端运行代码时,他们会给予我correct output

'Ф.И.О.:': 'Покровский\xa0Василий\xa0Васильевич'

当我做CSV FILE时,他们给予我错误的输出,如下所示:

Ф.И.О.:ПокровÑкий ВаÑилий ВаÑильевич
ql3eal8s

ql3eal8s1#

  • 您在for循环中使用了绝对XPath表达式,而它们应该是相对表达式。
  • 您还需要将FEED_EXPORT_ENCODING设置为“utf-8”。

例如:

import scrapy
from scrapy.http import Request

class PushpaSpider(scrapy.Spider):
    name = 'test'
    start_urls = ['https://adv-tver.ru/lawyer/Pokrovskij184']
    custom_settings = {"FEED_EXPORT_ENCODING": 'utf-8'}

    def parse(self, response):
        rows = response.xpath("//table[@class='table']")[1:]
        for row in rows:
            a=row.xpath("./tr[1]//td[2]//text()").get()
            b=row.xpath("./tr[2]//td[2]//text()").get()
            c=row.xpath("./tr[3]//td[2]//text()").get()
            d=row.xpath("./tr[4]//td[2]//text()").get()
            e=row.xpath("./tr[5]//td[2]//text()").get()
            f=row.xpath("./tr[6]//td[2]//text()").get()
            g=row.xpath("./tr[7]//td[2]//text()").get()
            yield ...

相关问题