第一个选项

您可以将蜘蛛中的项目保存为Scrapy教程，例如：

import scrapy
import json

DICT = {
    'https://quotes.toscrape.com/page/1/': 'domain1.json',
    'https://quotes.toscrape.com/page/2/': 'domain2.json',
}

class MydomainSpider(scrapy.Spider):
    name = "mydomain"
    start_urls = [
        'https://quotes.toscrape.com/page/1/',
        'https://quotes.toscrape.com/page/2/',
    ]

    def parse(self, response):
        filename = DICT[response.url]
        with open(filename, 'w') as fp:
            json.dump({"content": response.body.decode("utf-8")}, fp)

变量DICT只用于指定JSON文件名，但您也可以使用域作为文件名。

第二个选项

您可以尝试在pipelines.py中使用process_item，如下所示：

from scrapy.exporters import JsonItemExporter

class SaveJsonPipeline:
    def process_item(self, item, spider):
       filename = item['filename']
       del item['filename']
       JsonItemExporter(open(filename, "wb")).export_item(item)
       return item

item['filename']用于保存每个start_url的文件名，您也需要设置items.py，例如：

import scrapy

class MydomainItem(scrapy.Item):
    filename = scrapy.Field()
    content = scrapy.Field()

你蜘蛛：

import scrapy
from ..items import MydomainItem

DICT = {
    'https://quotes.toscrape.com/page/1/': 'domain1.json',
    'https://quotes.toscrape.com/page/2/': 'domain2.json',
}

class MydomainSpider(scrapy.Spider):
    name = 'mydomain'
    allowed_domains = ['mydomain.com']
    start_urls = [
        'https://quotes.toscrape.com/page/1/',
        'https://quotes.toscrape.com/page/2/',
    ]

    def parse(self, response):
        item = MydomainItem()
        item["filename"] = DICT[response.url]
        item["content"] = response.body.decode("utf-8")
        yield item

在运行之前，您需要在设置中添加管道。

ITEM_PIPELINES = {
    'myproject.pipelines.SaveJsonPipeline': 300,
}