我组装了一个爬虫程序来提取在网站上找到的URL，并将它们保存到jsonline文件中：

import scrapy
from scrapy.crawler import CrawlerProcess

class MySpider(scrapy.Spider):
    name = 'medscape_crawler'
    allowed_domains = ['medscape.com']
    start_urls = ['https://www.medscape.com/']
    custom_settings = {
        'ROBOTSTXT_OBEY': False,
        'DOWNLOAD_DELAY': 2,
        'FEEDS': {'medscape_links.jsonl': {'format': 'jsonlines',}},
        'FEED_EXPORT_BATCH_ITEM_COUNT': 10,
        'JOBDIR': 'crawl_state',
    }

    def parse(self, response):
        yield {'url': response.url}  # Save this page's URL

        for href in response.css('a::attr(href)').getall():
            if href.startswith('http://') or href.startswith('https://'):
                yield response.follow(href, self.parse)

process = CrawlerProcess()
process.crawl(MySpider)
process.start()

Crawler成功收集了链接，但没有使用任何输出填充jsonline文件：

2023-05-28 21:20:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://portugues.medscape.com> (referer: https://www.medscape.com/)
2023-05-28 21:20:31 [scrapy.core.scraper] DEBUG: Scraped from <200 https://portugues.medscape.com>
{'url': 'https://portugues.medscape.com'}

jsonline文件保持为空。添加'FEED_EXPORT_BATCH_ITEM_COUNT': 10,不会进行早期保存。
任何帮助将不胜感激。
谢谢你！