from scrapy.crawler import CrawlerProcess
from your_scraper_project.spiders.spider_file import spiderClass
# if your project name is carScraper and spider file is named car_spider and class is carSpider
# from carScraper.spiders.car_spider import carSpider
#
def run_car_scraper(car_model):
process = CrawlerProcess()
process.crawl(carSpider, first=car_model) # Assuming that you are passing the argumment of the car_model to scrape specific models
process.start() # This will start your spider
1条答案
按热度按时间mec1mxoz1#
我认为您可以使用CrawlerProcess来执行此操作。CrawlerProcess documentation
你所要做的就是通过点击按钮来运行python函数。我在这里附加了一个简单的函数,希望它能对你有所帮助。
当爬行器将数据保存在数据库中或从要显示数据的任何位置保存数据后,您可以获取数据并从函数中返回。
如果您想将抓取的数据保存在变量中,这个答案可能对您有用。How to save the data from a scrapy crawler into a variable?