我的scrapy项目运行得很好,与“scrapy crawspider_1”命令。如何触发它(或调用scrapy命令)从气流匕首?
with DAG(<args>) as dag:
scrapy_task = PythonOperator(
task_id='scrapy',
python_callable= ?)
task_2 = ()
task_3 = ()
....
scrapy_task >> [task_2, task_3, ...]
1条答案
按热度按时间klsxnrf11#
使用BashOperator运行
source activate venv && scrapy crawl spider_1
使用Python运算符运行