我正在尝试部署我制作的蜘蛛爬虫,代码在我的计算机上运行良好,当我尝试将其部署到https://app.zyte.com/,以便在云上运行时,我得到以下错误:
warnings.warn("Setuptools is replacing distutils.")
Packing version 1.0
Deploying to Scrapy Cloud project "615653"
Deploy log last 30 lines:
Deploy log location: C:\Users\hp\AppData\Local\Temp\shub_deploy_4a5i8o6z.log
Error: Deploy failed: b'{"status": "error", "message": "Internal error"}'
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
_run(args, settings)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
_run_scrapy(args, settings)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
execute(settings=settings)
File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
cmd.add_options(parser)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
parser.add_option("--debug", action="store_true",
AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
Traceback (most recent call last):
File "/usr/local/bin/shub-image-info", line 8, in <module>
sys.exit(shub_image_info())
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 209, in shub_image_info
_run_usercode(None, ['scrapy', 'shub_image_info'] + sys.argv[1:],
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
_run(args, settings)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
_run_scrapy(args, settings)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
execute(settings=settings)
File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
cmd.add_options(parser)
File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
parser.add_option("--debug", action="store_true",
AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
{"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}
{"status": "error", "message": "Internal error"}
1条答案
按热度按时间tcbh2hod1#
初看起来像是..可能是
scrapinghub.yml
文件中定义的堆栈参数与项目中使用的Scrapy版本之间存在差异的结果。预期定义的堆栈应与使用的零碎版本匹配。
更多关于碎云堆栈的信息(docs)。