scrapy 属性错误:'module'对象没有'Spider'属性

crcmnpdw  于 2022-11-09  发布在  其他
关注(0)|答案(4)|浏览(273)

我刚开始学习scrapy。所以我跟着scrapy documentation。我刚写了那个网站上提到的第一个蜘蛛。

import scrapy

class DmozSpider(scrapy.Spider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        with open(filename, 'wb') as f:
            f.write(response.body)

在项目的根目录上运行此scrapy crawl dmoz命令时,将显示以下错误。

2015-06-07 21:53:06+0530 [scrapy] INFO: Scrapy 0.14.4 started (bot: tutorial)
2015-06-07 21:53:06+0530 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
Traceback (most recent call last):
  File "/usr/bin/scrapy", line 4, in <module>
    execute()
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help
    func(*a,**kw)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command
    cmd.run(args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run
    spider = self.crawler.spiders.create(spname,**opts.spargs)
  File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler
    self._crawler.configure()
  File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler
    return cls.from_settings(crawler.settings)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__
    for module in walk_modules(name):
  File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules
    submod = __import__(fullpath, {}, {}, [''])
  File "/home/avinash/tutorial/tutorial/spiders/dmoz_spider.py", line 3, in <module>
    class DmozSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'
mzsu5hc0

mzsu5hc01#

您使用的是 old Scrapy(0.14.4)和最新的文档。
解决方案:升级至最新版本的Scrapy或读取old docs,这适合当前安装的版本

quhf5bfb

quhf5bfb2#

使用sudo pip install scrapy。如果出现Python.h missing错误,则安装带有sudo apt-get install python-devreference)的python头文件

h79rfbju

h79rfbju3#

在我的例子中,错误消息稍有不同
属性错误:部分初始化的模块'scrapy'没有属性'Spider'(很可能是因为循环汇入)
这是因为我的脚本文件名(scrapy.py)。请确保您运行的目录中没有另一个名为scrapy.py的文件,因为它将在真实的的scrapy模块之前导入。

wr98u20j

wr98u20j4#

也许可以试试:

from scrapy import Spider

如果你想使用模块的类,仅仅导入模块是不够的

相关问题