python 警告:选择器同时获得文本和根,根被忽略

5ktev3wc  于 2023-05-05  发布在  Python
关注(0)|答案(3)|浏览(352)

我写了一个scrappy spider,它正常工作,但突然开始收到这个警告:
/home/user/github-repos/scrapper/scrapper/env/lib/python3.8/site-packages/scrapy/selector/unified.py:83: UserWarning: Selector got both text and root, root is being ignored. super().__init__(text=text, type=st, root=root, **kwargs)
进一步检查后,产生错误的部分如下

__slots__ = ["response"]
    selectorlist_cls = SelectorList

    def __init__(self, response=None, text=None, type=None, root=None, **kwargs):
        if response is not None and text is not None:
            raise ValueError(
                f"{self.__class__.__name__}.__init__() received "
                "both response and text"
            )

        st = _st(response, type)

        if text is not None:
            response = _response_from_text(text, st)

        if response is not None:
            text = response.text
            kwargs.setdefault("base_url", response.url)

        self.response = response
        super().__init__(text=text, type=st, root=root, **kwargs)

该警告指定根被忽略,即使构造函数需要它。这是scrappy包中的一个类,因此它可能与代表它们的更新有关。
这是我的代码中唯一与选择器交互的部分:

def load_item(self, response: TextResponse, app_id, db_id, urls):
        loader = AppLoader(response=response)
        loader.add_value("app_id", app_id)
        loader.add_value("db_id", db_id)
        loader.add_value("url", response.url)
        loader.add_css("game_title", "#appHubAppName::text")
        loader.add_css("publisher", "#game_highlights .dev_row+ .dev_row a::text")
        loader.add_css("developer", "#developers_list a::text")
        loader.add_css("publish_date", ".date::text")
        loader.add_css("tags", "#glanceCtnResponsiveRight a::text")
        loader.add_css(
            "review_count", "#review_type_all+ label .user_reviews_count::text"
        )
        loader.add_css(
            "positive_review_count",
            "#review_type_positive+ label .user_reviews_count::text",
        )
        loader.add_css(
            "negative_review_count",
            "#review_type_negative+ label .user_reviews_count::text",
        )
        loader.add_value("file_urls", urls)

        return loader.load_item()
kx5bkwkv

kx5bkwkv1#

1.8.1版中的依赖包parselhttps://github.com/scrapy/parsel/blob/master/parsel/selector.py)有一些变化。(使用commit 3b3ec90)与版本1.7.0相比在class Selector__init__中,kwarg root: Optional[Any] = None更改为root: Optional[Any] = _NOT_SET
scrapy包(https://github.com/scrapy/scrapy/blob/master/scrapy/selector/unified.py)中的class Selector默认为parsel包中的超类提供root=None。这导致parsel包中class Selector__init__中出现警告消息。我会在那里开一个问题。

mu0hgdu0

mu0hgdu03#

我以前就遇到过这个问题。这是因为我的xpath中的一个元素缺少引号。

相关问题