我写了一个scrappy spider,它正常工作,但突然开始收到这个警告:/home/user/github-repos/scrapper/scrapper/env/lib/python3.8/site-packages/scrapy/selector/unified.py:83: UserWarning: Selector got both text and root, root is being ignored. super().__init__(text=text, type=st, root=root, **kwargs)
进一步检查后,产生错误的部分如下
__slots__ = ["response"]
selectorlist_cls = SelectorList
def __init__(self, response=None, text=None, type=None, root=None, **kwargs):
if response is not None and text is not None:
raise ValueError(
f"{self.__class__.__name__}.__init__() received "
"both response and text"
)
st = _st(response, type)
if text is not None:
response = _response_from_text(text, st)
if response is not None:
text = response.text
kwargs.setdefault("base_url", response.url)
self.response = response
super().__init__(text=text, type=st, root=root, **kwargs)
该警告指定根被忽略,即使构造函数需要它。这是scrappy包中的一个类,因此它可能与代表它们的更新有关。
这是我的代码中唯一与选择器交互的部分:
def load_item(self, response: TextResponse, app_id, db_id, urls):
loader = AppLoader(response=response)
loader.add_value("app_id", app_id)
loader.add_value("db_id", db_id)
loader.add_value("url", response.url)
loader.add_css("game_title", "#appHubAppName::text")
loader.add_css("publisher", "#game_highlights .dev_row+ .dev_row a::text")
loader.add_css("developer", "#developers_list a::text")
loader.add_css("publish_date", ".date::text")
loader.add_css("tags", "#glanceCtnResponsiveRight a::text")
loader.add_css(
"review_count", "#review_type_all+ label .user_reviews_count::text"
)
loader.add_css(
"positive_review_count",
"#review_type_positive+ label .user_reviews_count::text",
)
loader.add_css(
"negative_review_count",
"#review_type_negative+ label .user_reviews_count::text",
)
loader.add_value("file_urls", urls)
return loader.load_item()
3条答案
按热度按时间kx5bkwkv1#
1.8.1版中的依赖包
parsel
(https://github.com/scrapy/parsel/blob/master/parsel/selector.py)有一些变化。(使用commit 3b3ec90)与版本1.7.0相比在class Selector
的__init__
中,kwargroot: Optional[Any] = None
更改为root: Optional[Any] = _NOT_SET
scrapy
包(https://github.com/scrapy/scrapy/blob/master/scrapy/selector/unified.py)中的class Selector
默认为parsel
包中的超类提供root=None
。这导致parsel
包中class Selector
的__init__
中出现警告消息。我会在那里开一个问题。oxcyiej72#
https://github.com/scrapy/scrapy/issues/5913关于此问题的问题
此警告无害
mu0hgdu03#
我以前就遇到过这个问题。这是因为我的xpath中的一个元素缺少引号。