Warning: Selector got both text and root, root is being ignored

I wrote a scrapy spider that was working as normal, but suddenly started getting this warning:

/home/user/github-repos/scrapper/scrapper/env/lib/python3.8/site-packages/scrapy/selector/unified.py:83: UserWarning: Selector got both text and root, root is being ignored. super().__init__(text=text, type=st, root=root, **kwargs)

Upon further inspection, the section which produces the error is the following

    __slots__ = ["response"]
    selectorlist_cls = SelectorList

    def __init__(self, response=None, text=None, type=None, root=None, **kwargs):
        if response is not None and text is not None:
            raise ValueError(
                f"{self.__class__.__name__}.__init__() received "
                "both response and text"
            )

        st = _st(response, type)

        if text is not None:
            response = _response_from_text(text, st)

        if response is not None:
            text = response.text
            kwargs.setdefault("base_url", response.url)

        self.response = response
        super().__init__(text=text, type=st, root=root, **kwargs)

The warning specifies that the root is being ignored, even though the constructor requires it. This is a class within the scrapy package so it may relate to an update on their behalf.

This is the only part of my code that interacts with selectors:

    def load_item(self, response: TextResponse, app_id, db_id, urls):
        loader = AppLoader(response=response)
        loader.add_value("app_id", app_id)
        loader.add_value("db_id", db_id)
        loader.add_value("url", response.url)
        loader.add_css("game_title", "#appHubAppName::text")
        loader.add_css("publisher", "#game_highlights .dev_row+ .dev_row a::text")
        loader.add_css("developer", "#developers_list a::text")
        loader.add_css("publish_date", ".date::text")
        loader.add_css("tags", "#glanceCtnResponsiveRight a::text")
        loader.add_css(
            "review_count", "#review_type_all+ label .user_reviews_count::text"
        )
        loader.add_css(
            "positive_review_count",
            "#review_type_positive+ label .user_reviews_count::text",
        )
        loader.add_css(
            "negative_review_count",
            "#review_type_negative+ label .user_reviews_count::text",
        )
        loader.add_value("file_urls", urls)

        return loader.load_item()

Solution

There are changes in the dependency package parsel (https://github.com/scrapy/parsel/blob/master/parsel/selector.py) in version 1.8.1. (with commit 3b3ec90) compared to version 1.7.0 In the __init__ of class Selector the kwarg root: Optional[Any] = None was changed to root: Optional[Any] = _NOT_SET

The class Selector in the scrapypackage (https://github.com/scrapy/scrapy/blob/master/scrapy/selector/unified.py) provides root=None by default to the super class in the parsel package.

This causes that warning message in the __init__ of class Selector in the parsel package, were root=None is not equal to the sentinel object _NOT_SET. This warning is incorrect, but harmless.

That said, I will open an issue there.