pythonweb-scrapingapify

Apify Scrapy template returns Attribute error

I install the Scrapy template from Apify CLI and run it as intructed. I don't do any changes.

When I deploy it to Apify, the same problem occurs. Even when I used the same template from Apify console and create a new one, the same error happens again.

Error:

AttributeError: 'AsyncioSelectorReactor' object has no attribute '_handleSignals'

I'm using Python 3.11 on M1 Macbook. How can I solve it?

Solution

  • This is a bug on Scrapy's (or even Twisted's) side - https://github.com/scrapy/scrapy/issues/6024.

    Adding the following constraint:

    Twisted < 23.0.0

    to your requirements.txt file should solve the issue.

    We already updated the Scrapy template this way (https://github.com/apify/actor-templates/pull/179).

    It should work now.