pythonpython-asynciotwistedtwisted.internettwisted.client

How can I run a simple twisted client on top of asyncio?


I have the following client code that I borrowed from twisted's docs:

https://docs.twistedmatrix.com/en/twisted-20.3.0/web/howto/client.html#the-agent

And I am trying to run it with asyncio since I am building an asyncio project that requires compatibility with twisted. Here is the code:

import asyncio

from twisted.internet import asyncioreactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers


asyncioreactor.install()


async def request():
    agent = Agent(asyncioreactor.AsyncioSelectorReactor)

    d = agent.request(
        b'GET',
        b'http://httpbin.com/anything',
        Headers({'User-Agent': ['Twisted Web Client Example']}),
        None)

    def cbResponse(ignored):
        print('Response received')

    d.addCallback(cbResponse)

    def cbShutdown(ignored):
        asyncioreactor.AsyncioSelectorReactor.stop()

    d.addBoth(cbShutdown)
    print("This is where it always get stuck")
    res = await d.asFuture(asyncio.get_event_loop())
    print("SUCCESSS!!!!")


if __name__ == "__main__":
    asyncio.run(request())

I saved this as a request.py file and run it with python request.py, but it always hangs when reaching this line:

    print("This is where it always get stuck")
    res = await d.asFuture(asyncio.get_event_loop())

Is it possible to run this with asyncio? I am not too familiar with twisted and my ultimate goal is to be able to run a twisted client with asyncio.


Solution

  • Your example code uses twisted.internet.asyncioreactor.AsyncioSelectorReactor as if it were a reactor. Instead, it is a class that implements a reactor.

    The reactor is twisted.internet.reactor. Use it like this:

    from twisted.internet import asyncioreactor
    asyncioreactor.install()
    from twisted.internet import reactor
    

    Also, your example calls asyncio.run but it should call twisted.internet.reactor.run instead:

    reactor.run()
    

    If you want the reactor to stop in response to some event, call reactor.stop in the handler for that event.

    twisted.internet.task.react is a convenient reactor start/stop helper function.

    Here's an example from https://meejah.ca/blog/python3-twisted-and-asyncio

    from twisted.internet.task import react
    from twisted.internet.defer import ensureDeferred
    # our "real" main
    async def _main(reactor):
        await some_deferred_returning_function()
    # a wrapper that calls ensureDeferred
    def main():
        return react(
            lambda reactor: ensureDeferred(
                _main(reactor)
            )
        )
    if __name__ == '__main__':
        main()