rabbitmqtwistedpikapython-pika

Why is the looping call required when consuming from queue?

I am trying to consume from rabbit queues using pika and twisted:

  1. Constantly (new message -> consume)
  2. Once (new message -> consume once, don't consume again until I say so)

The only input I have is this example. It covers use case 1. What about use case 2?

Isn't basic_consume implemented in a way that it will "notify" when a new message is ready? Why is it my "job" to start the looping call? It seems that polling is against the event loop pattern in twisted, no?

When making a http request with twisted, it sends off the request and continues the execution (through deferreds) once it comes back. Why does that not work in RabbitMQ/pika?

Here is how I would expect it to work:

  1. subscribe to queue
  2. consume from queue (doesn't fire immediately unless there is a message in the queue, will fire every time when there is a new message in the queue, I am ignoring qos for now.)

How do I consume a single message, e.g. basic_get? Do I need to start the looping call, and once I received the message, stop it?

Solution

  • You're asking several questions so I'll try to answer them all. I'm one of Pika's maintainers but am not as familiar with Twisted as the other connection adapters.

    Isn't basic_consume implemented in a way that it will "notify" when a new message is ready? Why is it my "job" to start the looping call? It seems that polling is against the event loop pattern in twisted, no?

    Twisted's documentation for LoopingCall states this:

    Call a function repeatedly. If f returns a deferred, rescheduling will not take place until the deferred has fired.

    The example code uses @defer.inlineCallbacks and the yield statement to return deferred functions. So, LoopingCall will not call the function repeatedly, it will only schedule a new call after the deferreds fire.

    Here is how I would expect it to work... consume from queue (doesn't fire immediately unless there is a message in the queue, will fire every time when there is a new message in the queue, I am ignoring qos for now.)

    This is exactly how the example code works.

    How do I consume a single message, e.g. basic_get? Do I need to start the looping call, and once I received the message, stop it?

    If you set prefetch (QoS) to 1, then RabbitMQ will only send one message to the consumer and will not send the next one until the first is acknowledged by basic_ack.