Which form of connection to use with pika

Question

I've been trying to figure out which form of connection i should use when using pika, I've got two alternatives as far as I understand.

Either the BlockingConnection or the SelectConnection, however I'm not really sure about the differences between these two (i.e. what is the BlockingConnection blocking? and more)

The documentation for pika says that SelectConnection is the preferred way to connect to rabbit since it provides "multiple event notification methods including select, epoll, kqueue and poll."

So I'm wondering what are the implications of these two different kinds of connections?

PS: I know I shouldn't put a tag in the title but in this case I think it does help to clarify the question.

Answer 1

The SelectConnection is useful if your application architecture can benefit from an asynchronous design, e.g. doing something else while the RabbitMQ IO completes (e.g. switch to some other IO etc) . This type of connection uses callbacks to indicate when functions return. For example you can declare callbacks for

on_connected, on_channel_open, on_exchange_declared, on_queue_declared etc.

...to perform operations when these events are triggered.

The benefit is especially good if your RabbitMQ server (or connection to that server) is slow or overloaded.

BlockingConnection on the hand is just that - it blocks until the called function returns. so it will block the execution thread until connected or channel_open or exchange_declared or queue_declared return for example. That said, its often simpler to program this sort of serialized logic than the async SelectConnection logic. For simple apps with responsive RabbitMQ servers these also work OK IMO.

I suppose you've read the Pika documentation already http://pika.readthedocs.io/en/stable/intro.html, if not, then this is absolutely vital information before you use Pika!

Cheers!

Answer 2

The Pika documentation is quite clear about the differences between the connection types. The main difference is that the pika.adapters.blocking_connection.BlockingConnection() adapter is used for non-asynchronous programming and that the pika.adapters.select_connection.SelectConnection() adapter is used for asynchronous programming.

If you don't know what the difference is between non-asynchronous/synchronous and asynchronous programming I suggest that you read this question or for the more deeper technical explanation this article.

Now let's dive into the different Pika adapters and see what they do, for the example purpose I imagine that we use Pika for setting up a client connection with RabbitMQ as AMQP message broker.

BlockingConnection()

In the following example, a connection is made to RabbitMQ listening to port 5672 on localhost using the username guest and password guest and virtual host '/'. Once connected, a channel is opened and a message is published to the test_exchange exchange using the test_routing_key routing key. The BasicProperties value passed in sets the message to delivery mode 1 (non-persisted) with a content-type of text/plain. Once the message is published, the connection is closed:

import pika

parameters = pika.URLParameters('amqp://guest:guest@localhost:5672/%2F')

connection = pika.BlockingConnection(parameters)

channel = connection.channel()

channel.basic_publish('test_exchange',
                      'test_routing_key',
                      'message body value',
                      pika.BasicProperties(content_type='text/plain',
                                           delivery_mode=1))

connection.close()

SelectConnection()

In contrast, using this connection adapter is more complicated and less pythonic, but when used with other asynchronous services it can have tremendous performance improvements. In the following code example, all of the same parameters and values are used as were used in the previous example:

import pika

# Step #3
def on_open(connection):

    connection.channel(on_open_callback=on_channel_open)

# Step #4
def on_channel_open(channel):

    channel.basic_publish('test_exchange',
                            'test_routing_key',
                            'message body value',
                            pika.BasicProperties(content_type='text/plain',
                                                 delivery_mode=1))

    connection.close()

# Step #1: Connect to RabbitMQ
parameters = pika.URLParameters('amqp://guest:guest@localhost:5672/%2F')

connection = pika.SelectConnection(parameters=parameters,
                                   on_open_callback=on_open)

try:

    # Step #2 - Block on the IOLoop
    connection.ioloop.start()

# Catch a Keyboard Interrupt to make sure that the connection is closed cleanly
except KeyboardInterrupt:

    # Gracefully close the connection
    connection.close()

    # Start the IOLoop again so Pika can communicate, it will stop on its own when the connection is closed
    connection.ioloop.start()

Conclusion

For those doing simple, non-asynchronous/synchronous programming, the BlockingConnection() adapter proves to be the easiest way to get up and running with Pika to publish messages. But if you are looking for a way to implement asynchronous message handling, the SelectConnection() handler is your better choice.

Happy coding!