How to avoid max channels per tcp connection with amqp php, persistent connections and php-fpm

Question

I`m just learning rabbitMQ and I ran into a problem.

Using http://pecl.php.net/package/amqp version 1.4 (latest now) and RabbitMQ 3.3.1. We must use php5-fpm and persistent connections with amqp->pconnect().

After a while (I guess 65500 requests) a problem occurs halting all writes "

Could not create channel. Connection has no open channel slots remaining

"

From what I have read in the sources is because every tcp connection have an autoincrement channel ID that reaches its max. This happens because every request must use the channel, and is no way to use the same channel ( I could not find a way into the php-amqp channel classs to make it persistent) and scripts cannot communicate ( to use the same instance of the channel as php object).

To lower the php-fpm lifetime is not an option, and either decoupling the application-rabbitmq communication trough another technology/library etc.

Is there any easy way to fix this ?

Theoretically it should be one channel per thread (php5-fpm worker in this case) but how can be achieved using this library ?

The code I`m using now (similar)

$this->con = new AMQPConnection(array(
    'host'          => $this->con_params['host'],
    'port'          => $this->con_params['port'],
    'vhost'         => $this->con_params['vhost'],
    'login'         => $this->con_params['user'],
    'password'      => $this->con_params['pass'],
    'read_timeout'  => 1,//seconds
    'write_timeout' => 1,//seconds
'connect_timeout' => 1,//seconds
));
$this->con->pconnect();
$channel = new AMQPChannel($this->con);
$queue = new AMQPQueue($channel);
$queue->setName($queueName);
$queue->setFlags(AMQP_DURABLE);
//$queue->declareQueue();//make sure it exists
$exchange = new AMQPExchange($channel);
$exchange->setName($exchangeName);
$exchange->setFlags(AMQP_DURABLE);
$exchange->setType(AMQP_EX_TYPE_DIRECT);
//$exchange->declareExchange();
$this->queues[$queueName]->bind($exchangeName);

Thanks !!

Answer 1

Short answer: do not use persistent connections with php-amqp extension and use regular connect() performance degradation for opening connection should not be significant even on highload (like 2k+req/sec).

Long answer:

There are hardcoded max simultaneously opened channels at the same time per connection limit in php-amqp extension (#define DEFAULT_CHANNELS_PER_CONNECTION 255). Closed channels are tried to be reused after it get closed.

But there are also another limit in rabbitmq-c (aka librabbitmq) to max channel number inside one physical connection - #define AMQP_DEFAULT_MAX_CHANNELS 0, it means no custom limit applied and thus protocol limit applied. According to specification (section 4.9 Limitations) the protocol limit is:

Number of channels per connection: 16-bit channel number.

which is according to wikipedia Unsigned: From 0 to 65,535. In AMQP 0 is never used as channel number and interpreted as error.

So when you close all your channels locally but do not close, RabbitMQ will continue channels number sequence and thus it will reach specified above limit.

There are no other way to get rid of such behavior unless close your connection.

Also, I would just recommend you to do not use persistent connections at all as it has potential memory leak and in general is a bit unstable. There was plans to completely remove persistence from php-amqp.