Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

JOB_TOO_BIG Pheanstalk - what can be done?

On Laravel 4.2 & Laravel Forge

I Made a mistake and accidentally pushed some code on to the production sever, but there was a bug and it pushed a job to the queue without deleting it once done. Now I can't push anything in the queue anymore, I get:

Pheanstalk_Exception JOB_TOO_BIG: job data exceeds server-enforced limit

What can I do?

like image 608
commandantp Avatar asked Mar 22 '15 20:03

commandantp


People also ask

What is pheanstalk?

Pheanstalk is a pure PHP 7.1+ client for the beanstalkd workqueue. It has been actively developed, and used in production by many, since late 2008. Created by Paul Annesley, Pheanstalk is rigorously unit tested and written using encapsulated, maintainable object oriented design.

How do you handle models in queue jobs?

For example if your queue job involves using models, just pass the model ID into the queue and as part of the job fetch them from the database, rather than passing the queue the entire model instance. If you're using eloquent models, they're automatically handled in this way.

What is the problem with the Beanstalk protocol?

When using a socket implementation that supports read timeouts, like SocketSocket which uses the socket extension we use read and write timeouts to detect broken connections; the issue with the beanstalk protocol is that it allows for no packets to be sent for extended periods of time.


2 Answers

You can increase the max job size with the -z option for Beanstalkd: http://linux.die.net/man/1/beanstalkd

To do this on Forge you need to SSH into the server and edit the /etc/default/beanstalkd file.

Add the following line (or uncomment the existing BEANSTALKD_EXTRA line and edit it): BEANSTALKD_EXTRA="-z 524280"

Restart beanstalkd after making the change: sudo service beanstalkd restart

The size should be specified in bytes.

I am not sure if this could have serious performance effects - so far, so good for me. I would appreciate any comments on performance.

like image 160
HPage Avatar answered Sep 28 '22 06:09

HPage


This is because you're trying to store too much data in the queue itself. Try to cut down the data you're pushing to the queue.

For example if your queue job involves using models, just pass the model ID into the queue and as part of the job fetch them from the database, rather than passing the queue the entire model instance.

If you're using eloquent models, they're automatically handled in this way.

like image 24
Wader Avatar answered Sep 28 '22 06:09

Wader