I have a problem with the launch of celery tasks. When you run the following command locally celery -A codebase worker --loglevel=debug --beat
I get the following response
/local/lib/python2.7/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
from twisted.internet.interfaces import (
DEBUG 2014-09-05 14:58:02,450 base 25908 139809439647552 Configuring Raven for host: http://69a23c7e80df4276822e771e1dec86e4:26eedecf43e34073b987b81151fd1ff9@localhost:9000:/10
[2014-09-05 14:58:02,723: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-09-05 14:58:02,726: DEBUG/MainProcess] | Worker: Building graph...
[2014-09-05 14:58:02,727: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, StateDB, Autoscaler, Consumer}
[2014-09-05 14:58:02,739: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-09-05 14:58:02,740: DEBUG/MainProcess] | Consumer: Building graph...
[2014-09-05 14:58:02,757: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Gossip, Heart, Tasks, Control, event loop}
-------------- celery@krassi-Satellite-L505 v3.1.13 (Cipater)
---- **** -----
--- * *** * -- Linux-3.13.0-35-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: codebase:0x7f27e88a4f90
- ** ---------- .> transport: django://localhost//
- ** ---------- .> results: djcelery.backends.database:DatabaseBackend
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
. geoloc.tasks.get_real_geolocation
. sofiatraffic_crawler.tasks.crawl_sofitraffic
. sofiatraffic_data.tasks.delete_stop
. sofiatraffic_data.tasks.delete_transport
[2014-09-05 14:58:02,767: DEBUG/MainProcess] | Worker: Starting Beat
[2014-09-05 14:58:02,771: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,772: DEBUG/MainProcess] | Worker: Starting Pool
[2014-09-05 14:58:02,777: INFO/Beat] beat: Starting...
[2014-09-05 14:58:02,780: DEBUG/Beat] DatabaseScheduler: intial read
[2014-09-05 14:58:02,781: INFO/Beat] Writing entries...
[2014-09-05 14:58:02,805: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,806: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-09-05 14:58:02,806: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-09-05 14:58:02,818: INFO/MainProcess] Connected to django://localhost//
[2014-09-05 14:58:02,819: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,819: DEBUG/MainProcess] | Consumer: Starting Events
[2014-09-05 14:58:02,829: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,829: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 14:58:02,832: DEBUG/MainProcess] | Consumer: Starting Heart
[2014-09-05 14:58:02,832: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,833: DEBUG/MainProcess] | Consumer: Starting Tasks
[2014-09-05 14:58:02,865: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,866: DEBUG/MainProcess] | Consumer: Starting Control
[2014-09-05 14:58:02,871: DEBUG/Beat] Current schedule:
<ModelEntry: scrap-sofitraffic-every-day sofiatraffic_crawler.tasks.crawl_sofitraffic(*[], **{}) {4}>
<ModelEntry: delete-stop-every-day sofiatraffic_data.tasks.delete_stop(*[], **{}) {4}>
<ModelEntry: celery.backend_cleanup celery.backend_cleanup(*[], **{}) {4}>
<ModelEntry: delete-transport-every-day sofiatraffic_data.tasks.delete_transport(*[], **{}) {4}>
<ModelEntry: get-real-location-every-day geoloc.tasks.get_real_geolocation(*[], **{}) {4}>
[2014-09-05 14:58:02,873: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,873: DEBUG/MainProcess] | Consumer: Starting event loop
[2014-09-05 14:58:02,874: WARNING/MainProcess] celery@krassi-Satellite-L505 ready.
[2014-09-05 14:58:03,540: INFO/Beat] DatabaseScheduler: Schedule changed.
[2014-09-05 14:58:03,541: INFO/Beat] Writing entries...
[2014-09-05 14:58:03,541: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 14:58:03,557: DEBUG/Beat] Current schedule:
<ModelEntry: scrap-sofitraffic-every-day sofiatraffic_crawler.tasks.crawl_sofitraffic(*[], **{}) {4}>
<ModelEntry: delete-stop-every-day sofiatraffic_data.tasks.delete_stop(*[], **{}) {4}>
<ModelEntry: celery.backend_cleanup celery.backend_cleanup(*[], **{}) {4}>
<ModelEntry: delete-transport-every-day sofiatraffic_data.tasks.delete_transport(*[], **{}) {4}>
<ModelEntry: get-real-location-every-day geoloc.tasks.get_real_geolocation(*[], **{}) {4}>
[2014-09-05 14:58:03,558: DEBUG/Beat] beat: Ticking with max interval->5.00 seconds
[2014-09-05 14:58:03,573: DEBUG/Beat] beat: Waking up in 4.99 seconds.
Everything is OK, the process continues and go. Then transfer to the server, run it with the same command and I get the following response:
/local/lib/python2.7/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
from twisted.internet.interfaces import (
DEBUG 2014-09-05 15:03:14,511 base 7100 140161706530560 Configuring Raven for host: http://69a23c7e80df4276822e771e1dec86e4:26eedecf43e34073b987b81151fd1ff9@localhost:9000:/10
[2014-09-05 15:03:14,694: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-09-05 15:03:14,696: DEBUG/MainProcess] | Worker: Building graph...
[2014-09-05 15:03:14,696: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, Autoscaler, StateDB, Consumer}
[2014-09-05 15:03:14,723: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-09-05 15:03:14,723: DEBUG/MainProcess] | Consumer: Building graph...
[2014-09-05 15:03:14,731: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Gossip, Agent, Heart, event loop}
-------------- [email protected] v3.1.13 (Cipater)
---- **** -----
--- * *** * -- Linux-3.2.0-34-generic-x86_64-with-Ubuntu-12.04-precise
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: codebase:0x138e5d0
- ** ---------- .> transport: django://localhost//
- ** ---------- .> results: djcelery.backends.database:DatabaseBackend
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
[2014-09-05 15:03:14,735: DEBUG/MainProcess] | Worker: Starting Beat
[2014-09-05 15:03:14,737: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,737: DEBUG/MainProcess] | Worker: Starting Pool
[2014-09-05 15:03:14,739: INFO/Beat] beat: Starting...
[2014-09-05 15:03:14,751: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,751: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-09-05 15:03:14,751: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-09-05 15:03:14,756: INFO/MainProcess] Connected to django://localhost//
[2014-09-05 15:03:14,756: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,756: DEBUG/MainProcess] | Consumer: Starting Events
[2014-09-05 15:03:14,758: DEBUG/Beat] DatabaseScheduler: intial read
[2014-09-05 15:03:14,758: INFO/Beat] Writing entries...
[2014-09-05 15:03:14,760: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,760: DEBUG/MainProcess] | Consumer: Starting Tasks
[2014-09-05 15:03:14,763: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 15:03:14,766: DEBUG/Beat] Current schedule:
[2014-09-05 15:03:14,772: DEBUG/MainProcess] | Worker: Closing Beat...
[2014-09-05 15:03:14,772: DEBUG/MainProcess] | Worker: Closing Pool...
[2014-09-05 15:03:15,046: DEBUG/Beat] beat: Ticking with max interval->5.00 seconds
[2014-09-05 15:03:15,052: DEBUG/Beat] beat: Waking up in 4.99 seconds.
[2014-09-05 15:03:15,549: DEBUG/MainProcess] | Worker: Closing Consumer...
[2014-09-05 15:03:15,549: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2014-09-05 15:03:15,550: DEBUG/MainProcess] | Worker: Stopping Pool...
[2014-09-05 15:03:15,778: DEBUG/MainProcess] | Worker: Stopping Beat...
[2014-09-05 15:03:15,779: INFO/MainProcess] beat: Shutting down...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2014-09-05 15:03:15,779: INFO/Beat] Writing entries...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2014-09-05 15:03:15,780: INFO/Beat] Writing entries...
[2014-09-05 15:03:15,780: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2014-09-05 15:03:15,780: DEBUG/MainProcess] | Consumer: Shutdown Connection...
And once you start worker immediately stops without errors without anything. Any idea what could be the problem?
The problem turned out to be in the Kombu 3.0.22. There is a problem in this version.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With