Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery Flower Security in Production

I am looking to use Flower (https://github.com/mher/flower) to monitor my Celery tasks in place of the django-admin as reccomended in their docs (http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor). However, because I am new to this I am a little confused about the way Flower's page is only based on HTTP, and not HTTPS. How can I enable security for my Celery tasks such that any old user can't just visit the no-login-needed website http://flowerserver.com:5555 and change something?

I have considered Celery's own documentation on this, but they unfortunately there is no mention of how to secure Flower's api or web ui. All it says: [Need more text here]

Thanks!

Update: My question is in part a duplicate of here: How do I add authentication and endpoint to Django Celery Flower Monitoring?

However, I clarify his question here by asking how to run it using an environment that includes nginx, gunicorn, and celery all on the same remote machine. I too am wondering about how to set up Flower's outside accessible url, but also would prefer something like https instead of http if possible (or some way of securing the webui and accessing it remotely). I also need to know if leaving Flower running is a considerable security risk for anyone who may gain access to Flower's internal API and what the best way for securing this could be, or if it should just be disabled altogether and used just on an as-needed basis.

like image 471
mh00h Avatar asked Oct 30 '13 17:10

mh00h


People also ask

How does flower work celery?

It can be used as a bucket where programming tasks can be dumped. The program that passed the task can continue to execute and function responsively. The Celery Flower is a tool for monitoring your celery tasks and workers. It's web based and allows you to see task progress, details, worker status.

What is flower in airflow?

Flower is a web based tool for monitoring and administrating Celery clusters. This topic describes how to configure Airflow to secure your flower instance. This is an optional component that is disabled by default in Community deployments and you need to configure it on your own if you want to use it.

How do you get celery flowers?

This instructs docker compose to run our flower on port 9090 and open the port for us to access it. Run this command to start your project. When all services are up you can open http://127.0.0.1:9090/ and see celery flower interface.


2 Answers

You can run flower with --auth flag, which will authenticate using a particular google email:

celery flower [email protected] 

Edit 1:

New version of Flower requires couple more flags and a registered OAuth2 Client with Google Developer Console:

celery flower \     [email protected] \     --oauth2_key="client_id" \     --oauth2_secret="client_secret" \     --oauth2_redirect_uri="http://example.com:5555/login" 

oauth2_redirect_uri has to be the actual flower login url, and it also has to be added to authorized redirect url's in Google Development Console.

Unfortunately this feature doesn't work properly in current stable version 0.7.2, but it is now fixed in development version 0.8.0-dev with this commit.

Edit 2:

You can configure Flower using basic authentication:

celery flower --basic_auth=user1:password1,user2:password2 

Then block 5555 port for all but localhost and configure reverse proxy for nginx or for apache:

ProxyRequests off ProxyPreserveHost On ProxyPass / http://localhost:5555 

Then make sure proxy mod is on:

sudo a2enmod proxy sudo a2enmod proxy_http 

In case you can't set it up on a separate subdomain, ex: flower.example.com (config above), you can set it up for example.com/flower:

run flower with url_prefix:

celery flower --url_prefix=flower --basic_auth=user1:password1,user2:password2 

in apache config:

ProxyPass /flower http://localhost:5555 

Of course, make sure SSL is configured, otherwise there is no point :)

like image 152
lehins Avatar answered Sep 30 '22 20:09

lehins


I have figured out it using proxy on Django side https://pypi.org/project/django-revproxy/. So Flower is hidden behind Django auth which is more flexible than basic auth. And you don't need rewrite rule in NGINX.

Flower 0.9.5 and higher

URL prefix must be moved into proxy path: https://github.com/mher/flower/pull/766

urls.py

urlpatterns = [     FlowerProxyView.as_url(),     ... ] 

views.py

class FlowerProxyView(UserPassesTestMixin, ProxyView):     # `flower` is Docker container, you can use `localhost` instead     upstream = 'http://{}:{}'.format('flower', 5555)     url_prefix = 'flower'     rewrite = (         (r'^/{}$'.format(url_prefix), r'/{}/'.format(url_prefix)),      )      def test_func(self):         return self.request.user.is_superuser      @classmethod     def as_url(cls):         return re_path(r'^(?P<path>{}.*)$'.format(cls.url_prefix), cls.as_view()) 

Flower 0.9.4 and lower

urls.py

urlpatterns = [     re_path(r'^flower/?(?P<path>.*)$', FlowerProxyView.as_view()),     ... ] 

views.py

from django.contrib.auth.mixins import UserPassesTestMixin from revproxy.views import ProxyView   class FlowerProxyView(UserPassesTestMixin, ProxyView):     # `flower` is Docker container, you can use `localhost` instead     upstream = 'http://flower:5555'      def test_func(self):         return self.request.user.is_superuser 
like image 41
Petr Přikryl Avatar answered Sep 30 '22 20:09

Petr Přikryl