Flask APScheduler + Gunicorn workers - Still running task twice after socket fix

Question

I have a Flask app where i use Flask-APScheduler to run a scheduled query on my database and send an email via a cron job.

I'm running my app via Gunicorn with the following config and controlled via supervisor:

[program:myapp]
command=/home/path/to/venv/bin/gunicorn -b localhost:8000 -w 4 myapp:app --preload
directory=/home/path/here
user=myuser
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true

The job details are stored in my config.py:

...config stuff
JOBS = [
    {
        'id': 'sched_email',
        'func': 'app.tasks:sched_email',
        'trigger': 'cron',
        'hour': 9,
    },
]
SCHEDULER_API_ENABLED = True

Originally the email was being sent 4 times due to the 4 workers initialising the app and the scheduler. I found a similar article which suggested opening a socket when the app is initialised so the other workers can't grab the job.

My init.py:

# Third-party imports
import logging
from logging.handlers import SMTPHandler, RotatingFileHandler
import os
from flask import Flask
from flask_mail import Mail, Message
from flask_sqlalchemy import SQLAlchemy
from flask_apscheduler import APScheduler
from flask_migrate import Migrate
from flask_login import LoginManager
import sys, socket

# Local imports
from config import app_config

# Create initial instances of extensions
mail = Mail()
db = SQLAlchemy()
scheduler = APScheduler()
migrate = Migrate()
login_manager = LoginManager()

# Construct the Flask app instance
def create_app(config_name):
    app = Flask(__name__)
    app.config.from_object(app_config[config_name])
    app_config[config_name].init_app(app)

    migrate.init_app(app, db)
    mail.init_app(app)
    db.init_app(app)

    # Fix to ensure only one Gunicorn worker grabs the scheduled task
    try:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.bind(("127.0.0.1", 47200))
    except socket.error:
        pass
    else:
        scheduler.init_app(app)
        scheduler.start()

    login_manager.init_app(app)
    login_manager.login_message = "You must be logged in to access this page."
    login_manager.login_message_category = 'danger'
    login_manager.login_view = "admin.login"

    # Initialize blueprints
    from .errors import errors as errors_blueprint
    app.register_blueprint(errors_blueprint)

    from .main import main as main_blueprint
    app.register_blueprint(main_blueprint)

    from .admin import admin as admin_blueprint
    app.register_blueprint(admin_blueprint)

    # Setup logging when not in debug mode
    if not app.debug:
        ... logging email config

        ... log file config

    return app

Now the email is being sent twice!

Can anyone suggest why this is happening? Is there any logs i can dive into to work out what's going on?

I also read about using the @app.before_first_request decorator but as i'm using the app factory pattern i'm not sure how to incorporate this.

Thanks!

Answer 1

So it turns out the issue was a silly mistake i made.

I didn't configure supervisor correctly so my --preload flag was not actually being applied.

After i fixed supervisor and reloaded, my task is now running correctly and i'm receiving one email.

I hope my setup above will help others as being a beginner this took me a long time to get working.