How to set up Airflow Send Email?

Question

I followed online tutorial to set up Email SMTP server in airflow.cfg as below:

[email]
email_backend = airflow.utils.email.send_email_smtp


[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
# Uncomment and set the user/pass settings if you want to use SMTP AUTH 
# smtp_user =                       
# smtp_password =  
smtp_port = 587
smtp_mail_from = [email protected]

And my DAG is as below:

from datetime import datetime
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import PythonOperator
from airflow.operators.email_operator import EmailOperator

def print_hello():
    return 'Hello world!'

default_args = {
        'owner': 'peter',
        'start_date':datetime(2018,8,11),
}

dag = DAG('hello_world', description='Simple tutorial DAG',
          schedule_interval='* * * * *',
          default_args = default_args, catchup=False)

dummy_operator = DummyOperator(task_id='dummy_task', retries=3, dag=dag)

hello_operator = PythonOperator(task_id='hello_task', python_callable=print_hello, dag=dag)

email = EmailOperator(
        task_id='send_email',
        to='[email protected]',
        subject='Airflow Alert',
        html_content=""" <h3>Email Test</h3> """,
        dag=dag
)

email >> dummy_operator >> hello_operator

I assumed the email operator will run after the other two operators and then send me an email. But email was not sent to me. I really appreciate your help. Thank you very much.

Best

Answer 1

Setting up SMTP Server for Airflow Email alerts using Gmail:

Create an email id from which you want to send alerts about DAG failure or if you want to use EmailOperator. Edit airflow.cfg file to edit the smtp details for the mail server.

For demo you can use any gmail account.

Create a google App Password for your gmail account. [Instruction here] This is done so that you don't use your original password or 2 Factor authentication.

1. Visit your App passwords page. You may be asked to sign in to your Google Account.
2. At the bottom, click Select app and choose the app you’re using.
3. Click Select device and choose the device you’re using.
4. Select Generate.
5. Follow the instructions to enter the App password (the 16 character code in the yellow bar) on your device.
6. Select Done.

Once you are finished, you won’t see that App password code again. However, you will see a list of apps and devices you’ve created App passwords for.

Edit airflow.cfg and edit the [smtp] section as shown below:

[smtp]
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
smtp_user = YOUR_EMAIL_ADDRESS
smtp_password = 16_DIGIT_APP_PASSWORD
smtp_port = 587
smtp_mail_from = YOUR_EMAIL_ADDRESS

Edit the below parameters to the corresponding values:

YOUR_EMAIL_ADDRESS = Your Gmail address
16_DIGIT_APP_PASSWORD = The App password generated above

Answer 2

I had same issue and i solved it by making sure i am doing a volume mount in my compose file

volumes:
  - ./dags:/usr/local/airflow/dags
  - ./config/airflow.cfg:/usr/local/airflow/airflow.cfg

Answer 3

I also encountered this issue. I changed the airflow.cfg with correct setting, but it's still not working. Finally, I found out that I should restart the airflow scheduler to load the changes of airflow.cfg. After I restarted the scheduler, it worked fine. The following two setting are both ok.

[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
smtp_user = [email protected]                      
smtp_password =  your password
smtp_port = 587
smtp_mail_from = [email protected]

---------or------------------

[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = smtp.gmail.com
smtp_starttls = False
smtp_ssl = True
smtp_user = [email protected]                      
smtp_password =  your password
smtp_port = 465
smtp_mail_from = [email protected]