Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

kombu.exceptions.EncodeError: <Flask 'src'> is not JSON serializable

Tags:

python

flask

celery

I am trying to send emails asynchronously using Celery and RabbitMQ. This is the first time I am ever using Celery so I am not very familiar with some of the errors. I realize the traceback is coming from a package called kombu which I know is a dependency for Celery. I am just not able to debug this.

The traceback happens whenever I test out trying to send an email.

Traceback:

[2017-05-14 12:35:08,093] ERROR in app: Exception on /home [POST]
Traceback (most recent call last):
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/utils/json.py", line 72, in dumps
    **dict(default_kwargs, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/__init__.py", line 237, in dumps
    **kw).encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 198, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 256, in iterencode
    return _iterencode(o, 0)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/utils/json.py", line 62, in default
    return super(JSONEncoder, self).default(o)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 179, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <Flask 'src'> is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/kai/github-projects/Ticket-System/src/views.py", line 139, in home
    email_notification(cust_f_name, cust_email, tix_num)
  File "/Users/kai/github-projects/Ticket-System/src/notifications.py", line 71, in email_notification
    c_name=cust_name, tix=tix))
  File "/Users/kai/github-projects/Ticket-System/src/notifications.py", line 54, in send_email
    send_async_email.delay(app, msg)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/celery/app/task.py", line 412, in delay
    return self.apply_async(args, kwargs)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/celery/app/task.py", line 535, in apply_async
    **options
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/celery/app/base.py", line 737, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/celery/app/amqp.py", line 558, in send_task_message
    **properties
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/messaging.py", line 169, in publish
    compression, headers)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/messaging.py", line 252, in _prepare
    body) = dumps(body, serializer=serializer)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 54, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/vine/five.py", line 175, in reraise
    raise value.with_traceback(tb)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/utils/json.py", line 72, in dumps
    **dict(default_kwargs, **kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/__init__.py", line 237, in dumps
    **kw).encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 198, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 256, in iterencode
    return _iterencode(o, 0)
  File "/Users/kai/github-projects/Ticket-System/venv/lib/python3.5/site-packages/kombu/utils/json.py", line 62, in default
    return super(JSONEncoder, self).default(o)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 179, in default
    raise TypeError(repr(o) + " is not JSON serializable")
kombu.exceptions.EncodeError: <Flask 'src'> is not JSON serializable

The code for sending the email, is inside of my notifications.py

notifications.py

import json

from flask import render_template
from flask_mail import Message

from src import app
from src.config import mail

from celery import Celery

with open('src/config_values.json') as f:
    config_f = json.load(f)

celery = Celery('tasks', broker='amqp://localhost//')

@celery.task()
def send_async_email(app, msg):

    with app.app_context():
        mail.send(msg)


def send_email(subject, sender, recipients, html_body):

    msg = Message(subject, sender=sender, recipients=recipients)
    msg.html = html_body
    send_async_email.delay(app, msg)


def email_notification(cust_name, cust_email, tix):

    send_email("[Support Ticket #{tix}]!".format(tix=tix),
               config_f['MAIL_USERNAME'],
               [cust_email],
               render_template("ticket_email.html",
                               c_name=cust_name, tix=tix))

To initialize the email, I use the email_notification function inside of my views.py file

views.py

from src.notifications import email_notification

@app.route('/')
def home():
    form = TicketForm()

    if form.validate_on_submit() and request.method == 'POST':

        # Handle form here ....

        email_notification(cust_f_name, cust_email, tix_num)

EDIT:

This is my config.py file with all my app configurations and the __init__.py file is where I "create" the app:

config.py

import json
import os
import sys

from flask_admin import Admin
from flask_bootstrap import Bootstrap
from flask_mail import Mail
from flask_socketio import SocketIO
from flask_sqlalchemy import SQLAlchemy

from src import app

# JSON config file
with open('src/config_values.json') as f:
    config_f = json.load(f)
def db_uri(system):
    """
    Check system type for database URI setup

    :param system: `sys.platform()` will be passed in
    :return: system type
    """
    # Mac
    if system == 'darwin':
        uri = 'sqlite:////' + os.getcwd() + '/ticket_system.sqlite'
    # Windows
    elif system == 'win32':
        uri = r'sqlite:///' + os.getcwd() + '\ticket_system.sqlite'
    # Linux
    elif system == 'linux':
        uri = 'sqlite:////' + os.getcwd() + '/ticket_system.sqlite'
    # Linux2
    elif system == 'linux2':
        uri = 'sqlite:////' + os.getcwd() + '/ticket_system.sqlite'
    # If system could not be determined
    else:
        raise FileNotFoundError('SQLite File was not able to be found')
    # Return system type
    return uri


# All configuration needed for Flask
app.secret_key = os.urandom(24)
app.config['SQLALCHEMY_DATABASE_URI'] = db_uri(sys.platform)
app.config['DATABASE_FILE'] = config_f['DATABASE_FILE']
app.config['SQLALCHEMY_ECHO'] = config_f['SQLALCHEMY_ECHO']
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = config_f['SQLALCHEMY_TRACK_MODIFICATIONS']
app.config['MAIL_SERVER'] = config_f['MAIL_SERVER']
app.config['MAIL_PORT'] = config_f['MAIL_PORT']
app.config['MAIL_USERNAME'] = config_f['MAIL_USERNAME']
app.config['MAIL_PASSWORD'] = config_f['MAIL_PASSWORD']
app.config['MAIL_USE_TLS'] = config_f['MAIL_USE_TLS']
app.config['MAIL_USE_SSL'] = config_f['MAIL_USE_SSL']
app.config['RECAPTCHA_PUBLIC_KEY'] = config_f['cap_pub']
app.config['RECAPTCHA_PRIVATE_KEY'] = config_f['cap_sec']

Bootstrap(app)
db = SQLAlchemy(app)
mail = Mail(app)
admin = Admin(app, name='Tickets', template_mode='bootstrap3')
socketio = SocketIO(app)

__init__.py

from flask import Flask

app = Flask(__name__)
from src.views import app

This is my entire folder structure

Ticket-System\
    src\
        __init__.py
        config.json
        config_values.json
        decorators.py
        forms.py
        models.py
        notifications.py
        views.py
    venv\
    api_check.py
    requirements.txt
    run.py
like image 243
kstullich Avatar asked Dec 06 '22 14:12

kstullich

1 Answers

When you execute a task asynchronously with Celery, you send a message to the broker. So under the hood there is a serialization of the message, this means the arguments you pass to the task are also serialized. Since Celery v4.0 the default serializer is JSON.

In your task send_async_email you pass an argument app which is probably not JSON serializable that's why you have an error.

In my opinion, it's better to avoid to pass object instances to task if you can. So in your case here is what I would do :

notifications.py 

@celery.task()
def email_notification(cust_name, cust_email, tix):
    subject = "[Support Ticket #{tix}]!".format(tix=tix)
    sender = config_f['MAIL_USERNAME']
    recipients = [cust_email]
    html_body =  render_template("ticket_email.html", c_name=cust_name, tix=tix)

    msg = Message(subject, sender=sender, recipients=recipients)
    msg.html = html_body
    with app.app_context():
        mail.send(msg)

views.py

from src.notifications import email_notification
@app.route('/')
def home():
    form = TicketForm()

    if form.validate_on_submit() and request.method == 'POST':

        # Handle form here ....

        email_notification.delay(cust_f_name, cust_email, tix_num)
like image 116
arthur Avatar answered Dec 08 '22 04:12

arthur
Sign in to Comment

Related questions

PyQt5 - QThread: Destroyed while thread is still running
Converting object to Int pandas
How to create all combinations column wise for multiple variables in pandas?

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!