Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flask not releasing memory

Tags:

python

flask

I'll begin this question with pointing out that I'm very new to web development.

Now to the point: I've recently built a rather small Flask application that loads data and then outputs visualizations of it using bokeh. And as such it will have to keep quite a lot of data stored in memory (about 10-20 mb). This in itself is not really a problem, however, the application does not release the objects in memory after the view functions have sent the requests. Which means that after just a few uses the app will have drained most of the memory.

My question is therefore: How do I force Flask to release the used objects after the view functions have returned any requests? Or am I going about this problem the wrong way? It could be worth mentioning that I use the built in server of Flask as we're still only prototyping. ¨ Thanks, Tingis

EDIT Here's one of my view functions. What it does is that it loads data from a db using SQLAlchemy and then does some timeseries manipulations (such as a kernel density estimate and calculating cumulative return) and outputs a joined div and script string of a bokeh figure, which are the variables ending with _plt.

from app import app
from app.business_logic.classes.interface_classes import Company
from app.business_logic.functions.functions import get_session

@app.route('/analysis_tool/company_performance', methods=['GET', 'POST'])
def analysis_tool__company_performance():
    session = get_session()

    companies_to_analyse = {
        'Company A': {'ts_to_use': 'Returns of Company A'},
        'Company B': {'ts_to_use': 'Returns of Company B'}
    }

    chosen_company = request.form.get('security')
    types_of_plots = {}

    if chosen_company is not None:
        company = Company(session, chosen_company)
        company.load_company()

        company.load_timeseries(companies_to_analyse[chosen_platform]['ts_to_use'])
        company.unpack_observations_of_ts_as_df()

        ret_df = company.manipulate_dataframe('convert timeseries to returns',
                                               frequency='monthly',
                                               ts_type=company.loaded_ts_orm_obj.ts_type_name)
        cum_ret_df = company.manipulate_dataframe('calculate cumulative return', df=ret_df)

        cum_ret_plt = company.plot_dataframe(cum_ret_df, legend=False)
        kde_plt = company.plot_kde(ret_df)

        types_of_plots = {'Cumulative_return': cum_ret_plt, 'KDE': kde_plt}

    return render_template('plotting/plot_using_panels.html',
                           items=sort_dictionary(platforms_to_analyse),
                           plot_types=sort_dictionary(types_of_plots),
                           selected=chosen_company)

Does this help?

EDIT 2 I tried the solution that was provided in the following question, by adding a gc.collect() after each call to the plotting interface, as well as the function

@app.teardown_request
def teardown_request(exception):
    gc.collect()

But the memory still isn't released.

like image 919
Tingiskhan Avatar asked Jan 14 '16 16:01

Tingiskhan


1 Answers

Is it possible you are creating a circular reference between get_session and the Company object?

like image 69
Cory Dolphin Avatar answered Sep 23 '22 16:09

Cory Dolphin