Where in Django can I run startup to load data?

Question

I have a django application that loads a huge array into memory after django is started for further processing.

What I do now is , after loading a specific view, I execute the code to load the array as follows:

try:
    load_model = load_model_func()
except:
    #Some exception
    

The code is now very bad. I want to create a class to load this model once after starting django and I want to be able to get this model in all other this model in all other methods of the application

So, is there a good practice for loading data into memory after Django starts?

Answer 1

@Ken4scholar 's answer is a good response in terms of "good practice for loading data AFTER Django starts". It doesn't really address storing/accessing it.

If your data doesn't expire/change unless you reload the application, then using cache as suggested in comments is redundant, and depending on the cache storage you use you might end up still pulling from some db, creating overhead.

You can simply store it in-memory, like you are already doing. Adding an extra class to the mix won't really add anything, a simple global variable could do.

Consider something like this:

# my_app/heavy_model.py
data = None

def load_model():
    global data
    data = load_your_expensive_model()

def get_data():
  if data is None:
      raise Exception("Expensive model not loaded")
  return data

and in your config (you can lean more about applications and config here):

# my_app/app_config.py
from django.apps import AppConfig

from my_app.heavy_model import load_model

class MyAppConfig(AppConfig):
    # ...
    def ready(self):
        load_model()

then you can simply call my_app.heavy_model.get_data directly in your views.

⚠️ You can also access the global data variable directly, but a wrapper around it may be nicer to have (also in case you want to create more abstractions around it later).

Answer 2

You can run a startup script when apps are initialized. To do this, call the script in the ready method of your Config class in apps.py like this:

class MyAppConfig(AppConfig):
    name = 'myapp'

    def ready(self):
        <import and run script>