Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Object store for objects in Django between requests

Tags:

python

django

I had the following idea: Say we have a webapp written using django which models some kind of bulletin board. This board has many threads but a few of them get the most posts/views per hour. The thread pages look a little different for each user, so you can't cache the rendered page as whole and caching only some parts of the rendered page is also not an option.

My idea was: I create an object structure of the thread in memory (with every post and other data that is needed to display it). If a new message is posted the structure is updated and every X posts (or every Y minutes, whatever comes first) the new messages are written back to the database. If the app crashes, some posts are lost, but this is definitely okay (for users and admins).

The question: Can I create such a persistent in memory storage without serialization (so no serialize->memcached)? As I understand it, WSGI applications (like Django) run in a continuous process without shutting down between requests, so it should be possible in theory. Is there any API I could use? If not: any point to look?

/edit1: I know that "persistent" usually has a different meaning, but in this case I strictly mean "in between request".

like image 894
Martin Thurau Avatar asked Aug 29 '09 12:08

Martin Thurau


People also ask

How object is stored in Django model?

To create an object, instantiate it using keyword arguments to the model class, then call save() to save it to the database. This performs an INSERT SQL statement behind the scenes. Django doesn't hit the database until you explicitly call save() . The save() method has no return value.

What kind of data is passed into a view in the request object?

The request object has view input field values in name/value pairs. When we create a submit button then the request type POST is created and calls the POST method. We have four data, those are in Name-Value pairs.

What is QueryDict Django?

class QueryDict. In an HttpRequest object, the GET and POST attributes are instances of django. http. QueryDict , a dictionary-like class customized to deal with multiple values for the same key. This is necessary because some HTML form elements, notably <select multiple> , pass multiple values for the same key.

What is request object in Django?

Django uses request and response objects to pass state through the system. When a page is requested, Django creates an HttpRequest object that contains metadata about the request. Then Django loads the appropriate view, passing the HttpRequest as the first argument to the view function.


1 Answers

In a production WSGI environment, you would probably have multiple worker processes serving requests at the same time. These worker processes would be recycled from time to time, meaning local memory objects would be lost.

But if you really need this (and make sure you do), I suggest you look into Django's caching framework, check out local-memory caching. Also, have a look at sessions.

But even the local-memory caching uses serialization (with pickle). It is easy to implement local-memory cache without serialization by implementing a custom cache back-end (see the docs). You could use the code in locmem.py as a starting point to create a cache without serialization.

But I suspect you are doing a bit of premature optimization here?

like image 92
codeape Avatar answered Sep 20 '22 12:09

codeape