Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django how to debug a frozen save operation on a queryset object

Tags:

python

django

I have the following code in a Django project (within the create method of a Django Rest Framework serializer)

def create(self, validated_data):
     <...>
     log.info("\n\n\n")
     log.info(f"django model: {self.Meta.model}")
     log.info("CREATING CASE NOW .....")
     case = self.Meta.model(**kwargs)
     log.info(f"Case to be saved: {case}")
     case.save()
     log.info(f"Case object Created: {case}")

When I'm posting to the endpoint, it's just freezing up completely on .save(). Here's example output:

2020-06-15 02:47:46,008 - serializers - INFO ===> django model: <class 'citator.models.InternalCase'>
2020-06-15 02:47:46,008 - serializers - INFO ===> django model: <class 'citator.models.InternalCase'>
2020-06-15 02:47:46,009 - serializers - INFO ===> CREATING CASE NOW .....
2020-06-15 02:47:46,009 - serializers - INFO ===> CREATING CASE NOW .....
2020-06-15 02:47:46,010 - serializers - INFO ===> Case to be saved: seychelles8698
2020-06-15 02:47:46,010 - serializers - INFO ===> Case to be saved: seychelles8698

No error is thrown and the connection isn't broken. How can I debug this? Is there a way to get logging from the save method?

like image 991
Neil Avatar asked Jun 15 '20 00:06

Neil


People also ask

How does queryset work in Django?

The first time a QuerySet is evaluated – and, hence, a database query happens – Django saves the query results in the QuerySet ’s cache and returns the results that have been explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the QuerySet reuse the cached results.

How do I retrieve objects from the database in Django?

Django will complain if you try to assign or add an object of the wrong type. To retrieve objects from your database, construct a QuerySet via a Manager on your model class. A QuerySet represents a collection of objects from your database. It can have zero, one or many filters. Filters narrow down the query results based on the given parameters.

How to save an object in a queryset?

A queryset isn't a single object, it's a group of objects so it doesn't make sense to call save () on a queryset. Instead you save each individual object IN the queryset: game_participants = GameParticipant.objects.filter (player=player, game=game) for object in game_participants: object.save ()

How can I see the underlying query in Django?

If you don’t want to log all queries, but you have access to a QuerySet object you can call .query and print the result to see the underlying query: If you want even more information, there’s a great app you can add to your project called the Django Debug Toolbar which lets you see the queries happening while you navigate around your website.


Video Answer


2 Answers

The error likely unrelated to the use of the Django rest serializers as the code that hangs simple creates a new model and saves it. Now you did not specify how kwargs is defined, but the most likely candidate is that it gets stuck talking to the DB.

To debug the code, you should learn how to step in the code. There are a number of options depending on your preferences.

Visual studio code

Install the debugpy package.

Run python3 -m debugpy --listen localhost:12345 --pid <pid_of_django_process>

Run the "Python: Remote Attach" command.

CLI

Before the line case.save() do

import pdb; pdb.set_trace()

This assumes you are running the Django server interactively and not e.g. through gunicorn. You will get a debug console right before the save line. When the console appears, type 'c' and press enter to continue execution. Then press Ctrl+C when the process appears stuck. Type bt to find out what goes on in the process.

Native code

If the stack trace points to native code, you could switch over to gdb. To debug this (make sure to exit any Python debugger or restart the process without a debugger). Run

gdb -p <pid_of_django>

when the process appears stuck. Then type 'bt' and press enter to get a native traceback of what is going on. This should help you identifiy e.g. database clients acting up.

like image 68
Krumelur Avatar answered Oct 21 '22 02:10

Krumelur


It is very probable that Django is waiting for a response from database server and it is a configuration problem, not a problem in the Python code where it froze. It is better to check and exclude this possibility before debugging anything. For example it is possible that a table is locked or an updated row is locked by another frozen process and the timeout in the database for waiting for end of lock is long and also the timeout of Django waiting for the database response is very long or infinite.

It is confirmed if a similar save operation takes an abnormally long time in another database client, preferably in your favorite database manager.

Waiting for socket responce is excluded if you see CPU % activity of the locked Python process.

It may be easier explored if you can reproduce the problem in CLI by python manage.py shell or python manage.py runserver --nothreading --nothreading. Then you can press Ctrl+C and maybe after some time Ctrl+C again. If you are lucky you kill the process and will see a KeyboardInterrupt with a traceback. It helps you identify if the process was waiting for something else than for a database server socket response.

Another possible cause in Django could be related to a custom callback code connected to a pre_save or post_save signal.

Instead of plain python manage.py ... you can run python -m pdb manage.py ... and optionally set a break point or simply press "c" "Enter" (continue). The process will run and will be not killed after any exception, but stay in pdb (the native Python DeBugger).

like image 2
hynekcer Avatar answered Oct 21 '22 02:10

hynekcer