Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django: How can you stop long queries from killing your database? [closed]

I'm using Django 1.1 with Mysql 5.* and MyISAM tables.

Some of my queries can take a TON of time for outliers in my data set. These lock the tables and shut the site down. Other times it seems some users cancel the request before it is done and some queries will be stuck in the "Preparing" phase locking all other queries out.

I'm going to try to track down all the corner cases, but its nice to have a safety net so the site doesn't come down.

How do I avoid this? Can I set maximum query times?

like image 275
Paul Tarjan Avatar asked Aug 30 '09 06:08

Paul Tarjan


People also ask

Does Django close DB connection?

At the end of each request, Django closes the connection if it has reached its maximum age or if it is in an unrecoverable error state. If any database errors have occurred while processing the requests, Django checks whether the connection still works, and closes it if it doesn't.

Can a SQL query be too long?

Issue. When adding a calculated field to a view, the following error may occur: The query is too large. The maximum standard SQL query length is 1024.00K characters, including comments.


1 Answers

Unfortunately MySQL doesn't allow you an easy way to avoid this. A common method is basically to write a script that checks all running processes every X seconds (based on what you think is "long") and kill ones it sees are running too long. You can at least get some basic diagnostics, however, by setting log_slow_queries in MySQL which will write all queries that take longer than 10 seconds into a log. If that's too long for what you regard as "slow" for your purposes, you can set long_query_time to a value other than 10 to change the threshold.

like image 189
Nick Bastin Avatar answered Oct 16 '22 03:10

Nick Bastin