Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Quickly select random ID from mysql table with millions of non-sequential records

I've looked around and there doesnt seem to be any easy way to do this. It almost looks like it's easier just to grab a subset of records and do all the randomizing in code (perl). The methods I've seen online seem like theyre geared more to at most hundreds of thousands, but certainly not millions.

The table I'm working with has 6 million records (and growing), the IDs are auto incremented, but not always stored in the table (non-gapless).

I've tried to do the LIMIT 1 query that's been recommended, but the query takes forever to run -- is there a quick way to do this, given that there are gaps in the record? I can't just take the max and randomize over the range.

Update:

One idea I had maybe was to grab the max, randomize a limit based on the max, and then grab a range of 10 records from random_limit_1 to random_limit_2 and then taking the first record found in that range.

Or if I know the max, is there a way i can just pick say the 5th record of the table, without having to know which ID it is. Then just grabbing the id of that record.

Update:

This query is somewhat faster-ish. Still not fast enough =/

SELECT t.id FROM table t JOIN (SELECT(FLOOR(max(id) * rand())) as maxid FROM table) as tt on t.id >= tt.maxid LIMIT 1
like image 266
qodeninja Avatar asked Dec 09 '11 17:12

qodeninja


People also ask

How can you pick random records from SQL table?

To get a single row randomly, we can use the LIMIT Clause and set to only one row. ORDER BY clause in the query is used to order the row(s) randomly. It is exactly the same as MYSQL. Just replace RAND( ) with RANDOM( ).

How can we get a random number between 1 and 100 in MySQL?

To create a random integer number between two values (inclusive range), you can use the following formula: SELECT FLOOR(RAND()*(b-a+1))+a; Where a is the smallest number and b is the largest number that you want to generate a random number for.


1 Answers

SELECT * FROM TABLE ORDER BY RAND() LIMIT 1;

Ok, this is slow. If you'll search for ORDER BY RAND() MYSQL, you will find alot of results saying that this is very slow and this is the case. I did a little research and I found this alternative MySQL rand() is slow on large datasets I hope this is better

like image 51
cristian Avatar answered Oct 20 '22 22:10

cristian