Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MySQL full-text search: need fast insert and fast search

I have a mysql database where users can input text. They then need to be able to search for this text. I have just implemented mysql full texts search and it's definitely made the searches a lot faster.

However, it has, not surprisingly, made the inserts slower. But I was surprised at how much slower. A single insert can take .5 - 1.5 seconds.

The table has 3 indexed columns:

title (max length 200)
description (max length 3000)
content  (max length 10000)

I only have about 2000 records in the table at this point, which is nothing compared to what it will be later.

Any suggestions? How is this issue normally handled? Is it normal for an insert to take so long?

I don't need all the capabilities of full text search. I really just need the equivalent of AND, OR, -, +, " ". So no weights etc. Can that indexing be turned of to make it faster?

like image 505
user984003 Avatar asked Nov 02 '22 17:11

user984003


1 Answers

Answer based on eggyal's comment.

I ended up installing Sphinx. It's awesome. I'm using its real time indexing. Fast search and fast insert. Way faster than mysql free-text insert/search. Granted, my database is pretty small. They have tricks for fast real time indexing with large databases (split into 2 indexes, one with old data, one with newer data), but I have not needed that.

BTW: I'm using Python/Django and did not not need to install any api or library, besides sphinx itself:

import MySQLdb
connection = MySQLdb.connect(host='127.0.0.1', port = 9306) 
cursor = connection.cursor()
cursor.execute("select id from my_index where match('stackoverflow')")
results = cursor.fetchall()
# I use my regular connection to insert the IDs into a table in my regular database and then join with that to get actual data.
like image 124
user984003 Avatar answered Dec 10 '22 11:12

user984003