Tags or Search or Both

Question

Example of my project.

Many Users Many Entries (think twitter size)

I want to make it very easy for the users to search their entries. The question is: Do I implement a tagging system or just a search mechanism? I've done a fair amount of research on the subject, specifically about the schema's required. But, I still have some lingering questions.

If I uses TAGS. The idea would be I strip the entry into words and whatever meta data that comes with it into TAGS. The schema would be the Toxi (many to many) thing as recommended by the mysql folks. The problem I have with TAGS is they aren't as flexible as SEARCH. For example, if CATS were a tag but CAT wasn't you couldn't "search" for it because it's not a TAG. Unless, you implement search ON the TAGS as well. At which point I worry about the performance or why not just use straight up search. The second problem is keeping TAG duplicates and similarities down to a minimum. TAG garbage collection if you will.

If I use SEARCH I would have to use LIKE or FULLTEXT (myISM isn't that good though) and search trough the entries and their metadata. The metadata could store tag like data in this scenario. The scheme would be much simpler but I fear performance would be much worse than using tags. But, again search is more flexible and I wouldn't have to worry about garbage collecting tags as much.

Now, what's interesting is that I have read that to improve search performance people have taken to the search + tagging approach. Tagging all the words in the entry and then searching the TAGS. Which I kind of hinted at happening with TAG system anyway. At this point the line between tagging and searching seems to be getting fuzzy and I am getting really confused. So, I am writing this in hope you can sort me out.

I have heaps of users and heaps of small text entries. What's the best way for the users to be able to search it. Both from a performance perspective as well as a user interaction perspective.

Also, any additional information on this subject would be greatly appreciated.

Cheers.

Answer 1

I'd suggest you using a dedicated search engine for this feature, for example Sphinx or KinoSearch. Obtaining relevant search results while having maximum flexibility regarding search queries is a business in its own (google's business for example), so I would always adopt a dedicated service for this task