Keyword (OR, AND) search in Lucene

Question

I am using Lucene in my portal (J2EE based) for indexing and search services.

The problem is about the keywords of Lucene. When you use one of them in the search query, you'll get an error.

For example:

searchTerms = "ik OR jij"

This works fine, because it will search for "ik" or "jij"

searchTerms = "ik AND jij"

This works fine, it searches for "ik" and "jij"

But when you search:

searchTerms = "OR"
searchTerms = "AND"
searchTerms = "ik OR"
searchTerms = "OR ik"

Etc., it will fail with an error:

Component Name: STSE_RESULTS  Class: org.apache.lucene.queryParser.ParseException  Message: Cannot parse 'OR jij': Encountered "OR" at line 1, column 0. 
Was expecting one of: 
... 

It makes sense, because these words are keywords for Lucene are probably reserved and will act as keywords.

In Dutch, the word "OR" is important because it has a meaning for "Ondernemings Raad". It is used in many texts, and it needs to be found. For example "or" does work, but does not return texts matching the term "OR". How can I make it searchable?

How can I escape the keyword "or"? Or How can I tell Lucene to treat "or" as a search term NOT as a keyword.

Answer 1

I suppose you have tried putting the "OR" into double quotes?

If that doesn't work I think you might have to go so far as to change the Lucene source and then recompile the whole thing, as the operator "OR" is buried deep inside the code. Actually, compiling probably isn't even enough: you'll have to change the file QueryParser.jj in the source package that serves as input for JavaCC, then run JavaCC, then recompile the whole thing.

The good news, however, is that there's only one line to change:

| <OR: ("OR" | "||") >

becomes

| <OR: ("||") >

That way, you'll have only "||" as logical OR operator. There is a build.xml that also contains the invocation of JavaCC, but you have to download that tool yourself. I can't try it myself right now, I'm afraid.

This is perhaps a good question for the Lucene developer mailing list, but please let us know if you do that and they come up with a simpler solution ;-)