Filtering and sorting using a function in solr

Question

I have documents that look like this:

{attr_a: 1}, {attr_a: 2, changes_user1_a: 3}

Second document says that user1 made a change that is not yet "committed" and not visible to other users.

The fields are dynamic:

<dynamicField name="changes_*" type="string" indexed="true" stored="true"/>
<dynamicField name="attr_*" type="string" indexed="true" stored="true"/>

The thing is, user1 needs to do filtering, paging and sorting taking into account his new values, while other users need to do it on the old values.

I was thinking of resolving this problem the following way: if user1 wants to filter and sort by attr_a, I would issue a query like

fq=def(changes_user1_a, attr_a): 'somefilter'&sort=def(changes_user1_a, attr_a) asc

However, this approach generates the following error for fq:

"error":{
"metadata":[
  "error-class","org.apache.solr.common.SolrException",
  "root-error-class","org.apache.solr.parser.ParseException"],
"msg":"org.apache.solr.search.SyntaxError: Cannot parse 'def(changes_user1_a, attr_a): 3': Encountered \" \":\" \": \"\" at line 1, column 28.\r\nWas expecting one of:\r\n    <EOF> \r\n    <AND> ...\r\n    <OR> ...\r\n    <NOT> ...\r\n    \"+\" ...\r\n    \"-\" ...\r\n    <BAREOPER> ...\r\n    \"(\" ...\r\n    \"*\" ...\r\n    \"^\" ...\r\n    <QUOTED> ...\r\n    <TERM> ...\r\n    <PREFIXTERM> ...\r\n    <WILDTERM> ...\r\n    <REGEXPTERM> ...\r\n    \"[\" ...\r\n    \"{\" ...\r\n    <LPARAMS> ...\r\n    \"filter(\" ...\r\n    <NUMBER> ...\r\n    ",
"code":400}

And when sorting only, this one:

error":{
"trace":"java.lang.UnsupportedOperationException\r\n\tat org.apache.lucene.queries.function.FunctionValues.doubleVal(FunctionValues.java:47)\r\n\tat org.apache.lucene.queries.function.ValueSource$ValueSourceComparator.copy(ValueSource.java:153)\r\n\tat org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:206)\r\n\tat org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:56)\r\n\tat org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)\r\n\tat org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)\r\n\tat org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)\r\n\tat org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:260)\r\n\tat org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1810)\r\n\tat org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)\r\n\tat org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)\r\n\tat org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)\r\n\tat org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)\r\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\r\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\r\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\r\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\r\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\r\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:518)\r\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\r\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\r\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\r\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\r\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\r\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n",
"code":500}

Although fl=test:def(changes_user1_a, attr_a) works like I would expect.

So my questions would be:

1. Am I taking the right approach in solving this problem?
2. If yes, why can't I sort and filter with the def function?

I am using Solr version 6.1.0

Thanks

Answer 1

For a filter with a function you have to use the frange parser. See e.g.: solr: Create filter query with a function .

For example following query will return all docs having changes_user1_a=3, or attr_a=3 if changes_user1_a field doesn't exist.

fq={!frange u=3 l=3}def(changes_user1_a, attr_a)

---

Your sorting command seems ok to me, but it apparently doesn't work for string fields. I can reproduce your exception. It woks when both the fields are numeric. Maybe bug in solr?

 null:java.lang.UnsupportedOperationException
    at org.apache.lucene.queries.function.FunctionValues.doubleVal(FunctionValues.java:47)
    at org.apache.lucene.queries.function.valuesource.DefFunction$1.doubleVal(DefFunction.java:88)
    at org.apache.lucene.queries.function.ValueSource$ValueSourceComparator.copy(ValueSource.java:264)
    at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:141)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:246)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:197)
    at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
    at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:225)
    at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:2027)
    at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1844)
    at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
 ...

See also:

• Solr sort by the value of function query does not work
• Apache Solr: sort by one of fields (with condition)
• How can I sort solr result bases on dynamic fields