Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Solr 'Invalid Date String' Exception

Tags:

date

utc

solr

I'm new to Solr. I successfully indexed some data, but after passing a date attribute to an appropriate solr field I'm receiving an exception:

I'm passing the String "15.06.2011 13:51:41", which is a common date format for countries like Germany. However, after some googling I found out, that Solr expects a date to be in the UTC format.

My specific question is: Can I transform my format to UTC at some point at Solr? Would this be a case for the DateFormatTransformer or is it a MUST to pass it in the correct format initially?

EDIT: I am not using a DataImportHandler. I'm using the DirectUpdateHandler2 by passing the data directly to Solr. Since it's called 'direct update', are my chances pretty bad in transforming anything there?

For the records, the exception I got was:

ERROR - 2013-09-13 15:52:07.705; org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: Invalid Date String:'15.06.2011 13:51:41'
    at org.apache.solr.schema.DateField.parseMath(DateField.java:182)
    at org.apache.solr.schema.TrieField.createField(TrieField.java:616)
    at org.apache.solr.schema.TrieField.createFields(TrieField.java:655)
    at org.apache.solr.schema.TrieDateField.createFields(TrieDateField.java:157)
    at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
    at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
    at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73)
    at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:210)
    at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
    at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
    at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)
like image 373
Benjamin Brandmeier Avatar asked Sep 16 '13 15:09

Benjamin Brandmeier


1 Answers

According to the Solr DateField documentation, this format is required:

date field shall be of the form 1995-12-31T23:59:59Z The trailing "Z" designates UTC time and is mandatory (See below for an explanation of UTC). Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored). All other parts are mandatory.

The only option is to transform the date prior to sending it into Solr. If you are using DataImportHandler, it would be possible within that context/process.

like image 68
Paige Cook Avatar answered Oct 04 '22 02:10

Paige Cook