How to fix 'collection1' which is not available due to init failure: Cannot create directory?

Question

I am installing solr 4.5.1 with tomcat 7 in Ubuntu 13.10.

I alway meet error:

org.apache.solr.core.CoreContainer; Unable to create core: collection1
org.apache.solr.common.SolrException: Cannot create directory: /opt/solr/example/solr/collection1/data

When I've created data and data/index folders in collection1 for Solr, it gave me error:

org.apache.solr.common.SolrException: Lock obtain timed out: NativeFSLock@/opt/solr/example/solr/collection1/data/index/write.lock: java.io.FileNotFoundException: /opt/solr/example/solr/collection1/data/index/write.lock (Permission denied)

I have tried jetty mode in same dictionary and it works fine.

The following is solrconfig.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<config>
<luceneMatchVersion>4.5</luceneMatchVersion>
<lib dir="../../../contrib/extraction/lib" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-cell-\d.*\.jar" />
<lib dir="../../../contrib/clustering/lib/" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-clustering-\d.*\.jar" />
<lib dir="../../../contrib/langid/lib/" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-langid-\d.*\.jar" />
<lib dir="../../../contrib/velocity/lib" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-velocity-\d.*\.jar" />
<dataDir>${solr.data.dir:}</dataDir>
...

the solrconfig.xml file is just same as the solr created one.

and the following is the messages in solr.log:

INFO  - 2013-11-06 11:12:59.479; org.apache.solr.servlet.SolrDispatchFilter;      SolrDispatchFilter.init()
INFO  - 2013-11-06 11:12:59.506; org.apache.solr.core.SolrResourceLoader; Using JNDI solr.home: /opt/solr/example/solr
INFO  - 2013-11-06 11:12:59.508; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/opt/solr/example/solr/'
INFO  - 2013-11-06 11:12:59.791; org.apache.solr.core.ConfigSolr; Loading container configuration from /opt/solr/example/solr/solr.xml
INFO  - 2013-11-06 11:13:00.180; org.apache.solr.core.ConfigSolrXml; Config-defined core root directory: 
INFO  - 2013-11-06 11:13:00.200; org.apache.solr.core.CoreContainer; New CoreContainer 2105334598
INFO  - 2013-11-06 11:13:00.200; org.apache.solr.core.CoreContainer; Loading cores into CoreContainer [instanceDir=/opt/solr/example/solr/]
INFO  - 2013-11-06 11:13:00.263; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 0
INFO  - 2013-11-06 11:13:00.264; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
INFO  - 2013-11-06 11:13:00.264; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 0
INFO  - 2013-11-06 11:13:00.264; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
INFO  - 2013-11-06 11:13:00.264; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
INFO  - 2013-11-06 11:13:00.265; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
INFO  - 2013-11-06 11:13:00.265; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
INFO  - 2013-11-06 11:13:00.265; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
INFO  - 2013-11-06 11:13:00.265; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
INFO  - 2013-11-06 11:13:00.283; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
INFO  - 2013-11-06 11:13:00.744; org.apache.solr.logging.LogWatcher; SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
INFO  - 2013-11-06 11:13:00.745; org.apache.solr.logging.LogWatcher; Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
INFO  - 2013-11-06 11:13:00.839; org.apache.solr.core.CorePropertiesLocator; Looking for core definitions underneath /opt/solr/example/solr
INFO  - 2013-11-06 11:13:00.920; org.apache.solr.core.CorePropertiesLocator; Found core collection1 in /opt/solr/example/solr/collection1/
INFO  - 2013-11-06 11:13:00.921; org.apache.solr.core.CorePropertiesLocator; Found 1 core definitions
INFO  - 2013-11-06 11:13:00.929; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /opt/solr/example/solr/collection1
INFO  - 2013-11-06 11:13:00.941; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/opt/solr/example/solr/collection1/'
INFO  - 2013-11-06 11:13:01.082; org.apache.solr.core.SolrConfig; Adding specified lib dirs to ClassLoader
INFO  - 2013-11-06 11:13:01.087; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/fontbox-1.8.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.087; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/poi-scratchpad-3.9.jar' to classloader
INFO  - 2013-11-06 11:13:01.087; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/netcdf-4.2-min.jar' to classloader
INFO  - 2013-11-06 11:13:01.087; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/xz-1.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.087; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/metadata-extractor-2.6.2.jar' to classloader
INFO  - 2013-11-06 11:13:01.088; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/bcprov-jdk15-1.45.jar' to classloader
INFO  - 2013-11-06 11:13:01.088; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/jempbox-1.8.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.088; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/poi-ooxml-3.9.jar' to classloader
INFO  - 2013-11-06 11:13:01.088; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/tika-parsers-1.4.jar' to classloader
INFO  - 2013-11-06 11:13:01.089; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/commons-compress-1.4.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.089; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/isoparser-1.0-RC-1.jar' to classloader
INFO  - 2013-11-06 11:13:01.089; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/apache-mime4j-core-0.7.2.jar' to classloader
INFO  - 2013-11-06 11:13:01.089; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/juniversalchardet-1.0.3.jar' to classloader
INFO  - 2013-11-06 11:13:01.090; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/icu4j-49.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.090; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/dom4j-1.6.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.090; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/tagsoup-1.2.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.090; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/pdfbox-1.8.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.090; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/xmlbeans-2.3.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/poi-3.9.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/tika-core-1.4.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/boilerpipe-1.1.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/poi-ooxml-schemas-3.9.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/xercesImpl-2.9.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.091; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/vorbis-java-core-0.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.092; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/vorbis-java-tika-0.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.092; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/rome-0.9.jar' to classloader
INFO  - 2013-11-06 11:13:01.092; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/jdom-1.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.092; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/bcmail-jdk15-1.45.jar' to classloader
INFO  - 2013-11-06 11:13:01.093; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/extraction/lib/apache-mime4j-dom-0.7.2.jar' to classloader
INFO  - 2013-11-06 11:13:01.125; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/dist/solr-cell-4.5.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.129; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/jackson-mapper-asl-1.7.4.jar' to classloader
INFO  - 2013-11-06 11:13:01.129; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/carrot2-mini-3.8.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.129; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/jackson-core-asl-1.7.4.jar' to classloader
INFO  - 2013-11-06 11:13:01.130; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/hppc-0.5.2.jar' to classloader
INFO  - 2013-11-06 11:13:01.130; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/mahout-collections-1.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.130; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/mahout-math-0.6.jar' to classloader
INFO  - 2013-11-06 11:13:01.130; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/attributes-binder-1.2.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.130; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/clustering/lib/simple-xml-2.7.jar' to classloader
INFO  - 2013-11-06 11:13:01.131; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/dist/solr-clustering-4.5.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.133; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/langid/lib/jsonic-1.2.7.jar' to classloader
INFO  - 2013-11-06 11:13:01.133; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/langid/lib/langdetect-1.1-20120112.jar' to classloader
INFO  - 2013-11-06 11:13:01.134; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/dist/solr-langid-4.5.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.135; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/velocity/lib/commons-beanutils-1.7.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.136; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/velocity/lib/velocity-tools-2.0.jar' to classloader
INFO  - 2013-11-06 11:13:01.137; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/velocity/lib/commons-collections-3.2.1.jar' to classloader
INFO  - 2013-11-06 11:13:01.137; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/contrib/velocity/lib/velocity-1.7.jar' to classloader
INFO  - 2013-11-06 11:13:01.137; org.apache.solr.core.SolrResourceLoader; Adding 'file:/opt/solr/dist/solr-velocity-4.5.1.jar' to classloader
INFO  - 2013-11-06 11:13:02.100; org.apache.solr.update.SolrIndexConfig; IndexWriter infoStream solr logging is enabled
INFO  - 2013-11-06 11:13:02.107; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_45
INFO  - 2013-11-06 11:13:02.421; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
INFO  - 2013-11-06 11:13:02.429; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
INFO  - 2013-11-06 11:13:02.648; org.apache.solr.schema.IndexSchema; [collection1] Schema name=example
INFO  - 2013-11-06 11:13:03.770; org.apache.solr.schema.IndexSchema; unique key field: id
INFO  - 2013-11-06 11:13:03.977; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
INFO  - 2013-11-06 11:13:03.996; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
INFO  - 2013-11-06 11:13:04.302; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
INFO  - 2013-11-06 11:13:04.313; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /opt/solr/example/solr/collection1/, dataDir=/opt/solr/example/solr/collection1/data/
INFO  - 2013-11-06 11:13:04.314; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3527c4a
INFO  - 2013-11-06 11:13:04.350; org.apache.solr.core.SolrCore; [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
INFO  - 2013-11-06 11:13:04.351; org.apache.solr.core.SolrCore; [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
INFO  - 2013-11-06 11:13:04.392; org.apache.solr.core.CachingDirectoryFactory; return new directory for /opt/solr/example/solr/collection1/data
INFO  - 2013-11-06 11:13:04.392; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/opt/solr/example/solr/collection1/data/index/
WARN  - 2013-11-06 11:13:04.393; org.apache.solr.core.SolrCore; [collection1] Solr index directory '/opt/solr/example/solr/collection1/data/index' doesn't exist. Creating new index...
INFO  - 2013-11-06 11:13:04.407; org.apache.solr.core.CachingDirectoryFactory; return new directory for /opt/solr/example/solr/collection1/data/index
INFO  - 2013-11-06 11:13:04.435; org.apache.solr.core.CachingDirectoryFactory; looking to close /opt/solr/example/solr/collection1/data/index [CachedDir<<refCount=0;path=/opt/solr/example/solr/collection1/data/index;done=true>>]
INFO  - 2013-11-06 11:13:04.441; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /opt/solr/example/solr/collection1/data/index
INFO  - 2013-11-06 11:13:04.441; org.apache.solr.core.SolrCore; [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1cb4ab3e
INFO  - 2013-11-06 11:13:04.442; org.apache.solr.update.SolrCoreState; Closing SolrCoreState
INFO  - 2013-11-06 11:13:04.442; org.apache.solr.update.DefaultSolrCoreState; SolrCoreState ref count has reached 0 - closing IndexWriter
INFO  - 2013-11-06 11:13:04.443; org.apache.solr.core.SolrCore; [collection1] Closing main searcher on request.
INFO  - 2013-11-06 11:13:04.443; org.apache.solr.core.CachingDirectoryFactory; Closing NRTCachingDirectoryFactory - 1 directories currently being tracked
INFO  - 2013-11-06 11:13:04.443; org.apache.solr.core.CachingDirectoryFactory; looking to close /opt/solr/example/solr/collection1/data [CachedDir<<refCount=0;path=/opt/solr/example/solr/collection1/data;done=false>>]
INFO  - 2013-11-06 11:13:04.444; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /opt/solr/example/solr/collection1/data
ERROR - 2013-11-06 11:13:04.445; org.apache.solr.core.CoreContainer; Unable to create core: collection1
org.apache.solr.common.SolrException: Cannot create       directory: /opt/solr/example/solr/collection1/data/index
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:834)
 Caused by: java.io.IOException: Cannot create  directory: /opt/solr/example/solr/collection1/data/index
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:171)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:695)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:506)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:755)
... 11 more
  ERROR - 2013-11-06 11:13:04.448; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: Unable to create core: collection1
at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:934)
Caused by: org.apache.solr.common.SolrException: Cannot create directory: /opt/solr/example/solr/collection1/data/index
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:834)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:625)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
... 8 more
Caused by: java.io.IOException: Cannot create directory: /opt/solr/example/solr/collection1/data/index
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:171)
... 11 more

 INFO  - 2013-11-06 11:13:04.451; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/var/lib/tomcat7
INFO  - 2013-11-06 11:13:04.451; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
ERROR - 2013-11-06 11:13:05.373; org.apache.solr.update.SolrIndexWriter; SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
ERROR - 2013-11-06 11:13:05.400; org.apache.solr.update.SolrIndexWriter; Error closing IndexWriter, trying rollback

---

Does someone have idea? Thanks

I find the problem is because of folder's priviage. after I changed the owner of $SOLR_HOME and it's subdictionary to tomcat7, it worked.

Answer 1

Quoting SOLR-3753 issue:

"In the absence of a solr.xml file, current 4.0 behavior is to use defaults declared in CoreContainer.java. This needs to be documented; probably in solr.xml and/or on the CoreAdmin page. (See line 94 of CoreAdmin.java where the default name "collection1" is declared. Without this documentation, users can get confused about where the "collection1" core name is coming from. (I'm one of them)."

Hence, a simple solution is to create a solr.xml file under $SOLR_HOME/examples/solr/

Reference SOLR-3753

Answer 2

The folder must be writeable for the server that runs Solr.

So check /opt/solr/example/solr/collection1/.

Folder permissions and enable writing, something like:

sudo chmod a+w /opt/solr/example/solr/collection1/ -R

or change the folder owner:

sudo chown tomcat7. /opt/solr/example/solr/collection1/ -R

Answer 3

Check permissions and ensure that the solr home directory and subdirectories are readable and writable by tomcat7. Then in you solrconfig.xml file that you posted above, see the line: <dataDir>${solr.data.dir:}</dataDir>

Replace ${solr.data.dir:} with the path to the data directory, which you also have to create and ensure read/write permissions, to look like this:

<dataDir>${solr.data.dir:/home/solr/solr-4.6.0/example/solr/collection1/data}</dataDir>

Then save, and restart the solr application in the Tomcat manager, or just: #sudo service tomcat7 restart`

These were my commands and paths I set using Tomcat 7 and Solr 4.6, so yours may be different. I created a solr user and added him to the tomcat 7 group, but there are numerous other ways to make the permissions work and maintain security. I also had to create a solr.xml file in /etc/tomcat7/Catalina/localhost with tomcat7 ownership that had the following lines:

<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/home/solr/solr-4.6.0/dist/solr-4.6.0.war" debug="0" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/home/solr/solr-4.6.0/example/solr" override="true"/>
</Context>

Again, stop and start solr in the Tomcat manager GUI. Hope this helps!