Magento reindexing loses Solr docs

Question

This is driving me a bit crazy. I'm running Magento EE 1.11.1 with SOLR running. We have a cron that runs every night which reindexes the entire site. Every time it does this, I check the SOLR configuration and the numDocs and maxDocs values are only a fraction of what should be currently indexed (27000 versus ~90000). This means that when I do a search on the site, the results are only a fraction of what they should be.

The only way to get search working properly is to stop SOLR, delete and recreate the /apache-solr/site_name/solr/data folder, restart, and reindex just the Catalog Search index via shell. If I try to run this specific reindex via shell without deleting and recreating the data folder, I get only about half of the docs that I should be getting (~51000).

All index files in the data folder are owned by root, and the SOLR jar is running as root. I have all logs set to warning but nothing is currently being logged. I manage other sites using Solr and never have this issue - this install however has many attributes (330) and many products (~100,000). Could that be part of the issue? Thanks!

Answer 1

EE1.12 may not be a solution either. We have a client on EE1.12 that is having problems with the SOLR integration. In their case, all attempts to index fail when the indexer accesses custom product attributes.

Nexcess and Magento support have been working on this for over 6 weeks, the current status from Magento support is -

Unfortunately the patch is still under development, and I'm unable to advise on when our developer will complete it.