reindex Catalog URL Rewrites in magento never ends

Question

I've imported 6K categories and 16K products to magento using a custom import profile. When I'm trying to reindex everything works except for 'Catalog URL Rewrites' that keep showing PROCESSING but never completes.

log and exceptions files don't show anything.

Is there something I can do to make the index work? Can I just ignore this index and not use it? (I don't know what it does).

Thanks

Answer 1

You can sort of ignore this index if you do not care about pretty, search engine "friendly" URL's. The products will still appear in the catalog but will have their default "Zend Framework" type URL (/catalog/product/view/id/123)

My store has 150,000+ SKUs in two store views. On my development environment it took almost two weeks to complete.

The best way to determine how fast it is running is to look at the core_url_rewrite database table. It appears that the process starts at product ID 1 in store ID 1 and looks to make sure all of its URLs exist and creates the ones that do not yet exist.

For me the reason it took so long was that it had to go through my entire catalog twice to make all the URLs. One thing I did notice was that this process creates a ton of URLs that are completely unnecessary. In our store easily 90% of our products belong to a configurable product so their visibility is set to "Not Visible Individually" so they never would need to have a URL. This index creates those URLs anyways.

Hopefully this will shed some light on to how this URL works. I would keep an eye on that database table so you sorta know how far the process has come. I would also seriously consider running the reindex process for the command line. I have included a link that explains how to do this (disclosure: it's from my blog)

http://overlycaffeinated.com/2011/02/when-reindexing-in-magento-fails-use-the-command-line/