Thinking Sphinx - reindexing on a per-table basis

Question

Reindexing the Thinking Sphinx indices in my development configuration takes several hours, mostly because of several huge tables (we're not using delta updates). Even when I'm changing the index for another, much smaller table, reindexing takes well over an hour because of the presence of the large tables.

Is there any way to force TS to re-index only a specified table in the db?

Answer 1

There's no way to tell Thinking Sphinx to process one index at a time, but it's certainly possible (and not particularly difficult) via Sphinx's indexer tool directly.

indexer --config config/development.sphinx.conf model_core

A few things to note: unless you're giving your index definitions custom names, it'll default to the model name, downcased and underscored, with the _core suffix. If you had delta indices in play, they'd have a _delta suffix. Also, add the --rotate flag to that call if Sphinx is running.

You can specify as many indices as you like. Should you ever want to process all indices at once (just like Thinking Sphinx's rake task), --all will do that for you.

As an aside - how many records are we talking about? How complex are your index definitions? Several hours for index processing is not common at all. Do you have database indexes on any foreign key columns used by associations referenced in the Sphinx indices?