How to full-text index a Mercurial repository?

Question

What to do when hg log -k is not sufficient, and hg grep is just way too slow (cca. 100k changesets)? We have very bad experiences with Fisheye (way too slow), and Kiln seems to tie us into the FogCreek empire just a little bit too much.

What other options are there to provide full-text search capabilities over a repository?

Answer 1

What are you looking for in a full-text search? If you want to know the revision when text was added that's easier, and if you want to know all revisions in which text exists that's bigger.

Generally hg grep is as fast as you're going to get without pre-building an index, or at least pre-building versioned files you can use traditional grep on.

If you're willing to pre-build a greppable file structure you could do something like this:

hg export -o 'changeset-%r-%h.patch --rev 0:tip

That would export each changeset out to a textfile suitable for grepping using normal command line grep or indexing using lucene or similar. You could easily keep that current with a changeset hook.

Having only the changset diffs lets you look for revisions where text was added or removed, but not the list of all revisions where that text existed. For that you could pre-create a copy of every file at every revision, but that's a lot of space even if it's easily to automate.

Another option if you're looking for a specific revision where something happened is to make sure you're conversant with hg bisect. It automates a binary search for you, so if you want to find the first revision that has the string CHEESE you could do something like:

hg bisect --command "grep -s CHEESE" # might need to reverse the exit code of grep -s

though that updates your working dir which hg grep doesn't.

Answer 2

Have you looked at RhodeCode? -- http://demo.rhodecode.org/