How to speed up OpenGrok indexing

Question

lately I was asked by my boss to explore OpenGrok possibilities in the company I'm working for. First I started with a few projects at my virtualbox lubuntu, it was working ok, but kind of slowly. I blamed my laptop with mediocre parameters for that.

Now I'm having virtual of bigger proportions and I'm also running indexing on larger volume of data (SVN repository - 100 different projects, some of them with multiple branches, tags and trunk, about 100 000 files in total, few GB in size). All files are checked out directly in the SRC_ROOT.

I was hoping for reasonably fast indexing, but it's been running for more than five days now. I can see multiple threads running via htop, but CPU usage is 0.5-2.5%, memory usage 0.9%. So I guess it's not an issue of computing power. And unless there are terribly slow HDDs I don't know what the problem is.

Furthemore the indexing process seems to be slowing down. At the beginning it was approximately 1 sec/file, now it is about 5 sec/file. Unfortunately I haven't triggered the progress option, so I have no idea how long it's still going to run.

Any ideas how to make indexing faster? How to use resources more effectively? Current speed is simply unusable...

Answer 1

I think easy way to improve performance is to run opengrok index with setting up JAVA_OPTS and using 64 bit java. Also, using derby for storing generated index data increase performance too. More info about how to use and setup opengrok

• https://github.com/OpenGrok/OpenGrok/blob/master/README.txt#L862
• https://java.net/projects/opengrok/lists/discuss/archive/2013-03/thread/1#00000