Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to tune EGit for large repositories?

Tags:

egit

jgit

The problem:

I find EGit great and use it intensively, but it can be incredibly slow. It can get frustrating when it takes several minutes to complete operations that the C version of git (Cgit) does in less than a couple of seconds.

All operations are significantly slower than Cgit. For example switching branches will take 10's of seconds compared to near instant. A rebase can take several minutes compared to less than a couple of seconds.

Some details:

History size: 10114 commits as reported with: git rev-list HEAD --count

Current Working directory size: 63.7 MB

Current .git size: 77.4 MB

Largest file size: 4.0 MB

OS: Linux - CentOS 5.5

File System: ext3

JVM: Oracle - Java(TM) SE Runtime Environment (build 1.7.0_21-b11)

EGit and JGit version: 3.0.0.201306101825-r

I was previously running 2.3 but did not notice any change in performance after upgrading.

Could suitable window cache settings help:

I found the following quote in JGit's bugzilla here:

...EGit had to expose UI to allow users to configure it when working on bigger repositories.

Which sounds like it fits my case. So I looked around in eclipse and under Window -> Preferences -> Team -> Git found these Git Window Cache settings:

EGit Window Cache Settings

But how do I use them?

What do the different controls actually do? Has anyone had any success in getting EGit to be more responsive by using them?

like image 648
PeterSW Avatar asked Aug 14 '13 01:08

PeterSW


3 Answers

Recommendations

As Matthias Sohn suggested, the Window cache limit appears to be most significant of these parameters.

For me, increasing this from "10 m" to "500 m" made a huge difference to how responsive egit was.

Details of each parameter

From the source code of WindowCacheConfig.java:

Windows Size

packedGitWindowSize: size in bytes of a single window mapped or read in from the pack file

Default: 8 k

Window cache limit

packedGitLimit: maximum number bytes of heap memory to dedicate to caching pack file data.

Default: 10 m

Delta base cache limit

deltaBaseCacheLimit: maximum number of bytes to cache in delta base cache for inflated, recently accessed objects, without delta chains.

Default: 10 m

Stream File Threshold

streamFileThreshold: the size threshold beyond which objects must be streamed.

Objects smaller than this size can be obtained as a contiguous byte array, while objects bigger than this size require using anObjectStream.

Default: 50 m

Use virtual memory mapping

packedGitMMAP: true enables use of Java NIO virtual memory mapping for windows; false reads entire window into a byte[] with standard read calls.

Default: Unchecked

Not presented on the preferences page

packedGitOpenFiles: maximum number of streams to open at a time. Open packs count against the process limits.

Default: 128


† Thanks Jens Theeß for their comment on Matthias Sohn's answer containing a pointer to the source code.

like image 175
Mark Booth Avatar answered Oct 16 '22 14:10

Mark Booth


EGit 3.5.0 will bring huge performance fix for large repositories - without "tuning" anything. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=440722

You can use nightly EGit build update site to get the fix immediately: http://download.eclipse.org/egit/updates-nightly

like image 42
Andrey Loskutov Avatar answered Oct 16 '22 14:10

Andrey Loskutov


  • increase the window cache limit to a larger value, it's defining the size of jgit's cache used to map pack files into memory
  • do you run gc (either from egit or native git) on your repository ? JGit/EGit don't run it automatically (yet). YOu may check the number of loose object in EGit's repositories view (click "Properties" and select tab "Statistics")
  • how many commits does your repository have (age in year doesn't say anything since it could have an age of 10 years with 2 commits or 2 mio commits)
  • which OS / filesystem are you using ?
like image 36
Matthias Sohn Avatar answered Oct 16 '22 16:10

Matthias Sohn