Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Git gc using excessive memory, unable to complete

Final update and fix: The solution here turned out to be a combination of two things: using Windows Git rather than Cygwin Git as Graham Borland suggested, and the Git config settings pack.threads = 1 and gc.aggressiveWindow = 150.

I have a large local Git repository, a git svn clone of an SVN repository with about 40,000 commits. I'm trying to run git gc over this repository, and getting nowhere:

$ git gc --auto Auto packing the repository for optimum performance. You may also run "git gc" manually. See "git help gc" for more information. Counting objects: 25966, done. Compressing objects: 100% (25249/25249), done. fatal: Out of memory, malloc failed (tried to allocate 426523986 bytes) error: failed to run repack 

I'm running Git 1.7.5.1 inside Cygwin on a 64-bit dual-core Win7 machine with 4GB RAM. The .git directory is currently a little over 6.1GB.

I've tried running git gc --aggressive, to see if the more complete system is able to fix it, but no luck: I get a similar message to the above, with the same size attempted malloc, but a considerably higher object count (508,485 counted, 493,506 compressed).

I've also tried—as suggested by Google—assorted twiddles to the [pack] part of my .gitconfig file; the most complete being from another StackOverflow question. My .gitconfig now has the following relevant lines, but setting these appears to have made no difference:

[pack]         windowMemory = 16m         threads = 1         window = 1         depth = 1         deltaCacheSize = 1 

Any suggestions on how I can get git to gc my repository?

Edit: Mark Longair suggested some more .gitconfig file changes. Which I made, new lines below. But the changes made no difference whatsoever.

[core]         packedGitWindowSize = 1m         packedGitLimit = 256m [pack]         packSizeLimit = 128m 

Edit 2: Michael Krelin suggested increasing the swap/page file size (WinXP instructions here, and it's similar for Win7). I tried that, but it made no difference, and indeed I only increased the maximum size available, and it looks as if Windows never tried to increase the size of the page file it was using.

I'm now looking at whether this was caused by a memory limit within or imposed upon Cygwin. To check "imposed upon", I'm trying running Cygwin with administrator privileges. To check "within" (which looks more likely), I'm having a play with Cygwin's maximum memory settings.

Edit 3: Much though I may prefer using Cygwin, it turns out the Windows Git client deals with the memory issue just fine. Seems I'll be falling back to that every so often when my repository needs a tidy.

like image 730
me_and Avatar asked Nov 21 '11 15:11

me_and


People also ask

When should you not run git gc?

See gc. auto below for how to disable this behavior. Running git gc manually should only be needed when adding objects to a repository without regularly running such porcelain commands, to do a one-off repository optimization, or e.g. to clean up a suboptimal mass-import.

What is git gc -- prune?

git gc is a parent command and git prune is a child. git gc will internally trigger git prune . git prune is used to remove Git objects that have been deemed inaccessible by the git gc configuration.

What does git gc prune now do?

git gc --prune=now removes the commits themselves. Attention: Only using git gc --prune=now will not work since those commits are still referenced in the reflog. Therefore, clearing the reflog is mandatory. Also note that if you use rerere it has additional references not cleared by these commands.

How would you force git to trigger garbage collection?

The easiest option would be to use a scheduled task in windows or a cron job in Unix to run git gc periodically. This way you don't even need to think about it.


2 Answers

I had the same problem, tried the solutions mentioned so far without success. But my problems with git gc began after I added big image files to repo, so I created .gitattributes file and turned off delta compression for those big files:

*.tga -delta *.psd -delta 

It worked.

like image 83
andriej Avatar answered Oct 02 '22 17:10

andriej


You might have more luck running a native Windows client such as msysGit, rather than trying to do it inside Cygwin.

like image 33
Graham Borland Avatar answered Oct 02 '22 16:10

Graham Borland