Avoiding "warning: There are too many unreachable loose objects" during git svn clone/fetch

Question

When running a git svn clone or git svn fetch against a large Subversion repository (100k+ commits), the fetch regularly stops with:

Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

gc --auto: command returned error: 255

In order to recover I have to follow the instructions, run a more aggressive prune and gc, remove the log-file and continue, only to have it happen again after another batch of say 10k commits are read.

How can this problem be avoided?

Answer 1

I think if you set the config option gc.pruneExpire to now, at least temporarily during the import, it will avoid that message. With that option set, git gc will immediately delete all unreachable objects, instead of only ones that are at least two weeks old (the default). Coupled with a sensible value for gc.auto this should prevent them from accumulating to the point that you get that message.

Answer 2

Self-answering.

git svn operations are amongst those which launch background gc --auto housekeeping operations. In this case, I believe that the ongoing progress of the git svn fetch may result in the number of unreachable/loose objects at some point in the gc operation exceeding the auth-threshold, resulting in this warning. Unfortunately it is fatal for the ongoing fetch.

My solution is to temporarily disable/suspect these gc operations, by deactivating gc auto as described in its manual page:

git config gc.auto 0

Once the git svn fetch operation completes, you can remove this config if desired, and run manual full gc, prune and repack operations to optimize the final repository.

Answer 3

Started seeing this warning on git pull:

$ git pull
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 22 (delta 17), reused 22 (delta 17), pack-reused 0
Unpacking objects: 100% (22/22), done.
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Already up-to-date.

Check out the warning file, not much there:

$ cat .git/gc.log
warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Read the help:

$ git help gc

Sounds like we are supposed to be doing some of this on a regular basis

Run the occasionally recommended aggressive option

$ git gc --aggressive
Counting objects: 41544, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (40544/40544), done.
Writing objects: 100% (41544/41544), done.
Total 41544 (delta 30536), reused 7801 (delta 0)
Removing duplicate objects: 100% (256/256), done.
Checking connectivity: 46959, done.

Remove the log warning:

$ rm .git/gc.log 

Smile

:)