Menu
I am running git on a shared host (uberspace.de) and hit the memory limit when doing git push or git gc. Memory consumption reported by them is around 1500 MB.
So I followed the advice in
https://stackoverflow.com/a/8761464/288568
It seemed to work once, but after lots of commits it does not help anymore and the memory consumption is again around 1500MB and the git pack process gets killed.
I commit a 120 MB SQL dump with very fews changes (around 5 lines each time) - so the packed repository is around 150 MB only, but unpacked around 3 GB (and I hit bitbucket's limit).
Which pmap I see it is opening a lot of pack objects - is it that which adds up to the high memory? How can this be limited?
Finally I get "pack-objects died of signal 15" and a mail from the hoster, that my process was killed.
sh-4.1$ git config --list|grep pack
pack.windowmemory=25m
pack.packsizelimit=25m
pack.threads=1
pack.deltecachesize=25m
sh-4.1$ git gc
Counting objects: 2586, done.
^Zmpressing objects: 15% (163/1085)
[1]+ Stopped(SIGTSTP) git gc
sh-4.1$ bg
[1] git gc &
sh-4.1$ pmap 14190
14190: git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --unpack-unreachable=2.weeks.ago --local --delta-base-offset .git/objects/pack/.tmp-14187-pack
0000000000400000 1848K r-x-- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007ce000 36K rw--- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007d7000 280K rw--- [ anon ]
0000000000da1000 1680K rw--- [ anon ]
00007f735dc5d000 161648K rw--- [ anon ]
00007f7377555000 149580K rw--- [ anon ]
00007f73915ee000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.pack
00007f73990b6000 125592K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.pack
00007f73a0b5c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.pack
00007f73a8624000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.pack
00007f73b00ec000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.pack
00007f73b7bb4000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.pack
00007f73bf67c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.pack
00007f73c7144000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.pack
00007f73cec0c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.pack
00007f73d66d4000 125348K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.pack
00007f73de13d000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.pack
00007f73e5c05000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.pack
00007f73ed6cb000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.pack
00007f73f5191000 120872K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.pack
00007f73fc79b000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.pack
00007f7404263000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.pack
00007f740bd29000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.pack
00007f74137ef000 120868K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.pack
00007f741adf8000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.pack
00007f74228c0000 484K rw--- [ anon ]
00007f7422939000 96840K r---- /usr/lib/locale/locale-archive
00007f74287cb000 8K r-x-- /lib64/libdl-2.12.so
00007f74287cd000 2048K ----- /lib64/libdl-2.12.so
00007f74289cd000 4K r---- /lib64/libdl-2.12.so
00007f74289ce000 4K rw--- /lib64/libdl-2.12.so
00007f74289cf000 1576K r-x-- /lib64/libc-2.12.so
00007f7428b59000 2048K ----- /lib64/libc-2.12.so
00007f7428d59000 16K r---- /lib64/libc-2.12.so
00007f7428d5d000 8K rw--- /lib64/libc-2.12.so
00007f7428d5f000 16K rw--- [ anon ]
00007f7428d63000 92K r-x-- /lib64/libpthread-2.12.so
00007f7428d7a000 2048K ----- /lib64/libpthread-2.12.so
00007f7428f7a000 4K r---- /lib64/libpthread-2.12.so
00007f7428f7b000 4K rw--- /lib64/libpthread-2.12.so
00007f7428f7c000 16K rw--- [ anon ]
00007f7428f80000 28K r-x-- /lib64/librt-2.12.so
00007f7428f87000 2044K ----- /lib64/librt-2.12.so
00007f7429186000 4K r---- /lib64/librt-2.12.so
00007f7429187000 4K rw--- /lib64/librt-2.12.so
00007f7429188000 1768K r-x-- /usr/lib64/libcrypto.so.1.0.1e
00007f7429342000 2044K ----- /usr/lib64/libcrypto.so.1.0.1e
00007f7429541000 108K r---- /usr/lib64/libcrypto.so.1.0.1e
00007f742955c000 48K rw--- /usr/lib64/libcrypto.so.1.0.1e
00007f7429568000 16K rw--- [ anon ]
00007f742956c000 84K r-x-- /lib64/libz.so.1.2.3
00007f7429581000 2044K ----- /lib64/libz.so.1.2.3
00007f7429780000 4K r---- /lib64/libz.so.1.2.3
00007f7429781000 4K rw--- /lib64/libz.so.1.2.3
00007f7429782000 128K r-x-- /lib64/ld-2.12.so
00007f74297a6000 464K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.pack
00007f742981a000 52K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.pack
00007f7429827000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.pack
00007f7429828000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.pack
00007f7429829000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.pack
00007f742982a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.pack
00007f742982b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.pack
00007f742982c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.pack
00007f742982d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.pack
00007f742982e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.pack
00007f742982f000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.pack
00007f7429831000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.pack
00007f7429836000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.pack
00007f7429838000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.pack
00007f742983a000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.pack
00007f742983c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.pack
00007f742983d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.pack
00007f742983e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.pack
00007f742983f000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.pack
00007f7429875000 4K rw--- [ anon ]
00007f7429876000 400K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.pack
00007f74298da000 512K rw--- [ anon ]
00007f742995a000 44K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.idx
00007f7429965000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.idx
00007f7429966000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.idx
00007f7429967000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.idx
00007f7429968000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.idx
00007f7429969000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.idx
00007f742996a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.idx
00007f742996b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.idx
00007f742996c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.idx
00007f742996d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.idx
00007f742996e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.idx
00007f742996f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.idx
00007f7429970000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.idx
00007f7429971000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.idx
00007f7429972000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.idx
00007f7429973000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.idx
00007f7429974000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.idx
00007f7429975000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.idx
00007f7429976000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.idx
00007f7429977000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.idx
00007f7429978000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.idx
00007f7429979000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.idx
00007f742997a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.idx
00007f742997b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.idx
00007f742997c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.idx
00007f742997d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.idx
00007f742997e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.idx
00007f742997f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.idx
00007f7429980000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.idx
00007f7429981000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.idx
00007f7429982000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.idx
00007f7429983000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.idx
00007f7429984000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.idx
00007f7429985000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.idx
00007f7429986000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.idx
00007f7429987000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.idx
00007f7429988000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.idx
00007f7429989000 24K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.idx
00007f742998f000 20K rw--- [ anon ]
00007f7429998000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.idx
00007f7429999000 28K r--s- /usr/lib64/gconv/gconv-modules.cache
00007f74299a0000 4K rw--- [ anon ]
00007f74299a1000 4K r---- /lib64/ld-2.12.so
00007f74299a2000 4K rw--- /lib64/ld-2.12.so
00007f74299a3000 4K rw--- [ anon ]
00007ffe76939000 84K rw--- [ stack ]
00007ffe7698b000 4K r-x-- [ anon ]
ffffffffff600000 4K r-x-- [ anon ]
total 2809104K
sh-4.1$ error: pack-objects died of signal 15
error: failed to run repack
829
Running git gc manually should only be needed when adding objects to a repository without regularly running such porcelain commands, to do a one-off repository optimization, or e.g. to clean up a suboptimal mass-import.
git gc is a parent command and git prune is a child. git gc will internally trigger git prune . git prune is used to remove Git objects that have been deemed inaccessible by the git gc configuration.
DESCRIPTION. This command is used to combine all objects that do not currently reside in a "pack", into a pack. It can also be used to re-organize existing packs into a single, more efficient pack.
I have some big repository which I have to compress and I did not notice any memrory related options help. If git-pack-objects has to repack some objects, it will have to map roughly same big memory. The only way to manage it is to limit number of objects it has to repack.
In your case object already come compressed in small pack files (one for each push). So the git gc has to repack those small pack files into big ones. You could try to create empty pack-<HASH>.keep files next to the big pack files which were creates by last succesful repack. Then maybe next git gc will not try to deltify those objects again.
132
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
