Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

git push Out of memory, malloc failed

Tags:

git

github

I am trying to push my changes remotely to GitHub, every so often git fails due to

C:\dev\projects>git push -v Pushing to https://[email protected]/mycompany/My-Project.git Password for 'github.com': fatal: Out of memory, malloc failed (tried to allocate 524288000 bytes) fatal: write error: Invalid argument 

This is very, very aggravating. I have run the following commands, upgraded git (which wiped out my settings and caused lots of pain, but I digress)

git gc --auto --prune=today --aggressive git repack 

I have even bumped the value of

http.postbuffer 

but eventually it will fail again.

This is a typical Rails 3.1 application, total project size on disk is 9.69 MB.

like image 485
Jared Knipp Avatar asked Jan 13 '12 18:01

Jared Knipp


People also ask

What happens when a Git process exceeds its maximum memory limit?

If your git process exceeds the limits, the kernel will kill the process despite the max memory reported being available on your system. Note: be careful with these settings. You probably don't want to use the settings in that example as i pulled them from a server in our environment.

Why can't GitLab run on my Virtual Machine?

This happened to me because the memory of the server where GitLab is running went full. I increased the memory of that virtual machine and the problem was solved. Show activity on this post. Check if swap is enabled in your box. If not, you could create one. I tried this guide for ubuntu that works for Debian also.

Why can't I run Git on 32 bit?

You may just have one or more really big files. Check for files greater than 50MB, which is the normal github limit: Show activity on this post. In my case Git.exe just needed a little bit more then 32bit process could get.

How can I prevent the kernel from allocation max memory to processes?

With some kernel distributions, there are setting which prevent the kernel from allocation max memory to a single process: Set Kernel Parameters Modify the "/etc/sysctl.conf" file to include the lines appropriate to your operating system.


1 Answers

My advice is to try several git parameters related with pack:

[pack]    threads = 1    deltaCacheSize = 128m    windowMemory = 50m 

What it got better results for me was setting git config pack.threads 1 and git config pack.windowMemory 50m (default is 10m).

Still, my host didn't have enough RAM memory (2GB) and kept failing. I hard copied the repo and moved it to another machine with more RAM (8GB). It got better but still failed.

Finally, I downloaded the latest version of git (https://github.com/git/git), compile it and install it. That fixed the problem just by running git repack -adf with the same parameters. After that I run git gc --aggressive --prune=now

Once I got the repo fixed in my local machine I pushed it to master, overwriting the remote repo, git push -f origin master.

To prevent similar errors in the future try not to add unnecessary large files to the repo (in my case I got a SQL dump of 3.5GB :)) and disable delta compression for large files (such as images, PDFs, videos). Add the following lines to .gitattributes:

*.pdf -delta *.jpg -delta 
like image 53
Diego Pino Avatar answered Sep 20 '22 16:09

Diego Pino