git clone fails with "fatal: Out of memory, malloc failed" error

Question

When I perform a git clone from a bare repository of my project, on a local server, I get following error message:

fatal: Out of memory, malloc failed (tried to allocate 2251896833 bytes) warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry the checkout with 'git checkout -f HEAD'

I tried updating my ~/.gitconfig file as stated at answer under this question, closed the git bash, restarted and retried without any results.

I ended up trying following configuration but still same results:

$ cat .gitconfig
[core]
        packedGitLimit = 1024m
        packedGitWindowSize = 1024m
[pack]
        deltaCacheSize = 1024m
        packSizeLimit = 1024m
        windowMemory = 1024m
[http]
        postBuffer = 157286400

I even tried with git gc on another machine, but not sure how to get the bare repository to garbage collect as well.

I am using git version 2.14.2.windows.1 under a 32 bits machine with Windows 7 with 4GB of RAM.

How to resolve this fatal error on git clone?

Answer 1

If you are on Linux and have root access, the simple workaround of adding virtual memory could be the easiest solution. On most Linux systems you can do the following (as root) to add 4GiB of virtual memory, and then try your git clone again after:

dd if=/dev/zero of=/var/swap.img bs=1M count=4096
chmod 0600 /var/swap.img
mkswap /var/swap.img
swapon /var/swap.img

This ended up being the simplest workable solution for me when I ran into this issue.

Answer 2

Using the fact that a Git repo is self-contained and can just be copied, we only need some way to make a copy and convert from bare to non-bare for our first few attempts.

Cloning is still the best form of initial copying, if that option is possible (see work-arounds 1 and 2). If not, we can just straight copy from the server, if access is available, and convert it manually (see #3). Failing that, perhaps copying/cloning a smaller piece of the repo will work?

Work-around 1 - clone onto USB from server

Get on the server (if you have access), clone to a usb drive folder (as a non-bare repo), stick USB into target machine, and just move/copy it to the final location you want the repo. Git command should be:

git clone /path/to/bare/repo /local/repo/folder

I think this will not copy any orphan commits (those that will eventually be garbage-collected).

Work-around 2 - Clone to another PC, then copy

Clone repo onto another PC, copy/move folder somehow to target PC (USB, network connection, etc.). Similar to #1

Work-around 3 - Copy from Server, convert from bare

This assumes access to the server, perhaps through a network share rather than directly (otherwise use #1). Copy the bare repo locally, then do something like this to convert it to a non-bare repo.

Work-around 4a - Clone only one branch at a time

It appears you can clone only a single branch at a time, which may fit under the limits causing your memory issue:

git clone [url/folder] -b master --single-branch [local repo folder]

Note you could create new branches on the server that are a bit back in the history, to pull less at once, but based on the answer in the above link, you may have to jump through some hoops to remove the "single-branchedness" of your new repo.

Work-around 4b - Shallow Repo

Use --depth <depth> and/or other shallow tags to make a local repo with a limited commit history depth. From git clone --help man page:

--depth Create a shallow clone with a history truncated to the specified number of commits. Implies --single-branch unless --no-single-branch is given to fetch the histories near the tips of all branches. If you want to clone submodules shallowly, also pass --shallow-submodules.

--shallow-since= Create a shallow clone with a history after the specified time.

--shallow-exclude= Create a shallow clone with a history, excluding commits reachable from a specified remote branch or tag. This option can be specified multiple times.

--[no-]single-branch Clone only the history leading to the tip of a single branch, either specified by the --branch option or the primary branch remote’s HEAD points at. Further fetches into the resulting repository will only update the remote-tracking branch for the branch this option was used for the initial cloning. If the HEAD at the remote did not point at any branch when --single-branch clone was made, no remote-tracking branch is created.

Work-around 4c - Clone/Checkout only some files at a time

See this answer and it's link for instructions on how to pull down just part of a repo at a time. Ugly, but could work to clone part at a time.