Menu
When trying to clone a large repo (~700MB) over https, git fails with:
c:\git-projects>git clone https://git.mycompany.de/fs.git
Cloning into 'fs'...
Username for 'https://git.mycompany.de': mwlo
Password for 'https://[email protected]':
efrror: RPC failed; result=22, HTTP code = 500
atal: The remote end hung up unexpectedly
clone over ssh works:
c:\git-projects>git clone [email protected]:fs.git
Cloning into 'fs'...
remote: Counting objects: 144564, done.
remote: Compressing objects: 100% (30842/30842), done.
remote: Total 144564 (delta 95360), reused 143746 (delta 94542)
Receiving objects: 100% (144564/144564), 601.34 MiB | 1.33 MiB/s, done.
Resolving deltas: 100% (95360/95360), done.
Checking out files: 100% (4649/4649), done.
Cloning smaller repositories with https also works:
c:\git-projects>git clone https://git.mycompany.de/git-test.git
Cloning into 'git-test'...
remote: Counting objects: 135, done.
remote: Compressing objects: 100% (129/129), done.
remote: Total 135 (delta 68), reused 0 (delta 0)
Receiving objects: 100% (135/135), 18.77 KiB | 0 bytes/s, done.
Resolving deltas: 100% (68/68), done.
I have already tweaked some parameters but without success:
/etc/nginx/nginx.conf
worker_processes 2; # have two cpu's
keepalive_timeout 120;
client_max_body_size 3072m;
/home/git/gitlab/config/gitlab.yml
## Git settings
# CAUTION!
# Use the default values unless you really know what you are doing
git:
bin_path: /usr/bin/git
# Max size of a git object (e.g. a commit), in bytes
# This value can be increased if you have very large commits
max_size: 3221225472 # 3072.megabytes
# Git timeout to read a commit, in seconds
timeout: 120
We would like to use git clone https, as the visual studio tools for git still not have implemented ssh.
On the server are two processes, CPU load goes to 100% after a while, then the processes are terminated.
git pack-objects --revs --all --stdout --progress --delta-base-offset
Regards, Marco
System information
System: Debian 6.0.7
Current User: root
Using RVM: no
Ruby Version: 1.9.3p392
Gem Version: 1.8.23
Bundler Version:1.3.5
Rake Version: 10.0.4
GitLab information
Version: 5.3.0
Revision: 148eade
Directory: /home/git/gitlab
DB Adapter: mysql2
URL: https://git.mycompany.de
HTTP Clone URL: https://git.mycompany.de/some-project.git
SSH Clone URL: [email protected]:some-project.git
Using LDAP: yes
Using Omniauth: no
GitLab Shell
Version: 1.4.0
Repositories: /home/git/repositories/
Hooks: /home/git/gitlab-shell/hooks/
Git: /usr/bin/git
491
postBuffer is about: Maximum size in bytes of the buffer used by smart HTTP transports when POSTing data to the remote system. For requests larger than this buffer size, HTTP/1.1 and Transfer-Encoding: chunked is used to avoid creating a massive pack file locally.
As I told, git keeps track of each and every line change you made, it makes its history huge. But git uses a powerful compressing mechanism so that it will make all your codes to tiny tiny chunks. Also it stores the difference in between the files to reduce the size.
This is reported in issue 3079: https cloning requires a large amount of resources (CPU, but mainly memory) on the GitLab server, and currently (GitLab 5.x) large repos.
Even GitLab 6.0 comes with commits like 7ecebdd, mentioning timeouts when cloning large repo.
I haven't tested with GitLab 6, though (which should release tomorrow).
100
Consider implementing a chunking plugin for nginx, such as HttpChunkinModule.
I personally haven't deployed the above but have a client with similar issues.
In their case, the problem is on the client side, where we need to instruct devs to use the following tweak to their local git config:
git config http.postBuffer 524288000 #Set to 500MB
The above will simply allow larger git-related http requests on the client side (we've got plenty of memory on the server side).
34
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
