git push hangs over (dumb) https, after HTTP/1.1 100 Continue

Question

I'm trying to push some changes, but git push hangs. When I run git push, I see no output, and nothing seems to be happening. There's no activity in top, and no sign of anything happening.

I do not control the git hosting service. I'm using a HTTPS URL. I believe the hosting service is using dumb HTTPS, not git's "smart-HTTP" protocol. On the client side, I use Mac OS X, and I've got git 1.8.1.1 installed via Homebrew (but using the version of git included in Xcode's command-line tools doesn't seem to make a difference). Logging out and logging back in doesn't seem to help. I can pull and push to this hosting service/repository from a different Linux box.

Below is some debugging output that shows git push hanging after the client issues a PROPFIND request, gets a HTTP/1.1 100 Continue response from the server, and then nothing happens: it's just stuck.

How do I get this working? Are there any troubleshooting steps that I can try?

$ GIT_CURL_VERBOSE=1 git push -v
Pushing to https://secure2.svnrepository.com/redacted/redacted/
* About to connect() to secure2.svnrepository.com port 443 (#0)
*   Trying 67.228.18.88...
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*    subject: C=US; OU=Domain Control Validated; CN=secure2.svnrepository.com
*    start date: 2012-01-09 16:16:59 GMT
*    expire date: 2015-02-09 02:52:45 GMT
*    subjectAltName: secure2.svnrepository.com matched
*    issuer: O=AlphaSSL; CN=AlphaSSL CA - G2
*    SSL certificate verify ok.
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 401 Authorization Required
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< WWW-Authenticate: Basic realm="redacted"
< Content-Length: 493
< Content-Type: text/html; charset=iso-8859-1
< 
* Ignoring the response-body
* Connection #0 to host secure2.svnrepository.com left intact
* Issue another request to this URL: 'https://secure2.svnrepository.com/redacted/redacted/info/refs?service=git-receive-pack'
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< Last-Modified: Wed, 23 Jan 2013 03:00:40 GMT
< ETag: "143802e-3b-e6374600"
< Accept-Ranges: bytes
< Content-Length: 59
< Content-Type: text/plain; charset=UTF-8
< 
* Connection #0 to host (nil) left intact
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> GET /redacted/redacted/HEAD HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 23 Jan 2013 03:29:36 GMT
< Server: Apache/2.2.3 (Red Hat)
< Last-Modified: Wed, 16 Jan 2013 21:05:31 GMT
< ETag: "d1802c-17-3d0d7cc0"
< Accept-Ranges: bytes
< Content-Length: 23
< Content-Type: text/plain; charset=UTF-8
< 
* Connection #0 to host (nil) left intact
* About to connect() to secure2.svnrepository.com port 443 (#0)
*   Trying 67.228.18.88...
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* Connected to secure2.svnrepository.com (67.228.18.88) port 443 (#0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*    subject: C=US; OU=Domain Control Validated; CN=secure2.svnrepository.com
*    start date: 2012-01-09 16:16:59 GMT
*    expire date: 2015-02-09 02:52:45 GMT
*    subjectAltName: secure2.svnrepository.com matched
*    issuer: O=AlphaSSL; CN=AlphaSSL CA - G2
*    SSL certificate verify ok.
> PROPFIND /redacted/redacted/ HTTP/1.1
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 181
Expect: 100-continue

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 401 Authorization Required
< Date: Wed, 23 Jan 2013 03:29:37 GMT
< Server: Apache/2.2.3 (Red Hat)
< WWW-Authenticate: Basic realm="redacted"
< Content-Length: 493
< Content-Type: text/html; charset=iso-8859-1
* the ioctl callback returned 0
< 
* Ignoring the response-body
* Connection #0 to host secure2.svnrepository.com left intact
* Issue another request to this URL: 'https://secure2.svnrepository.com/redacted/redacted/'
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (67.228.18.88) port 443 (#0)
* Server auth using Basic with user 'redacted'
> PROPFIND /redacted/redacted/ HTTP/1.1
Authorization: Basic redacted=
User-Agent: git/1.8.1.1
Host: secure2.svnrepository.com
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 181
Expect: 100-continue

< HTTP/1.1 100 Continue

I don't have strace on my Mac OS X machine, and I can't figure out how to use dtruss to see what system calls it is hanging on because dtruss requires me to be root, and then git push will work differently.

Update: I've reproduced this on a Linux machine with git 1.8.1.4 and with strace. Running strace shows something like the following before it hangs:

sendto(4, <redacted>..., 314, 0, NULL, 0) = 314
recvfrom(4, "\27\3\1\0000", 5, 0, NULL, NULL) = 5
recvfrom(4, "E\202\271\21\236p\200\346\374\3641\355\t\275\rLi\202T)\326\271l/\351\f\357\2769Jb\22"..., 48, 0, NULL, NULL) = 48
select(5, [4], [4], [], {0, 729000}) = 1 (out [4], left {0, 728997})
poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}, {fd=4, events=POLLOUT|POLLWRNORM}], 2, 0) = 1 ([{fd=4, revents=POLLOUT|POLLWRNORM}])
select(5, [4], [], [], {0, 729000}) = 0 (Timeout)
poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
...last 2 lines repeat infinitely...

So it seems to be hanging expecting to receive something from the server.

Also, I tried a similar trace with GIT_CURL_VERBOSE=1 git push -v on an older Linux box running git 1.7.4.4, and it starts with the same prefix and then continues on from there. On the broken machine with the newer git:

$ grep '^> [A-Z]' git-1.8.1.1-trace.stderr
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
> GET /redacted/redacted/info/refs?service=git-receive-pack HTTP/1.1
> GET /redacted/redacted/HEAD HTTP/1.1
> PROPFIND /redacted/redacted/ HTTP/1.1
> PROPFIND /redacted/redacted/ HTTP/1.1

On the older machine with the older git, where it's all working:

$ grep '^> [A-Z]' git-1.7.4.4-trace.stderr
> GET /g_wagner/c79-s13/info/refs?service=git-receive-pack HTTP/1.1
> GET /g_wagner/c79-s13/info/refs?service=git-receive-pack HTTP/1.1
> GET /g_wagner/c79-s13/HEAD HTTP/1.1
> PROPFIND /g_wagner/c79-s13/ HTTP/1.1
> PROPFIND /g_wagner/c79-s13/ HTTP/1.1
> HEAD /g_wagner/c79-s13/info/refs HTTP/1.1
> HEAD /g_wagner/c79-s13/objects/info/packs HTTP/1.1
> MKCOL /g_wagner/c79-s13/info/ HTTP/1.1
> LOCK /g_wagner/c79-s13/info/refs HTTP/1.1
> GET /g_wagner/c79-s13/objects/info/packs HTTP/1.1
...
> UNLOCK /g_wagner/c79-s13/info/refs HTTP/1.1

Looking at the full trace on both machines, I can't see any difference in what is sent in the problematic PROPFIND request (the 2nd PROPFIND): both requests appear to be identical, except for the User-Agent: header.

Answer 1

I've got problem with same symptoms (process hangs at HTTP/1.1 100 Continue).

I've found a commit that resolves the issue for me: http-push.c: make CURLOPT_IOCTLDATA a usable pointer, it's available in GIT release v2.0.3.

Apparently user is not being authenticated correctly on server even it does seems so from the communication.

Problem is weird, because for you it shows up on one machine but not on another with same GIT versions installed, for me GIT just stopped working within period of one month and I can't really say what was the cause.

Alternatively if it's possible, use GIT service provider that allows other protocols: SSH or HTTPS, these are much better debugged.

---

For windows users, here's my take on git-http-push 2.1.0-rc2 build (based on msysGit project instructions)

To backport to your msysGit installation, unzip it and in msysGit installation folder copy into/ overwrite libexec\git-core\git-http-push.exe.

The issue has been reported here.

Update

Latest msysGit (Git-1.9.4-preview20140815) contains backported bugfix.