Request to web service with Basic authorization via proxy with NTLM authorization not working

Question

I have a web service that requires basic authorisation and a user behind an internet proxy that requires NTLM authorisation. I also have a forms application that makes calls to the web service and also asks the user for the web service credentials (which are different from the NTLM credentials).

I got the app configuration working (WCF ServiceModel), it's using the default proxy credentials, the request is authenticating with the proxy, but after it authenticates with the web service it does not send the request body for some reason.

The process works if I test locally without the NTLM proxy. Sorry about the long examples, but I had to include them.

1st request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Host: www.myservice.com
Content-Length: 329
Expect: 100-continue
Accept-Encoding: gzip, deflate
Proxy-Connection: Keep-Alive

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body>[...]</s:Body></s:Envelope>

Receive:

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NTLM
Proxy-Authenticate: BASIC realm="corporaterealm"
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Set-Cookie: BCSI-CS-36204A5A7BBD24D9=2; Path=/
Connection: close
Content-Length: 1057
Proxy-Support: Session-Based-Authentication

[...]

2nd request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Accept-Encoding: gzip, deflate,gzip, deflate
Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB7IIoggACAAxAAAACQAJACgAAAAFASgKAAAAD1dTUkswNDg3MENPTUVUTkVU
Host: www.myservice.com
Content-Length: 0

Receive:

HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NTLM TlRMTVNTUAACAAAAEAAQADgAAAAFgomiysOwieqrhFEAAAAAAAAAALIAsgBIAAAABQLODgAAAA9DAE8ATQBFAFQATgBFAFQAAgAQAEMATwBNAEUAVABOAEUAVAABABoAVgBJAC0AUgBJAEMASwBEAEMALQAwADAAMQAEABwAYwBvAG0AZQB0AG4AZQB0AC4AbABvAGMAYQBsAAMAOABWAEkALQBSAEkAQwBLAEQAQwAtADAAMAAxAC4AYwBvAG0AZQB0AG4AZQB0AC4AbABvAGMAYQBsAAUAHABjAG8AbQBlAHQAbgBlAHQALgBsAG8AYwBhAGwAAAAAAA==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: Keep-Alive
Set-Cookie: BCSI-CS-36204A5A7BBD24D9=2; Path=/
Connection: Keep-Alive
Content-Length: 1074
Proxy-Support: Session-Based-Authentication

[...]

3rd request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Accept-Encoding: gzip, deflate,gzip, deflate,gzip, deflate
Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAHgAAAAYABgAkAAAABAAEABIAAAADgAOAFgAAAASABIAZgAAAAAAAACoAAAABYKIogUBKAoAAAAPQwBPAE0ARQBUAE4ARQBUAFAAYQByAHMAbwBuAEoAVwBTAFIASwAwADQAOAA3ADAAlap7g+mPRMEAAAAAAAAAAAAAAAAAAAAARLAhi5lf3nd+l9xENAcu2W6xf6iJbyM6
Host: www.myservice.com
Content-Length: 329
Expect: 100-continue

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body>[...]</s:Body></s:Envelope>

Receive:

HTTP/1.1 401 Unauthorized
Content-Type: text/html
Server: Microsoft-IIS/6.0
WWW-Authenticate: Basic realm="myrealm"
X-Powered-By: ASP.NET
Date: Tue, 31 May 2011 13:09:33 GMT
Cache-Control: proxy-revalidate
Content-Length: 1656
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Proxy-support: Session-based-authentication
Age: 0

[...]

4th request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Accept-Encoding: gzip, deflate,gzip, deflate,gzip, deflate,gzip, deflate
Authorization: Basic Y29nZW50YVxjb21ldC1kbTM6Kmh0JTg2NCU=
Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB7IIoggACAAxAAAACQAJACgAAAAFASgKAAAAD1dTUkswNDg3MENPTUVUTkVU
Host: www.myservice.com
Content-Length: 0

Receive:

HTTP/1.1 400 Bad Request
Date: Tue, 31 May 2011 13:09:33 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
Cache-Control: private, proxy-revalidate
Content-Length: 0
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Age: 0

I don't understand why it doesn't send the envelope in the 4th request. The two handshakes are done, so theoretically everything should be fine.

When I do this locally without the NTLM proxy, the envelope gets sent as expected:

1st request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Host: www.myservice.com
Content-Length: 329
Expect: 100-continue
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body>[...]</s:Body></s:Envelope>

Receive:

HTTP/1.1 401 Unauthorized
Content-Length: 1656
Content-Type: text/html
Server: Microsoft-IIS/6.0
WWW-Authenticate: Basic realm="myrealm"
X-Powered-By: ASP.NET
Date: Tue, 31 May 2011 13:31:46 GMT

[...]

2nd request:

Send:

POST http://www.myservice.com/service.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://tempuri.org/MyMethod"
Accept-Encoding: gzip, deflate,gzip, deflate
Authorization: Basic Y29nZW50YVxjb21ldC1kbTM6Kmh0JTg2NCU=
Host: www.myservice.com
Content-Length: 329
Expect: 100-continue

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body>[...]</s:Body></s:Envelope>

Receive:

HTTP/1.1 200 OK
Date: Tue, 31 May 2011 13:31:47 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
Set-Cookie: ASP.NET_SessionId=svv4i11awg05v1j5viz1impo; path=/; HttpOnly
Cache-Control: private, max-age=0
Content-Type: text/xml; charset=utf-8
Content-Length: 5127

[...]

Answer 1

I had to change the server security configuration to accept Digest authentication. For some reason that works while Basic doesn't.

Answer 2

I encountered a potentially similar issue and believe in my case this may be related to the establishment of the original NTLM connection to the proxy.

In my experience I had three requests going out in sequence each hour, with a retry configured.

Of these requests two would work, one would fail with a 400, and then when the first retried it worked as well.

There is a similiar issue with IE identified here: http://blogs.msdn.com/b/asiatech/archive/2012/01/30/400-bad-request-when-posting-webservice-or-wcf-request-from-ie.aspx

I ran some network traces and watched the behaviour and noticed that often the TCP Streams seemed to get shared, so that one of the requests would end up sending an NTLM authentication message (the zero content length) to the proxy after the other had already authenticated. In this situation the proxy just passed through the zero content message (as the stream had already authenticated).

What fixed it for me was moving to a WCF-Custom customBinding (using textMessageEncoding with a messageVersion of Soap11 to maintain my basicHttpBinding behaviour) and (not sure if required) setting keepAliveEnabled to false.

Unfortunately I doubt this is much help for your particular situation, but more info never hurts.