Catch-22 prevents streamed TCP WCF service securable by WIF; ruining my Christmas, mental health

Question

WCF has gotchas in a few areas with streaming (I'm looking at you, MTOM¹) due to a fundamental issue in how it fails to perform preauthentication the way most people would think that should work (it only affects subsequent requests for that channel, not the first request) Ok, so this is not exactly your issue but please follow along as I will get to yours at the end. Normally the HTTP challenge works like this:

1. client hits server anonymously
2. server says, sorry, 401, I need authentication
3. client hits server with authentication token
4. server accepts.

Now, if you ever try to enable MTOM streaming on an WCF endpoint on the server, it will not complain. But, when you configure it on the client proxy (as you should, they must match bindings) it will explode in a fiery death. The reason for this is that the above sequence of events that WCF is trying to prevent is this:

1. client streams 100MB file to server anonymously in a single POST
2. server says sorry, 401, I need authentication
3. client again streams 100MB file to server with an authentication header
4. server accepts.

Notice that you just sent 200MB to the server when you only needed to send 100MB. Well, this is the problem. The answer is to send the authentication on the first attempt but this is not possible in WCF without writing a custom behaviour. Anyway, I digress.

Your Problem

First up, let me tell you that what you're trying is impossible². Now, in order for you to stop spinning your wheels, let me tell you why:

It strikes me that you are now wandering in a similar class of problem. If you enable message level security, the client must load the entire stream of data into memory before it can actually close out the message with the usual hash function and xml signature required by ws-security. If it has to read the entire stream to sign the single message (which is not really a message, but it's a single continuous stream) then you can see the problem here. WCF will have to stream it once "locally" to compute the message security, then stream it again to send it to the server. This is clearly a silly thing, so WCF does not permit message level security for streaming data.

So, the simple answer here is that you should send the token either as a parameter to the initial web service, or as a SOAP header and use a custom behaviour to validate it. You cannot use WS-Security to do this. Frankly, this is not just a WCF issue - I cannot see how it could practically work for any other stacks.

Solving the MTOM Problem

This is just for an example how I solved my MTOM streaming issue for basic authentication, so perhaps you could take the guts of this and implement something similar for your issue. The crux of it is that in order to enable your custom message inspector, you have to disable all notion of security on the client proxy (it remains enabled on the server,) apart from transport level (SSL):

this._contentService.Endpoint.Behaviors.Add(
    new BasicAuthenticationBehavior(
        username: this.Settings.HttpUser,
        password: this.Settings.HttpPass));
var binding = (BasicHttpBinding)this._contentService.Endpoint.Binding;
binding.Security.Mode = BasicHttpSecurityMode.Transport; // SSL only            
binding.Security.Transport.ClientCredentialType = 
   HttpClientCredentialType.None; // Do not provide

Note that I have turned off transport security here because I will be providing that myself using a message inspector and custom behaviour:

internal class BasicAuthenticationBehavior : IEndpointBehavior
{
    private readonly string _username;
    private readonly string _password;

    public BasicAuthenticationBehavior(string username, string password)
    {
        this._username = username;
        this._password = password;
    }
    public void AddBindingParameters(ServiceEndpoint endpoint, 
        BindingParameterCollection bindingParameters) { }
    public void ApplyClientBehavior(ServiceEndpoint endpoint,
        ClientRuntime clientRuntime)
    {
        var inspector = new BasicAuthenticationInspector(
            this._username, this._password);
        clientRuntime.MessageInspectors.Add(inspector);
    }
    public void ApplyDispatchBehavior(ServiceEndpoint endpoint,
        EndpointDispatcher endpointDispatcher) { }
    public void Validate(ServiceEndpoint endpoint) { }
}

internal class BasicAuthenticationInspector : IClientMessageInspector
{
    private readonly string _username;
    private readonly string _password;

    public BasicAuthenticationInspector(string username, string password)
    {
        this._username = username;
        this._password = password;
    }

    public void AfterReceiveReply(ref Message reply,
        object correlationState) { }

    public object BeforeSendRequest(ref Message request,
        IClientChannel channel)
    {
        // we add the headers manually rather than using credentials 
        // due to proxying issues, and with the 101-continue http verb 
        var authInfo = Convert.ToBase64String(
            Encoding.Default.GetBytes(this._username + ":" + this._password));

        var messageProperty = new HttpRequestMessageProperty();
        messageProperty.Headers.Add("Authorization", "Basic " + authInfo);
        request.Properties[HttpRequestMessageProperty.Name] = messageProperty;

        return null;
    }
}

So, this example is for anyone who is suffering with the MTOM issue, but also as a skeleton for you to implement something similar to authenticate your token generated by the primary WIF-secured token service.

Hope this helps.

(1) Large Data and Streaming

(2) Message Security in WCF (see "disadvantages.")