Minimal WebAPI2 + OAuth with JWT implementation: 401 always returned

Question

I'm trying to implement a simple WebAPI2 project securing my API functions with JWT tokens. As I'm fairly new to this, I followed mainly these tutorials as a guidance: http://bitoftech.net/2015/01/21/asp-net-identity-2-with-asp-net-web-api-2-accounts-management/ with its code at https://github.com/tjoudeh/AspNetIdentity.WebApi, and http://odetocode.com/blogs/scott/archive/2015/01/15/using-json-web-tokens-with-katana-and-webapi.aspx.

Edit #1: see at the bottom: resolved issue for client_id = null.

Of course, several details change in my implementation, which should be minimal as I'm learning, and my current requirements are not that complicated: I do not use 3rd party JWT or security libraries (like Thinktecture or Jamie Kurtz JwtAuthForWebAPI), but just stick to MS JWT component, nor I need 2FA or external login, as this will be a corporate API consumed by a client app with users registered by administrators.

I managed to implement an API which returns the JWT token, yet when I make a request with it to any protected API (of course, unprotected API do work) the request is constantly refused with a 401-Unauthorized error. A sample request/response at the api/token endpoint looks like this:

request:

POST http://localhost:50505/token HTTP/1.1
Host: localhost:50505
Connection: keep-alive
Content-Length: 56
Accept: application/json, text/plain, */*
Origin: http://localhost:50088
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Referer: http://localhost:50088/dist/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,it;q=0.6

grant_type=password&username=Zeus&password=ThePasswordHere

response:

HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 343
Content-Type: application/json;charset=UTF-8
Expires: -1
Server: Microsoft-IIS/8.0
Access-Control-Allow-Origin: http://localhost:50088
Access-Control-Allow-Credentials: true
X-SourceFiles: =?UTF-8?B?QzpcUHJvamVjdHNcNDViXEV4b1xJYW5pdG9yXElhbml0b3IuV2ViQXBpXHRva2Vu?=
X-Powered-By: ASP.NET
Date: Mon, 13 Apr 2015 22:16:50 GMT

{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1bmlxdWVfbmFtZSI6IlpldXMiLCJyb2xlIjoiYWRtaW5pc3RyYXRvciIsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6NTA1MDUiLCJhdWQiOiIwZDQ1ZTljZWM4MzY0NmI2YTE3Mzg0N2VjOWM5NmY3ZiIsImV4cCI6MTQyOTA0OTgwOSwibmJmIjoxNDI4OTYzNDA5fQ.-GFvtEfNI7Y8tf6Ln1MpxJc4yORuf2gzksGjRbSMEnU","token_type":"bearer","expires_in":86399}

If I inspect the token (at http://jwt.io/) I get this JSON for the JWT payload:

{
  "unique_name": "Zeus",
  "role": "administrator",
  "iss": "http://localhost:50505",
  "aud": "0d45e9cec83646b6a173847ec9c96f7f",
  "exp": 1429049809,
  "nbf": 1428963409
}

Yet, any request with a similar token (here to the 'canonical' API ValuesController used in sample templates), like this (I omit the preflight OPTIONS CORS request, which is correctly issued):

GET http://localhost:50505/api/values HTTP/1.1
Host: localhost:50505
Connection: keep-alive
Accept: application/json, text/plain, */*
Origin: http://localhost:50088
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1bmlxdWVfbmFtZSI6IlpldXMiLCJyb2xlIjoiYWRtaW5pc3RyYXRvciIsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6NTA1MDUiLCJhdWQiOiIwZDQ1ZTljZWM4MzY0NmI2YTE3Mzg0N2VjOWM5NmY3ZiIsImV4cCI6MTQyOTA4MzAzOCwibmJmIjoxNDI4OTk2NjM4fQ.i5ik6ggSzoV2Nz-1_Od5fZVKxBpgOmEJcQN00YsG_DU
Referer: http://localhost:50088/dist/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,it;q=0.6

fails with 401:

HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
Access-Control-Allow-Origin: http://localhost:50088
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?QzpcUHJvamVjdHNcNDViXEV4b1xJYW5pdG9yXElhbml0b3IuV2ViQXBpXGFwaVx2YWx1ZXM=?=
X-Powered-By: ASP.NET
Date: Tue, 14 Apr 2015 07:30:53 GMT
Content-Length: 61

{"message":"Authorization has been denied for this request."}

Given that this is a rather complex topic for security newbies like me, in what follows I'm describing the essential aspects of my solution, so that experts can hopefully point me to a solution, and newcomers can find some up-to-date guidance.

Data Layer

I created my data layer in a separate DLL project using EntityFramework and including my IdentityDbContext-derived data context and its entities (User and Audience). The User entity just adds a couple of string properties for first and last name. The Audience entity is used to provide infrastructure for multiple audiences; it has an ID (a GUID represented by a string property), a name (used only to provide human-friendly labels) and a base-64 encoded shared key.

Using migrations I created the database and seeded it with an administrator user and a test audience.

Web API

1. Start Template

I created an empty WebApp project, including WebAPI libraries, and no user authentication, as the default authentication template is too bloated for my limited purposes and has too moving parts for learners. I added the required NuGet packages manually, which in the end are:

EntityFramework
Microsoft.AspNet.Identity.EntityFramework
Microsoft.AspNet.Cors
Microsoft.AspNet.Identity.Owin
Microsoft.AspNet.WebApi
Microsoft.AspNet.WebApi.Client
Microsoft.AspNet.WebApi.Core
Microsoft.AspNet.WebApi.Owin
Microsoft.AspNet.WebApi.WebHost
Microsoft.Owin
Microsoft.Owin.Cors
Microsoft.Owin.Host.SystemWeb
Microsoft.Owin.Security
Microsoft.Owin.Security.Cookies
Microsoft.Owin.Security.Jwt
Microsoft.Owin.Security.OAuth
Newtonsoft.Json
Owin
System.IdentityModel.Tokens.Jwt

2. Infrastructure

As for infrastructure, I created a fairly standard ApplicationUserManager (the provider at the bottom is not required in my case, but I added this as a reminder for other projects):

public class ApplicationUserManager : UserManager<User>
{
    public ApplicationUserManager(IUserStore<User> store)
        : base(store)
    {
    }

    public static ApplicationUserManager Create(
        IdentityFactoryOptions<ApplicationUserManager> options,
        IOwinContext context)
    {
        var manager = new ApplicationUserManager
            (new UserStore<User>(context.Get<IanitorContext>()));

        manager.UserValidator = new UserValidator<User>(manager)
        {
            AllowOnlyAlphanumericUserNames = false,
            RequireUniqueEmail = true
        };

        manager.PasswordValidator = new PasswordValidator
        {
            RequiredLength = 6,
            RequireNonLetterOrDigit = true,
            RequireDigit = true,
            RequireLowercase = true,
            RequireUppercase = true,
        };

        manager.UserLockoutEnabledByDefault = true;
        manager.DefaultAccountLockoutTimeSpan = TimeSpan.FromMinutes(5);
        manager.MaxFailedAccessAttemptsBeforeLockout = 5;

        var dataProtectionProvider = options.DataProtectionProvider;
        if (dataProtectionProvider != null)
        {
            // for email confirmation and reset password life time
            manager.UserTokenProvider =
                new DataProtectorTokenProvider<User>(dataProtectionProvider.Create("ASP.NET Identity"))
                {
                    TokenLifespan = TimeSpan.FromHours(6)
                };
        }
        return manager;
    }

3. Providers

Also, I need an OAuth token provider: AFAIK, the core method here is GrantResourceOwnerCredentials, which validates the received username and password against my store; when this succeeds, I create a new ClaimsIdentity and populate it with the authenticated user's claims I want to publish in my token; I then use this plus some metadata properties (here the audience ID) to create an AuthenticationTicket, and pass this to context.Validated method:

public class ApplicationOAuthProvider : OAuthAuthorizationServerProvider
{
    public override Task ValidateClientAuthentication
        (OAuthValidateClientAuthenticationContext context)
    {
        context.Validated();
        return Task.FromResult<object>(null);
    }

    public override async Task GrantResourceOwnerCredentials
        (OAuthGrantResourceOwnerCredentialsContext context)
    {
        // http://www.codeproject.com/Articles/742532/Using-Web-API-Individual-User-Account-plus-CORS-En
        if (!context.OwinContext.Response.Headers.ContainsKey("Access-Control-Allow-Origin"))
            context.OwinContext.Response.Headers.Add("Access-Control-Allow-Origin", new []{"*"});

        if ((String.IsNullOrWhiteSpace(context.UserName)) || 
            (String.IsNullOrWhiteSpace(context.Password)))
        {
            context.Rejected();
            return;
        }

        ApplicationUserManager manager = 
            context.OwinContext.GetUserManager<ApplicationUserManager>();
        User user = await manager.FindAsync(context.UserName, context.Password);
        if (user == null)
        {
            context.Rejected();
            return;
        }

        // add selected claims for building the token
        ClaimsIdentity identity = new ClaimsIdentity(context.Options.AuthenticationType);
        identity.AddClaim(new Claim(ClaimTypes.Name, user.UserName));
        foreach (var role in manager.GetRoles(user.Id))
            identity.AddClaim(new Claim(ClaimTypes.Role, role));

        // add audience
        // TODO: why context.ClientId is null? I would expect an audience ID
        AuthenticationProperties props =
            new AuthenticationProperties(new Dictionary<string, string>
            {
                {
                    ApplicationJwtFormat.AUDIENCE_PROPKEY,
                    context.ClientId ?? ConfigurationManager.AppSettings["audienceId"]
                }
            });

        DateTime now = DateTime.UtcNow;
        props.IssuedUtc = now;
        props.ExpiresUtc = now.AddMinutes(context.Options.AccessTokenExpireTimeSpan.TotalMinutes);

        AuthenticationTicket ticket = new AuthenticationTicket(identity, props);
        context.Validated(ticket);
    }
}

A first issue here is that when debugging I can see that the received context client ID is null. I'm not sure about where it should be set. This is why here I'm falling back to a default audience ID (enough for my testing purposes, going to eat the elephant one bite at a time).

Another key component here is the JWT token formatter, which is in charge of building the JWT token from the ticket. In my implementation I inject in its constructor a function to retrieve my EF data context, as the formatter requires it in order to get the audience's secret key. The required audience ID comes from metadata properties set by the above code, and is used to lookup the store for an Audience entity. If not found I fall back to a default audience defined in my Web.config (this is the test client app I use). Once I have the audience secret key I can create the signing credentials for the token, and use it together with data from context to build my JWT.

public class ApplicationJwtFormat : ISecureDataFormat<AuthenticationTicket>
{
    private readonly Func<IanitorContext> _contextGetter;
    private string _sIssuer;
    public const string AUDIENCE_PROPKEY = "audience";

    private const string SIGNATURE_ALGORITHM = "http://www.w3.org/2001/04/xmldsig-more#hmac-sha256";
    private const string DIGEST_ALGORITHM = "http://www.w3.org/2001/04/xmlenc#sha256";

    public string Issuer
    {
        get { return _sIssuer; }
        set
        {
            if (value == null) throw new ArgumentNullException("value");
            _sIssuer = value;
        }
    }

    public ApplicationJwtFormat(Func<IanitorContext> contextGetter)
    {
        if (contextGetter == null) throw new ArgumentNullException("contextGetter");

        _contextGetter = contextGetter;
        Issuer = "http://localhost:50505";
    }

    public string Protect(AuthenticationTicket data)
    {
        if (data == null) throw new ArgumentNullException("data");

        // get the audience ID from the ticket properties (as set by ApplicationOAuthProvider
        // GrantResourceOwnerCredentials from its OAuth client ID)
        string sAudienceId = data.Properties.Dictionary.ContainsKey(AUDIENCE_PROPKEY)
            ? data.Properties.Dictionary[AUDIENCE_PROPKEY]
            : null;

        // get audience data
        Audience audience;
        using (IanitorContext db = _contextGetter())
        {
            audience = db.Audiences.FirstOrDefault(a => a.Id == sAudienceId) ??
                new Audience
                {
                    Id = ConfigurationManager.AppSettings["audienceId"],
                    Name = ConfigurationManager.AppSettings["audienceName"],
                    Base64Secret = ConfigurationManager.AppSettings["audienceSecret"]
                };
        }

        byte[] key = TextEncodings.Base64Url.Decode(audience.Base64Secret);

        DateTimeOffset? issued = data.Properties.IssuedUtc ?? 
            new DateTimeOffset(DateTime.UtcNow);
        DateTimeOffset? expires = data.Properties.ExpiresUtc;

        SigningCredentials credentials = new SigningCredentials(
            new InMemorySymmetricSecurityKey(key),
            SIGNATURE_ALGORITHM,
            DIGEST_ALGORITHM);

        JwtSecurityToken token = new JwtSecurityToken(_sIssuer,
            audience.Id,
            data.Identity.Claims,
            issued.Value.UtcDateTime,
            expires.Value.UtcDateTime,
            credentials);

        return new JwtSecurityTokenHandler().WriteToken(token);
    }

    public AuthenticationTicket Unprotect(string protectedText)
    {
        throw new NotImplementedException();
    }
}

4. Startup

Finally, the startup code to glue things together: the Global.asax code at Application_Start is simply a method call: GlobalConfiguration.Configure(WebApiConfig.Register);, which calls the typical WebAPI route setup code with a couple of additions to use only bearer authentication and return camel-cased JSON:

    public static void Register(HttpConfiguration config)
    {
        // Configure Web API to use only bearer token authentication.
        config.SuppressDefaultHostAuthentication();

        // Use camel case for JSON data
        config.Formatters.JsonFormatter.SerializerSettings.ContractResolver =
            new CamelCasePropertyNamesContractResolver();

        // Web API routes
        config.MapHttpAttributeRoutes();

        config.Routes.MapHttpRoute(
            name: "DefaultApi",
            routeTemplate: "api/{controller}/{id}",
            defaults: new { id = RouteParameter.Optional }
        );
    }

The OWIN startup configures the OWIN middleware:

public partial class Startup
{
    public void Configuration(IAppBuilder app)
    {
        HttpConfiguration config = new HttpConfiguration();

        app.UseCors(CorsOptions.AllowAll);
        app.UseWebApi(config);

        ConfigureAuth(app);
    }
}

The essential configuration is in the ConfigureAuth method, in a separated file as per template conventions (App_Start/Startup.Auth.cs): this has a couple of options wrapper classes for OAuth and JWT. Note that for JWT I add multiple audiences to the configuration, by getting them from the store. In ConfigureAuth I configure the dependencies for OWIN so that it can get instances of required objects (the EF data context and the user and role manager) and then setup OAuth and JWT using the specified options.

public partial class Startup
{
    public static OAuthAuthorizationServerOptions OAuthOptions { get; private set; }
    public static JwtBearerAuthenticationOptions JwtOptions { get; private set; }

    static Startup()
    {
        string sIssuer = ConfigurationManager.AppSettings["issuer"];

        OAuthOptions = new OAuthAuthorizationServerOptions
        {
            TokenEndpointPath = new PathString("/token"),
            AuthorizeEndpointPath = new PathString("/accounts/authorize"),  // not used
            Provider = new ApplicationOAuthProvider(),
            AccessTokenExpireTimeSpan = TimeSpan.FromHours(24),
            AccessTokenFormat = new ApplicationJwtFormat(IanitorContext.Create)
            {
                Issuer = sIssuer
            },
            AllowInsecureHttp = true   // do not allow in production
        };

        List<string> aAudienceIds = new List<string>();
        List<IIssuerSecurityTokenProvider> aProviders = 
            new List<IIssuerSecurityTokenProvider>();

        using (var context = IanitorContext.Create())
        {
            foreach (Audience audience in context.Audiences)
            {
                aAudienceIds.Add(audience.Id);
                aProviders.Add(new SymmetricKeyIssuerSecurityTokenProvider
                    (sIssuer, TextEncodings.Base64Url.Decode(audience.Base64Secret)));
            }
        }

        JwtOptions = new JwtBearerAuthenticationOptions
        {
            AllowedAudiences = aAudienceIds.ToArray(),
            IssuerSecurityTokenProviders = aProviders.ToArray()
        };
    }

    public void ConfigureAuth(IAppBuilder app)
    {
        app.CreatePerOwinContext(IanitorContext.Create);
        app.CreatePerOwinContext<ApplicationUserManager>(ApplicationUserManager.Create);
        app.CreatePerOwinContext<ApplicationRoleManager>(ApplicationRoleManager.Create);

        app.UseOAuthAuthorizationServer(OAuthOptions);
        app.UseJwtBearerAuthentication(JwtOptions);
    }
}

Edit #1 - client_id

While looking at several examples, I ended with this code in my ApplicationOAuthProvider:

public override Task ValidateClientAuthentication
    (OAuthValidateClientAuthenticationContext context)
{
    // http://bitoftech.net/2014/10/27/json-web-token-asp-net-web-api-2-jwt-owin-authorization-server/

    string sClientId;
    string sClientSecret;

    if (!context.TryGetBasicCredentials(out sClientId, out sClientSecret))
        context.TryGetFormCredentials(out sClientId, out sClientSecret);

    if (context.ClientId == null)
    {
        context.SetError("invalid_clientId", "client_Id is not set");
        return Task.FromResult<object>(null);
    }

    IanitorContext db = context.OwinContext.Get<IanitorContext>();
    Audience audience = db.Audiences.FirstOrDefault(a => a.Id == context.ClientId);

    if (audience == null)
    {
        context.SetError("invalid_clientId", 
            String.Format(CultureInfo.InvariantCulture, "Invalid client_id '{0}'", context.ClientId));
        return Task.FromResult<object>(null);
    }

    context.Validated();
    return Task.FromResult<object>(null);
}

While just validating, I make the actual check so that client_id is retrieved from the request's body, looked up in my audiences store, and validated if found. This seems to solve the issue noted above, so that now I get a non-null client ID in GrantResourceOwnerCredentials; I can also inspect the JWT contents and find the expected ID under aud. Yet, I keep getting 401 while passing any request with the received token, e.g.:

HTTP/1.1 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
Access-Control-Allow-Origin: http://localhost:50088
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?RDpcUHJvamVjdHNcNDViXEV4b1xJYW5pdG9yXElhbml0b3IuV2ViQXBpXGFwaVx2YWx1ZXM=?=
X-Powered-By: ASP.NET
Date: Wed, 22 Apr 2015 18:05:47 GMT
Content-Length: 61

{"message":"Authorization has been denied for this request."}

Answer 1

I implemented myself a JWT OAuth Authentication (with Bearer Token). I think you can definitely make your code lighter that what you currently have.

Here is the best post I found to read up on the fundamentals of how to secure a Web API with OAuth + JWT.

I don't have time to go further with your question for now. Good luck!

http://chimera.labs.oreilly.com/books/1234000001708/ch16.html#_resource_server_and_authorization_server

Also :

http://www.asp.net/web-api/overview/security/authentication-and-authorization-in-aspnet-web-api