MVC: 404 route not working properly in production

Question

I have the following route at the bottom of my global.asax:

//404 ERRORS:
routes.MapRoute(
    "404-PageNotFound",
    "{*url}",
    new { controller = "Error", action = "PageNotFound" }
);

Which works fine in Visual Studio, but in production I'm getting the IIS error page.

Shouldn't this route catch any URL not caught by the others, therefore no 404s from IIS' perspective? Is there anything else I need to do in web.config?

Note: I do not want to redirect to a 404-specific URL; rather I am serving up the 404 error page at the requested URL (I believe this is the proper approach from a usability point of view).

UPDATE
In my Error controller, I'm setting Response.StatusCode = 404;, which seems to be the issue. When I remove this and deploy to production again, I'm getting my friendly error pages again. However, I believe I do need the 404 status in the HTTP header -- for SEO purposes -- so my question now becomes:

REVISED QUESTION
How/why is IIS intercepting the response and sending its out-of-the-box 404 error, and how do I prevent this?

**SOLUTION**
Dommer gets the prize for suggesting Response.TrySkipIisCustomErrors=true; which (I think) was necessary. But there were 2 other key details:

• custom errors needs to be on in web.config (duh!), and
• the 404 action must have the [HandleErrors] attribute.

Making it work everywhere
Since some URLs may get mapped to routes other than "404-PageNotFound" but containing invalid parameters, and since I don't want to redirect to the 404 page, I created this action in my base controller:

[HandleError]
public ActionResult NotFound()
{
    Response.StatusCode = 404;
    Response.TrySkipIisCustomErrors = true;     
    return View("PageNotFound", SearchUtilities.GetPageNotFoundModel(HttpContext.Request.RawUrl));          
}

and in any controller-action that inherits the base, I simply need to call this whenever I catch an invalid route parameter:

return NotFound();

note: Not RedirectToAction()

Icing on the cake:
The model I've generated and passed into the view is about feeding the wordy bits of the URL into our search engine, and displaying the top 3 results as suggestions on the friendly 404 page.

Answer 1

The problem is that your request is matching one of your earlier routes and then failing. A possible solution is to try constraining the other routes like this:

// This will match too many things, so let's constrain it to those we know are valid
   routes.MapRoute(
    "Default",
    "{controller}/{action}/{id}",
    new { controller = "Home", action = "Index", id = UrlParameter.Optional },
    new { controller = "Home|OtherController|AnotherController|..." } // regular expression matching all valid controllers
);

//404 ERRORS:
routes.MapRoute(
    "404-PageNotFound",
    "{*url}",
    new { controller = "Error", action = "PageNotFound" }
);

If you want a much more comprehensive answer, then look at this: How can I properly handle 404 in ASP.NET MVC?

EDIT:

To get rid of the custom IIS error try changing your error controller to be like this:

Response.StatusCode = 404;
Response.TrySkipIisCustomErrors=true; // add this line

MSDN doc: http://msdn.microsoft.com/en-us/library/system.web.httpresponse.tryskipiiscustomerrors.aspx