I am developing an asp.net mvc application. I am creating robots.txt for my application to prevent from bots because my current site is getting many robot requests. So I found this link, Robots.txt file in MVC.NET 4 to create robots.txt. But I when I access my application like this entering url, "www.domain.com/robots.txt", it is always returning 404 page.
This is my action method in HomeController
public ActionResult Robots()
{
Response.ContentType = "text/plain";
return View();
}
This is my robots view
@{
Layout = null;
}
User-agent:*
Disallow:/
I configured route for robots.txt like this in RouteConfig
public static void RegisterRoutes(RouteCollection routes)
{
routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
//routes.MapMvcAttributeRoutes();
routes.MapRoute(
"Robots.txt",
"robots.txt",
new { controller = "Home", action = "Robots" },
new string[] { "AyarDirectory.Web.Controllers" }
);
//other routes
}
But when I access this url, "www.domain.com/robots.txt", it is always returning 404 page. How can I add robots.txt correctly to my application?
Creating a route ending with a file extension is not allowed by default in ASP.NET MVC. To get around this security restriction, you need to add the following to the Web.config file:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<!-- ...Omitted -->
<system.webServer>
<!-- ...Omitted -->
<handlers>
<!-- ...Omitted -->
<add name="RobotsText"
path="robots.txt"
verb="GET"
type="System.Web.Handlers.TransferRequestHandler"
preCondition="integratedMode,runtimeVersionv4.0" />
</handlers>
</system.webServer>
</configuration>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With