Best practice to create robots.txt file inside my asp.net mvc web site

Question

I want to create a robots.txt for my asp.net mvc-5 web site, now I find this link which talks about achieving this task:-

http://rehansaeed.com/dynamically-generating-robots-txt-using-asp-net-mvc/

where in this link they are creating a separate Controller & Route rule to build the robots.txt ,,,so I am not sure why i can not just create the robot.txt file and add it to the root of my web site as follow:-:-

where if I navigate to the following URL http://www.mywebsite.com/robots.txt the text content will be shown ,without having to create separate controller for this?

so my question is if it is valid to add the robots.txt directly to my web site root without having to do so inside a controller and separate route rule , to keep things simpler ??

Answer 1

On my website, asp net core 3.1, I just used the below simple action:

    [Route("/robots.txt")]
    public ContentResult RobotsTxt()
    {
        var sb = new StringBuilder();
        sb.AppendLine("User-agent: *")
            .AppendLine("Disallow:")
            .Append("sitemap: ")
            .Append(this.Request.Scheme)
            .Append("://")
            .Append(this.Request.Host)
            .AppendLine("/sitemap.xml");

        return this.Content(sb.ToString(), "text/plain", Encoding.UTF8);
    }