robots.txt content itself is indexed? [closed]

Question

The contents of my robots.txt file are actually itself indexed and show up in Google search results. It's only Google and not Yahoo for example.

I really think Google should understand not to index the contents of my robots file as it's only there to tell Google what not to index!

Am I missing something here?

Answer 1

That's normal. Mine was indexed a few months ago too. I think it's an issue with the crawler. edit: btw, you can delete the link on the webmaster tools website.

Answer 2

I wouldn't recommend disallowing the robots.txt file from itself. Instead you can use the X-Robots-Tag http header with a value of noindex which will prevent the search engines from Indexing the file.

This will allow them access to your robots.txt but prevent it from showing up in the search results.