Menu
Where should put robots.txt?
domainname.com/robots.txt
or
domainname/public_html/robots.txt
I placed the file in domainname.com/robots.txt, but it's not opening when I type this in browser.
alt text http://shup.com/Shup/358900/11056202047-My-Desktop.png
355
The robots. txt file must be located at the root of the website host to which it applies. For instance, to control crawling on all URLs below https://www.example.com/ , the robots. txt file must be located at https://www.example.com/robots.txt .
Warning: Don't use a robots. txt file as a means to hide your web pages from Google search results. If other pages point to your page with descriptive text, Google could still index the URL without visiting the page.
A robots. txt file is a text document that's located in the root directory of a site that contains information intended for search engine crawlers about which URLs—that house pages, files, folders, etc.
A site without a robots. txt will be fully-indexed.
Where the file goes in your filesystem depends on what host you're using, so it's hard for us to give a specific answer about that.
The best description is: put it wherever the index.html (or index.php or whatever) file is that represents your homepage. If that's domainname/public_html/index.html, for example, put it in domainname/public_html/robots.txt.
123
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
