Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to completely Hide website from search engines?

Whats the best recommended way yo hide my staging website from search engines, i Googled it and found some says that i should put a metatag, and some said that i should put a text file inside my website directory, i want to know the standard way.

my current website is in asp.net, while i believe that it must be a common way for any website whatever its programming language.

like image 700
Amr Elgarhy Avatar asked Feb 20 '10 20:02

Amr Elgarhy


3 Answers

Use a robots.txt file.

see here http://www.robotstxt.org/robotstxt.html

like image 54
NinjaBomb Avatar answered Nov 11 '22 21:11

NinjaBomb


You could also use your servers robots.txt:

User-agent: *
Disallow: /

Google's crawler actually respects these settings.

like image 29
amphetamachine Avatar answered Nov 11 '22 20:11

amphetamachine


Really easy answer; password protect it. If it’s a staging site then it quite likely is not intended to be publicly facing (private audience only most likely). Trying to keep it out of search engines is only treating a symptom when the real problem is that you haven’t appropriately secured it.

like image 5
Troy Hunt Avatar answered Nov 11 '22 21:11

Troy Hunt