Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Need to block subdomain using robots.txt which is on same directory level

Tags:

seo

robots.txt

I have one problem

I have domain name for example www.testing.com and new.testing.com so i do not want to new.testing.com display in any search engine. I have added one robots.txt to the new.testing.com. And both site has same parent directory

--httpdoc
----testing.com
----new.testing.com

So i want to know that can i handle both site using one robots.txt of testing.com is it possible???

Please suggest me solution if possible.

like image 784
Jalpesh Patel Avatar asked Sep 19 '14 07:09

Jalpesh Patel


1 Answers

The best thing you could do is to add separate robots.txt files. Put one in each directory. You should have

  • testing.com/robots.txt and
  • new.testing.com/robots.txt

After adding robots.txt file to new.testing.com, you should add following code to keep search engines away.

User-agent: *
Disallow: /
like image 198
bmiljevic Avatar answered Jan 02 '23 16:01

bmiljevic