Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Robots.txt, disallow multilanguage URL

I have a public page that is not supposed be possible for users to sign into. So I have a url that there is no link to and you have to enter manually and then sign in.

The url is multilanguage however, so it can be "/SV/Account/Logon" or "/EN/Account/Logon" etc etc. Can I disable this url to be indexed for all languages?

like image 537
Oskar Kjellin Avatar asked Sep 03 '10 08:09

Oskar Kjellin


1 Answers

This url might help you http://www.robotstxt.org/robotstxt.html. There will be no wildcard exclusion in your case you will have to add all the language specific login urls in robot.txt

Update

you can put Disallow: /folder/subfolder/ or Disallow: /folder/ in your robots.txt file

like image 186
ajay_whiz Avatar answered Oct 14 '22 00:10

ajay_whiz