Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does "Allow: /$" mean in robots.txt

When digging through a Google robots.txt file I noticed a line that I was not familiar with.

What does the below code mean in the context of a robots.txt file?

Allow: /$

Does the '$' change the meaning any from simply saying

Allow: /
like image 335
Kyle Piira Avatar asked Jan 19 '16 14:01

Kyle Piira


Video Answer


2 Answers

According to SearchEngineLand..

You can match the end of the string using the dollar sign ($). For example, to block URLs that end with .asp:

User-agent: Googlebot
Disallow: /*.asp$

Looks like it acts a bit like a regular expression.

like image 188
ddavison Avatar answered Oct 22 '22 15:10

ddavison


Some robots understand pattern matching. $ is used in regular expressions to mark end of line. So this rule should Allow / but not /foo.

I have no source for robots.txt particular though.

like image 3
ketilkn Avatar answered Oct 22 '22 16:10

ketilkn