I have the following .htaccess file in my root:
Options +FollowSymlinks
RewriteEngine on
RewriteBase /
RewriteRule ^(.*)\/(\?.*)?$ $1$2 [R=301,L]
RewriteRule ^([^/]*)$ index.php?page=$1 [NC]
This works as it should for shortening all my URLs to website.com/something
The problem is Google can't find my robots.txt
file in my root. The above file isn't letting it through. when It type website.com/robots.txt I get a 404 not found. But if I comment out the above .htaccess
code I can get to it just fine.
How can I edit my .htaccess
file to let robots.txt
through without interfering with my other URLs?
RewriteEngine on
RewriteRule ^robots.txt - [L]
Second line will exclude robots.txt from URL rewritting rules . Try above code
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With