Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Nginx rule to add x-robots-tag header

Tags:

nginx

I just moved from apache to nginx and cant fathom this rule.

I want to add this header to all pages that include a query string.

add_header X-Robots-Tag "noindex, nofollow";

The apache working rule was.

RewriteCond %{QUERY_STRING} (^|&)(sortOrder|sortField) [NC]
RewriteRule .* - [E=MY_SET_HEADER:1]
Header set X-Robots-Tag "noindex,nofollow" env=MY_SET_HEADER

Thanks in advance.

like image 591
h-y-b-r-i-d Avatar asked Dec 03 '15 22:12

h-y-b-r-i-d


People also ask

Where do I put the X-robots-tag?

Practical implementation of X-Robots-Tag You can add the X-Robots-Tag to a site's HTTP responses through the configuration files of your site's web server software. For example, on Apache-based web servers you can use . htaccess and httpd. conf files.

What is X-robots-tag HTTP header?

txt file and meta robots tag, though, in that the X-Robots-Tag is a part of the HTTP header that controls indexing of a page on the whole, in addition to specific elements on a page. According to Google: “Any directive that can be used in a robots meta tag can also be specified as an X-Robots-Tag.”

What is the value to be set for X-robots-tag header to prevent the crawlers from indexing a page?

HTTP response header Instead of a meta tag, you can also return an X-Robots-Tag header with a value of either noindex or none in your response.

How do you check X-robots-tag?

After crawling a site, you can easily check the “Noindex Pages” report to view all pages that are noindexed via the meta robots tag, the x-robots-tag header response, or by using noindex in robots. txt. You can export the list and then filter in Excel to isolate pages noindexed via the x-robots-tag.


1 Answers

The next line should be in the location block of your nginx config (usually in /etc/nginx/sites-avalable/):

add_header  X-Robots-Tag "noindex, nofollow, nosnippet, noarchive";

Should be nice to check before and after the modification with an:

# service nginx reload

Through this online tool: http://headers.cloxy.net

And it should show:

X-Robots-Tag: noindex, nofollow, nosnippet, noarchive
like image 88
Rafa Avatar answered Oct 07 '22 09:10

Rafa