Menu
XML sitemaps can also contain additional information about each URL, in the form of meta data. And just like robots. txt, an XML sitemap is a must-have. It's not only important to make sure search engine bots can discover all of your pages, but also to help them understand the importance of your pages.
An XML sitemap shouldn't override robots. txt. If you have Google Webmaster Tools setup, you will see warnings on the sitemaps page that pages being blocked by robots are being submitted.
The sitemap. xml provides search engines with a direct path to each page of your site, which offer quick indexing for all the pages whereas robots. txt is used to tell search engines what pages to crawl and what not to crawl.
According to the official documentation on sitemaps.org it needs to be a full URL:
You can specify the location of the Sitemap using a robots.txt file. To do this, simply add the following line including the full URL to the sitemap:
Sitemap: http://www.example.com/sitemap.xml
Google crawlers are not smart enough, they can't crawl relative URLs, that's why it's always recommended to use absolute URL's for better crawlability and indexability.
Therefore, you can not use this variation
> sitemap: /sitemap.xml
Recommended syntax is
Sitemap: https://www.yourdomain.com/sitemap.xml
Note:
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With