Menu
My site is comprised of 3 main sections: Reviews, Forum, and Blog. I have plugins for the forum and blog that automatically generate sitemaps for them. The forum plugin generates a sitemap INDEX file pointing to multiple indexes, and the blog plugin generates a regular sitemap file containing all my blog content. Here are their entries from robots.txt:
Sitemap: http://www.datesphere.com/forum/sitemap-index.xml
Sitemap: http://www.datesphere.com/blog/sitemap.xml
I just created a Reviews sitemap.xml file that contains all the content in the Reviews section. I was planning to just add a line to robots.txt so the whole thing would look like this:
Sitemap: http://www.datesphere.com/forum/sitemap-index.xml
Sitemap: http://www.datesphere.com/blog/sitemap.xml
Sitemap: http://www.datesphere.com/reviews-sitemap.xml
HERE'S MY QUESTION: I know you can list multiple sitemaps in robots.txt, but is it OK to have a sitemap index file as well as multiple sitemaps listed? Will Googlebot ignore the other sitemap files if it finds a sitemap-index.xml file in robots.txt? If so, do I have to put my blog and reviews sitemaps in another sitemap index file and just list that in robots.txt?
I've checked around but can only find answers to the question "can I list multiple sitemaps?"
776
You can specify more than one Sitemap file per robots. txt file. Highlight by me, this can not be misread I'd say, so simply spoken, this can be done. This is also necessary for cross-submits, for which btw.
Yes, you can! You might want to have multiple Sitemap files in a single directory for a number of reasons. For instance, if you have an auction site, you might want to have a daily Sitemap with new auction offers and a weekly Sitemap with less time-sensitive URLs.
Just Sign into Google Sitemaps and submit the Sitemap index file. You don't need to submit individual Sitemaps that are included in the index. Once we've processed your Sitemap index file, we'll let you know if we found errors in the Sitemap index itself, or in any of the individual Sitemaps.
If you already have a robots. txt file, you can add the rule Sitemap: https://www.example.com/sitemap_index.xml to your file via the file editor in the Tools section of Yoast SEO. Keep in mind that you should add the full URL to your XML sitemap. Multiple sitemaps go on multiple lines and all need full URLs.
Googlebot will not ignore any of the Sitemaps you list in robots.txt even if you list their parent Sitemap Index, too. We follow pretty much every link we find and if we're allowed to, we'll crawl them. Personally, I'd probably list only the Sitemap Indexes, though only for manageability's sake, but it's up to you, Googlebot won't mind if you list both the indexes and the Sitemaps.
70
When you have multiple sitemaps, you can either specify your sitemap index file URL in your robots.txt file as shown in the example below:
// robots.txt
Sitemap: http://www.example.com/sitemap_index.xml
User-agent:*
Disallow: /some/disallowed/path
Or, you can specify individual URLs of your multiple sitemap files, as shown in the example below:
// robots.txt
Sitemap: http://www.example.com/sitemap_host1.xml
Sitemap: http://www.example.com/sitemap_host2.xml
User-agent:*
Disallow: /some/disallowed/path
Finally, this is what you need to pay attention to when adding the Sitemap directive to the robots.txt file.
22
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
