sitemap for multiple domains of same site

Question

Here is the situation, i have a website that can be accessed from multiple domains, lets say www.domain1.com, www.domain2.net, www.domain3.com. the domains access the exact same code base, but depending on the domain, different CSS, graphics, etc are loaded.

everything works fine, but now my question is how do i deal with the sitemap.xml? i wrote the sitemap.xml for the default domain (www.domain1.com), but what about when the site is accessed from the other domains? the content of the sitemap.xml will contain the wrong domain.

i read that i can add multiple sitemap files to robots.txt, so does that mean that i can for example create sitemap-domain2.net.xml and sitemap-domain3.com.xml (containing the links with the matching domains) and simply add them to robots.txt?

somehow i have doubts that this would work thus i turn to you experts to shed some light on the subject :)

thanks

Answer 1

You should use server-side code to send the correct sitemap based on the domain name for requests to /sitemap.xml

Answer 2

Apache rewrite rules for /robots.txt requests

If you're using Apache as a webserver, you can create a directory called robots and put a robots.txt for each website you run on that VHOST by using Rewrite Rules in your .htaccess file like this:

# URL Rewrite solution for robots.txt for multidomains on single docroot
RewriteCond %{REQUEST_FILENAME} !-d # not an existing dir
RewriteCond %{REQUEST_FILENAME} !-f # not an existing file
RewriteCond robots/%{HTTP_HOST}.txt -f # and the specific robots file exists
RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [L]

NginX mapping for /robots.txt requests

When using NginX as a webserver (while taking yourdomain1.tld and yourdomain2.tld as example domains), you can achieve the same goal as post above with the following conditional variable (place this outside your server directive):

map $host $robots_file {
    default /robots/default.txt;
    yourdomain1.tld /robots/yourdomain1.tld.txt;
    yourdomain2.tld /robots/yourdomain2.tld.txt;
}

This way you can use this variable in a try_files statement inside your server directive:

location = /robots.txt {
    try_files /robots/$robots_file =404;
}

Content of /robots/*.txt

After setting up the aliases to the domain-specific robots.txt-files, add the sitemap to each of the robots files (e.g.: /robots/yourdomain1.tld.txt) using this syntax at the bottom of the file:

# Sitemap for this specific domain
Sitemap: https://yourdomain1.tld/sitemaps/yourdomain1.tld.xml

Do this for all domains you have, and you'll be set!