SEO implications of a multi lingual site with detection of system culture

Question

I have developed a multi-lingual site in ASP.NET, which detects the user's system culture, and displays content in the appropriate language.

This is all working nicely, but my client has since had an SEO audit. The SEO agency has expressed a concern that this is not good SEO practice, as there are not unique URLs for each language.

They have suggested that the site may be accused of cloaking, and that google may not index the site correctly for each different language.

Any ideas on whether these are valid concerns, and if there is an advantage to having unique URLs for each language version of the site?

Answer 1

Although you have done a beautiful job switching Language automatically, the SEO Agency is correct!

That google may not index the site correctly for each diffferent language.

This is true! Google doesn't send the accept-language header last time I checked. This means that Google will only index the default language.

They have suggested that the site may be accused of cloaking,

This differs from your Excact implementation, but it is possible your site will receive a penalty!

There IS advantage having unique URLs (for each language version) on the site!
First of all, for your users: they can link to the language they prefer. Secondary for the Search Engines as they can index your site correctly.

I advice most of the time to redirect the user only on the home page for a language switch using a 302 redirect to the correct URL (and so the correct language). (edit: You can review the post by Matt Cutts "SEO Advice: Discussing 302 redirects")

To verify my advice: install fiddler and surf to http://www.ibm.com. As shown below, i received a 302 redirect to the appropriate language, arriving at www.ibm.com/be/en.

    Result  Protocol    Host    URL Body    Caching Content-Type    
4   302 HTTP    www.ibm.com /   209     text/html
5   200 HTTP    www.ibm.com /be/en/ 5.073   no-cache  text/html;charset=UTF-8

There are a few solutions you can solve this:

• Start Rewriting Urls (adding e.g. a directory using the language)
• If you don't want to go through the hassle of adding directories (or rewriting url's) adding a QueryString would be the easiest solution (although try limiting them to maximum 2 parameters)
• Another option is using different sub-domains! www.website.com for the default language, es.website.com, fr.website.com

Just make sure you supply every time the same content for the same URL.

Good luck with it!

Answer 2

Hopefully we will see some people answer who know about the internals of Google (anyone?). But most suppositions about how Google and others' crawlers are... suppositions, and subject to change.

My guess is that you should use separate URLs for languages, even if they just have a ?language= difference (although better would be a truly different URL). I believe this because when you go to google.it it says, google.com in English and that link goes to... google.com. In other words, Google itself uses different URLs for different languages.

Also, another big site, Microsoft (they probably know about SEO) uses

http://www.microsoft.com/en/us/default.aspx

for US-English and

http://www.microsoft.com/it/it/default.aspx

for Italy-Italian so it's probably best practice to differentiate based on language (and country).

In any case, I am totally annoyed when I'm on an English language computer and I can't see a site in Italian or Spanish, and vice-versa. As a usability, not SEO strategy, the user should be able to override the language suggestion. This is how most big sites handle languages, too.