Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do Google and Bing crawlers treat hash tags in URLs as GET parameters?

Tags:

seo

Crawlers in genereal consider http://server/page and http://server/page?parameter=1 two different URLs.

How do Google and Bing crawler consider hash tag URLs, such as http://server/page#hash?


According to http://www.tynt.com/support/faq#technical everything after the hash tag is ignored. Are there other sources confirming this?

like image 798
David Andersson Avatar asked Jun 16 '11 10:06

David Andersson


People also ask

Does Google crawl query parameters?

Google May Crawl Parameter URLs Set to Crawl: None in the Parameter Handling Tool in GSC. Google may still crawl parameter URLs, even if you set the parameter to crawl: none in GSC. If you want to ensure the URLs never get crawled John recommends using the robots. txt file instead.

Are hashtags allowed in URLs?

Hash symbols in page URLs are acceptable if you are using them on the same page to point the users to different sections. However, it is not recommended to use hashtags (#) for different unique pages.

What is URL parameters in search console?

URL Parameters are parameters with values that are set dynamically within a pages URL. This enables a single page to show an infinite number of different views. Active parameters can change page content for users by transforming or sorting a page a certain way.


2 Answers

Your source is correct. Everything after the hash tag (aka URL fragment) is typically ignored. The reason being, the URL fragment is typically only used by the browser and doesn't cause additional information to be pulled from the server. So the content of the page should largely stay the same.

For instance, the hash parameter is typically used in 2 ways, either as a page anchor (think 'go to top of page') or as ways of passing information via javascript.

The exception to the rule is the AJAX crawlability implementation. In this case if you have a "hash-bang" (#!), Google and Bing (maybe) will attempt to crawl your AJAX content, which will treat your hash parameter values as separate pages.

Google => "Making AJAX Applications Crawlable"

Search Engine Land => Bing Now Supports Google’s Crawlable AJAX Standard?

like image 159
eywu Avatar answered Sep 18 '22 05:09

eywu


This strictly depends on the concrete crawler implementation, there is no general rule enforcing any behavior.

like image 28
Waldheinz Avatar answered Sep 20 '22 05:09

Waldheinz