Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Google's crawler won't understand own maps. How to workaround?

I found strange words, (have, here, imagery, sorry) that were supposed not to be on my site, being taken as keywords by the crawler from Google

first site

It seems like Google is having errors when crawling pages that use Google maps, so it is taking the error strings as great keywords!

I am using openlayers to show maps in both sites. The code is like this

<script src="http://openlayers.org/api/OpenLayers.js"></script>
<script src="http://maps.google.com/maps/api/js?v=3&amp;sensor=false"></script>
<script type="text/javascript">
$(function() {
  $("#mapOuter").html('<div class="thumbnail"><div id="map" style="height:250px"></div></div>')
  map = new OpenLayers.Map("map")
  //map.addLayer( new OpenLayers.Layer.OSM   ("OpenStreeetMap") )
  map.addLayer( new OpenLayers.Layer.Google("Google v3"     ) )
  vectors = new OpenLayers.Layer.Vector("vector")
  map.addLayer( vectors )

  map.addControl( new OpenLayers.Control.LayerSwitcher() );
  map.addControl( new OpenLayers.Control.Navigation({documentDrag:true}) );
  map.addControl( new OpenLayers.Control.PanZoom() );
  var in_options = {
      'internalProjection': map.baseLayer.projection,
      'externalProjection': new OpenLayers.Projection("EPSG:4326")
  };

    var lon=-57.954900
    var lat=-34.917000

  map.setCenter(new OpenLayers.LonLat(lon, lat) // Center of the map
    .transform(
      new OpenLayers.Projection("EPSG:4326"), // transform from WGS 1984
      new OpenLayers.Projection("EPSG:900913") // to Spherical Mercator Projection
    ), 15 // Zoom level
  )

});
</script>

How can I do to fix this "error" so the Google crawler can take good content from my site?

Bonus Google Search (to show that the errors are indexed)

Google search

UPDATE, "Solution" applied:

I had one different map per each page in my site, I ended up converting all maps to images and only keep one interactive map where I really needed user interaction with coordinates and mapping stuff. The solution I used led me to create and opensource osm-static-maps. Hope it helps somebody!

The site got several improvements:

  • Got rid of this awkward words in google webmasters.
  • More relevant SEO using static images with "alt" html img attribute instead of "unindexable" js map.
  • Faster page loading (got rid of all mapping libraries and tile loading).
  • Faster js performance (less js to process by client)
  • Improved user experience: scrolling page caused map zooming instead of actually scrolling (you can think that this could be solved by disabling map scroll to zoom, but it lead to a user expecting to zoom the map on scroll, both ways were ok and wrong at the same time).

On the downside, I found:

  • Less user interactivity (boring page).
  • Less context on the map (less informative map).

This two things could be "fixed" loading the map when the user clicks the map img. The bad side is that if the user clicks the map img unintentionally, the map load can be seen as unexpected behaviour by the user.

Edit2

I made an opensource project out of this. Check out! https://github.com/jperelli/osm-static-maps

like image 535
jperelli Avatar asked Jul 02 '13 23:07

jperelli


People also ask

How do I change the crawling rate in Google Maps?

The term crawl rate means how many requests per second Googlebot makes to your site when it is crawling it: for example, 5 requests per second. You cannot change how often Google crawls your site, but if you want Google to crawl new or updated content on your site, you can request a recrawl.

Does Google allow crawlers?

If you want to block or allow all of Google's crawlers from accessing some of your content, you can do this by specifying Googlebot as the user agent. For example, if you want all your pages to appear in Google Search, and if you want AdSense ads to appear on your pages, you don't need a robots.

What's the page size limit for Googlebot crawlers?

"Googlebot can crawl the first 15MB of an HTML file or supported text-based file. Any resources referenced in the HTML such as images, videos, CSS, and JavaScript are fetched separately. After the first 15MB of the file, Googlebot stops crawling and only considers the first 15MB of the file for indexing.


2 Answers

Unfortunately i saw this a lot too...

My assumption is that googlebot won't fully evaluate all js code on a page, but will use heuristics as well. Thus getting no imagery (which gets indexed). Based on this assumption I did the following:

  1. Create a div with a "random" ID (for the map) and style="display: none;"

  2. Create a noscript tag with an img tag in it with the SAME "random" ID (i used a static map image as fallback here, which is also good as a no-js fallback)

  3. Create a (custom) javascript function where the unique ID must be passed to initialize your map AND toggle the display style to block on the map-element.

So far, none of the maps "sorry we have no imagery" gets indexed.

Hope it helps

like image 61
Ronald Swets Avatar answered Sep 16 '22 16:09

Ronald Swets


perhaps you can add a bit more specific meta tags such as

<meta name="geo.region" content="US-WA" />



<meta name="geo.placename" content="Snohomish" />



<meta name="geo.position" content="-57.954900;-34.917000" />

also adding what Matt Rowles meta description and some of the word filters in Google Webmasters.

like image 37
Hashes Avatar answered Sep 20 '22 16:09

Hashes