How to Have Search Engines Index Database-Driven Content?

Question

How can I make it so that content from a database is available to search engines, like google, for indexing?

Example:

Table in mysql has a field named 'Headline' which equals 'BMW M3 2005'. My site name is 'MySite'

User enters 'BMW M3 2005 MySite' in google and the record will show up with results?

Answer 1

Google indexes web pages, so you will need to have a page for each of your records, this doesn't mean to say you need to create 1,000 HTML pages, but following my advice above will dynamically / easily provide a seemingly unique page for each product.

For example:

www.mydomain.com/buy/123/nice-bmw-m3-2005

You can use .htaccess to change this link to:

www.mydomain.com/product.php?id=123

Within this script you can dynamically create each page with up-to-date information by querying your database based on the product id in this case 123.

Your script will provide each record with it's own title ('Nice BMW M3 2005'), a nice friendly URL ('www.mydomain.com/buy/123/nice-bmw-m3-2006') and you can include the correct meta information too, as well as images, reviews etc etc.

Job done, and you don't have to create hundreds of static HTML pages.

For more information on .htaccess, check out this tutorial.

Answer 2

What ILMV is trying to explain is that you have to have HTML pages that Google and other search engines can 'crawl' in order for them to index your content.

Since your information is loading dynamically from a database, you will need to use a server-side language like PHP to dynamically load information from the database and then output that information to an HTML page.

You have any number of options for how to accomplish this specifically, ILMV's suggestion is one of the better ways though.

Basically what you need to do first is figure out how to pull the information from the database and then use PHP (or another server-side language) to output the information to an HTML page.

Then you will need to determine whether you want to use the uglier, default url style for php driven pages:

mysite.com/products.php?id=123

But this url is not very user or search engine friendly and will result in your content not being indexed very well.

Or you can use some sort of URL rewriting mechanism (mod_rewrite in a .htaccess file does this or you can look at more complex PHP oriented solutions like Zend Framework that provide what's called a Front Controller to handle mapping of all requests) to make it so that your url's look like:

mysite.com/products/123/nice-bmw-m3-2006

This is what ILMV is talking about with regard to url masking.

Using this method of dynamically loading content will allow you to develop a single page to load the information for a number of different products based on the Id thus making it seem to the various search engines as though you have a unique page for each product.