Google Custom Search while in development

Question

I'm doing a redesign for a client. On the new site I'd like to use Google Custom Search (CSE) as search engine. While I'm in development I can't let Google index the new site as it would be a horrible mess of duplication and half-done pages swimming around in Googles index.

So how do I test and refine the search result from Google CSE on my development site before launch?

Thanks, Daniel

Answer 1

Your sites have to be added to your webmaster tools account. Once I added my test sites, I was able to crawl the pages with the site search with a robots.txt file in the index that disallows site searching.

I was able to crawl the page but to verify I checked the url in webmaster tools and it says the page is still hidden from the main google index. I added this same url to the custom search index and if found it just fine.

So this will effectively allow you to search your test site but keep it hidden from the world search.