Clarify the usage of AppEngine Search API

Question

I have started to try out to use the new Search API, the demo is running smoothly, however, there are some points I am still confused about being outsider of the search world.

First of all is how to build a document. Obviously you can't hard-coded each line into a document, but what else can I do. Say if I have a user class (I'm using Java, but I guess Python makes no difference here), and I would add the user's information into the document, and be able to do a full-text search against the field of address.

class User {
   String username;
   String password;
   String address;
}

In my datastore, I have this entity with 10000 instances there, and if I will need to build this document, do I have to

Step 1: retrieve the 10000 instance from datastore

Step 2: Iterate through each of the user entity, and create 10000 documents

Step 3: Add all 10000 docs into an index, and then I will be able to search

Please correct me if above three steps I mentioned is wrong.

If that is the case, then does it that later each time a new User registered, we need to create a new document, and add to the index?

Answer 1

Unfortunately I haven't play around with that much. I learned a few things.

• When first implementing it, I hade to create a lot of documents as well (as you describe). But kept running in to deadline exceptions. So I ended upp using the task queue for building documents for all my old records.

• Remember to create a cross-reference between the search Document and you datastore entity. So you can easily update your document record. And from a search result get the match entity.

For cross-reference add a new property on your datastore model called something like search_document_id where you store the doc_id (I prefixed all my doc_id's with the datastore model name). And add a text field on you Document containing the entity key as a string.

But I would say in a nutshell you are correct.