Memcache and Eventual consistency

Question

I'm working on a small project to learn about Google App Engine, the project is in Java and has Customer objects, instances of Customer can have a policy. Each customer is in its own entity group so that transactions can be used to modify the customer.

The main page of the site is a list of Customers, when a new customer is added the customer list is displayed again.

Since each customer is in their own entity group there are times when the newly added customer does not appear in the new customer list, refreshing the customer list after a few seconds and the customer will appear. A similar problem exists when deleting customers, you delete the customer but it appears in the overall list for a few seconds. I understand that this is to be expected in Google App Engine because of the eventual consistency that the datastore provides.

So I've tried to get around this problem by using memcache to store the customers that have been recently added or recently deleted. The code I'm using is below.

public List<Customer> getCustomers() {
    List<Customer> cachedCustomers = myCache.getCached();
    List<Customer> recentlyDeleted = myCache.getDeleted();
    // Calls the real datastore.
    List<Customer> dbCustomers = customerDao.getCustomerList();
    Set<Customer> allCustomers = new HashSet<Customer>();

    //  Add cached first as these are most the most up todate.
    allCustomers.addAll(cachedCustomers);
    allCustomers.addAll(dbCustomers);
    allCustomers.removeAll(recentlyDeleted);

    List<Customer> allList = new ArrayList<Customer>();
    allList.addAll(allCustomers);
    Collections.sort(allList);        
    return allList;
}

I'm asking here because I think that the way I'm going about this does not feel the 'right' way to do it and would like to hear from those who know better ways to get around the issues that eventual consistency creates.

Answer 1

What you are doing is what the docs recommends. So I believe it is the right way to do it.

Also, I made a Google and GitHub search for a library that could handle that for you but couldn't find it. So for this way of caching recent insertions and deletions feel more 'right', I would suggest you to write a library that handles this for any persistent class you want.

For an extra, I suggest reading this post where Ikai Lan, the Developer Programs Engineer of Google App Engine, explains how fetches and insertions work in HR and what are the performance implications compared to the Master/Slave datastore.