Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Controling eventual AppEngine datastory consistency during testing

I have an AppEngine app written in Go, and I'm trying to improve my tests.

Part of the tests that I need to run are a series of create, update, delete queries on the same object. However given that the datastore is eventually consistent (these aren't child objects), I am currently stuck using a time.Sleep(time.Second * 5) to give the simulated datastore in the SDK enough time for consistency to propagate.

This results in tests that take a long time to run. How can I force something more like strong consistency for tests without rewriting my code to use ancestor queries?

like image 608
JohnGB Avatar asked May 12 '14 23:05

JohnGB


People also ask

What is the difference between eventual consistency vs strong consistency?

Strong Consistency offers up-to-date data but at the cost of high latency. While Eventual consistency offers low latency but may reply to read requests with stale data since all nodes of the database may not have the updated data.

Is Datastore strongly consistent?

If application logic requires strong consistency, then the developer should use one of these methods to read entities from Datastore. Datastore is specifically designed to provide strong consistency on these APIs.

How can we achieve strong consistency in distributed system?

Strong Consistency: Strong Consistency simply means the data must be strongly consistent at all times. All the server nodes across the world should contain the same value as an entity at any point in time. And the only way to implement this behavior is by locking down the nodes when being updated.


2 Answers

Have a look at the dev_server arguments. You will see there is an option for setting the consistency policy.

 --datastore_consistency_policy {consistent,random,time}
                        the policy to apply when deciding whether a datastore
                        write should appear in global queries (default: time)

Notice the default is time, you want consistent

like image 179
Tim Hoffman Avatar answered Sep 18 '22 14:09

Tim Hoffman


It's been a while, but the method that I found that works well is to call the context as follows:

c, err := aetest.NewContext(&aetest.Options{StronglyConsistentDatastore: true})

like image 35
JohnGB Avatar answered Sep 20 '22 14:09

JohnGB