What are the speed comparisons of NDB vs DB (on High Replication Datastore)?

Question

Taken from the Python NDB Overview:

When the application reads an entity, that entity is automatically cached; this gives fast (and inexpensive) reads for frequently-read entities.

...

The NDB function that writes the data (for example, put()) returns after the cache invalidation; the Apply phase happens asynchronously.

In watching on Youtube, Google I/O 2011: More 9s Please: Under The Covers of the High Replication Datastore, at 13:11-ish, the average latencies are:

Master/Slave:

• Read: 15ms
• Write: 20ms

High Replication:

• Read: 15ms
• Write: 45ms

How significantly does NDB affect these speeds, from the app's perspective?

Edit: Specifically curious about timing stats (in milliseconds).

Extra Credit: I've also heard Nick Johnson refer to queries taking around 160ms each (in 2009) [link] Does NDB provide any speed benefits on queries?

Answer 1

You'll have to benchmark for yourself -- times depend on many factors, like entity size and complexity: more properties or more items in repeated properties -> more complex.

The numbers you quote are really old and probably no longer reflect reality; most users' experience is that HRD is not slower than M/S, on average (in part because M/S has much higher variability).

There were some NDB benchmarks done here: http://code.google.com/p/appengine-ndb-experiment/issues/detail?id=118 -- but it doesn't compare the numbers to old db.

You can use Appstats to quickly do some timing of operations in a real app.

Answer 2

Using NDB makes your datastore calls appear, from your app's perspective, significantly faster.

READ: Best case scenario, reads are made from instance cache or memcache. In most cases, this will be significantly faster than reading from datastore.

WRITE: The NDB put/write method returns right after the cache invalidation. This is way faster than a normal write. So from your app's perspective, it's quite faster. The actual write, however, is performed asynchronously.

NDB vs DB (High Replication): In terms of speed from your app's perspective, NDB should be a clear win.