How is MegaStore different from BigTable?

Question

It's been noted that Google App Engine is moving its datastore implementation from BigTable to MegaStore. What's the difference between the two?

Answer 1

As this article explains, "Megastore is a transactional indexed record manager built by Google on top of BigTable".

What Megastore adds on top of BigTable, again according to the URL I gave (of course I cannot discuss anything that Google hasn't yet made public!), is stuff that might not be easy to see from an App Engine app's viewpoint, depending on what App Engine may have already added on its own on top of BigTable. E.g., Megastore adds entity groups for transactional behavior... but App Engine has had those for a while. Do you really care how App Engine internally implements, or will implement in the future, identical APIs...?

Megastore supports schemas... but who's to know whether they'll be made available to App Engine apps (so that puts with the wrong combination of types will raise exceptions instead of silently succeeding), which so far have always been schemaless except for whatever you, yourself, or Google's open-source app-level code, implemented at application level.

Answer 2

Now more details of Megastore have emerged, including James Hamilton's summary which links to the CIDR 2011 paper.

Answer 3

the existing answers have described the differences between bigtable and megastore pretty well. i'll just add one thing: app engine isn't moving from bigtable to megastore. it's been on megastore since the beginning. (ok, well, at least since very very early in development, years before it launched publicly.)

for example, see this sept 2009 app engine blog post about megastore replication.