Google AppEngine - BigTable

Google App Engine (TM) platform provides Java persistence (JDO or JPA) to its BigTable datastore using a datanucleus-appengine plugin. Version 1 of this plugin was developed by Google and is their responsibility, only working with DataNucleus version 1.1. Version 2 of this plugin was developed partially by us, working with DataNucleus version 3.x.

Please read our blog about the current shortcomings of GAE/J and its implementation of JDO/JPA

Whilst stating that DataNucleus supports Google AppEngine we also need to point out the following to answer some comments made about GAE/J and DataNucleus.

  • DataNucleus is an independent project and provides complete JDO/JPA implementations for RDBMS, as well as quite complete implementations for ODBMS, XML, Excel, ODF, HBase, JSON, Amazon S3, LDAP etc. DataNucleus did not provide the plugin for BigTable, though we did offer help in creating it
  • Limitations of GAE/J's DataNucleus plugin (provided by Google) are limitations in that and in the BigTable datastore. They are not limitations in DataNucleus, nor are they limitations in the JDO/JPA specifications
  • GAE/J's DataNucleus plugin supports some non-standard JDO query syntax. This is against the JDO spec and has been raised with them.
  • GAE/J's documentation for JDO implies that each field needs to be marked @Persistent. This is wrong and misleading and has also been raised with them. The definitive documentation for JDO is in the JDO spec and the DataNucleus documentation, nowhere else
  • GAE/J's DataNucleus plugin could be extended in the future to remove some of the obvious limitations
  • GAE/J's DataNucleus plugin could easily be changed to take advantage of DataNucleus provided in-memory query evaluation and this would alleviate many concerns of users about missing query constructs. Code to do this was contributed to Google but is not included in their plugin yet