Issue Details (XML | Word | Printable)

Key: NUCHBASE-10
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Minor Minor
Assignee: Andy Jefferson
Reporter: Tatsuya Kawano
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
DataNucleus Store HBase

Make the "primary key constraint" optional to get the maximum scalability from the distributed datastore

Created: 04/Oct/09 11:03 AM   Updated: 27/Jan/11 07:54 PM   Resolved: 18/Jan/11 03:09 PM
Component/s: None
Affects Version/s: 1.0.2
Fix Version/s: 3.0.0.m1

File Attachments: 1. File patch-NUCHBASE-10.diff (2 kB)

Environment:
Unix (Mac OS X 10.6.1)
HBase 0.19.3

Forum Thread URL: http://www.jpox.org/servlet/forum/viewthread_thread,5787


 Description  « Hide
The plugin's insertObject() calls locateObject() to check if another row with the same primary key already exists on the database. In HBase and other distributed column-oriented datastore, it's pretty common not to rely primary key constraint, but manage the uniqueness of the primary keys from the application codes. This is one of the trade-offs to get the maximum scalability from a distributed datastore.

So, it would be nice if you can use this option or not via JDO configuration per entity (persistent class) basis.


Also, you can optimize the current implementation by not fetching an actual data, but run a server-side existence check "HTable#exists(Get get)", so that you can keep the network traffic minimum.

--
I will contribute a patch to resolve this issue.

Sort Order: Ascending order - Click to sort in descending order
Tatsuya Kawano added a comment - 06/Oct/09 09:44 AM
Attached "patch-NUCHBASE-10.diff" is the patch to enhance primary key existence check. Please apply it from /platform/store.hbase/trunk/

Tatsuya Kawano added a comment - 06/Oct/09 10:00 AM - edited
I have attached a patch which only improves the current implementation of primary key existence check .

I was also thinking to introduce a new JDO configuration option to turn this check off to gain extra performance from HBase, but since this patch already yields a good performance, I changed my mind and hold off to implement this feature until someone really needs it.

So, please just apply the patch, then close this issue.

Also, please see NUCHBASE-8. The patch I attached there contains all changes to resolve NUCHBASE-8, NUCHBASE-9, and NUCHBASE-10.

Andy Jefferson added a comment - 18/Jan/11 03:09 PM
SVN trunk allows the user to specify a persistence property
datanucleus.hbase.enforceUniquenessInApplication=true
and the check for existence is not performed, but instead a log informing them that they have responsibility after that.