Issue Details (XML | Word | Printable)

Key: NUCCORE-904
Type: Bug Bug
Status: Open Open
Priority: Testcase Required Testcase Required
Assignee: Unassigned
Reporter: Daniel Baldes
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
DataNucleus Core

Optimistic Transactions / Queued Operations Problems

Created: 21/Aug/12 09:45 AM   Updated: Wednesday 09:21 AM
Component/s: Persistence
Affects Version/s: 3.0.11
Fix Version/s: None

File Attachments: 1. GZip Archive datanucleus-core-3.0.11-NUCCORE-904.patch.gz (7 kB)


Forum Thread URL: http://www.datanucleus.org/servlet/forum/viewthread_thread,7235


 Description  « Hide
The current technique used to queue operations in optimistic transactions is flawed: it queues operations *per collection* and does not honour the order of operations between different collections. If I have an optimistic transaction which does the following:

(given: two objects with collection properties, one object of element type)

objectA.collection.add(element)
objectA.collection.remove(element)
objectB.collection.add(element)
objectB.collection.remove(element)
objectA.collection.add(element)


The outcome after commit depends on the order in which the collections are processed.

A: add(element), remove(element), add(element)
B: add(element), remove(element)

With relationship management, element.owner would now be null; if B is processed first, it would be A. If everything was processed in order, it would be A (i.e. A would be correct / expected).

Please see the linked forum thread for further details. I've opened this issue so we do not forget to include a fix for this in DN 3.2.



Sort Order: Ascending order - Click to sort in descending order
Daniel Baldes added a comment - 21/Aug/12 10:07 AM
As a reference, I attached my patch against datanucleus-core 3.0.11 which implements per-transaction operation queueing (thereby fixing this issue).

As discussed in the forum, this patch is not meant to be incorporated in DataNucleus before 3.2.

Please also note that this is "alpha software" and still undergoing testing.

Andy Jefferson added a comment - 14/Oct/12 11:59 AM
Just a note that core is now branched, so trunk is for DN3.2.

Comments on what you attached :-
1. Don't use JDOHelper in core code, use ApiAdapter (for the API in use) which comes from ExecutionContext, so that way core code is API independent.
2. Any release with this in I'd expect to have it enablable using a persistence property (I know this hasn't been your focus thus far just on getting something that works, but just bear it in mind).
3. I'd expect changes to go into ObjectManagerImpl.flush() rather than in TransactionImpl just before flush. The ObjectManagerImpl should look after all changes, and syncing these to the DB.

Daniel Baldes added a comment - 15/Oct/12 10:09 AM
Thanks for the update. Unfortunately, I'm facing a close deadline on a different project and therefore could not proceed with this. When is DN3.2 going to be released?

Andy Jefferson added a comment - 15/Oct/12 10:46 AM
I've not even released milestone 1 so i'd guess at 4-6 months before 3.2.0.release. Plenty of time ;-)

Andy Jefferson added a comment - 28/Oct/12 05:25 PM
See http://www.datanucleus.org/servlet/jira/browse/NUCCORE-944 which may impact on your proposal (simplifying any patch)

Daniel Baldes added a comment - 29/Oct/12 11:00 AM
Thanks for the update. I think I'll find some time this or next week to work on this issue.

Andy Jefferson added a comment - 15/Feb/13 03:30 PM
nearing the last opportunity to put something of this form into 3.2

Daniel Baldes added a comment - 18/Feb/13 08:54 AM
I'm sorry, I'm getting nowhere these days. The issue is not extremely urgent for me right now, so I probably won't come up with a patch soon.

Andy Jefferson added a comment - 05/Mar/13 09:47 AM
NUCCORE-1005 moves OperationQueue into ExecutionContext (as SCOOperationQueue) and makes it a queue for all SCOs in that context (rather than just a queue for that SCO). That makes it possible to only apply the operations that are needed. For this particular sequence of operations SCOOperationQueue "perform" needs updating to check whether the datastore and relation type only allows a single owner for that particular field and, if so, then only apply the change if there are no subsequent additions/removals involving the same element.

In the same way SCOOperationQueue has all of the info there necessary to know if it should trigger cascade delete when an element is removed from a collection (if the same element has its owner reassigned later in the same queue then it should not trigger cascade delete).



Relationship Management (mentioned in the description of this issue) is a separate concept, that should be tackled in a separate issue. The RelationshipManager already has a List of operations that are being performed so could equally be given knowledge of when to apply a change and when not to (if having its owner reassigned).

Chris Colman added a comment - 05/Mar/13 12:46 PM
We had a scenario where there was a natural 'contains' relationship between 'owner' and 'child' for which we configured 'dependent' in the metadata (i.e. cascade delete) but we needed to occasionally reassign the child to a different owner object - but this caused an automatic deletion of the child. We ended up having to change the relationship to a 'uses', remove the 'dependent' attribute in the metadata and manage child deletion manually.

I guess the enhancement to relationship management that you describe would fix this.

+1 from me ;)

Andy Jefferson added a comment - 27/Jan/14 08:33 PM
Raiser now working elsewhere so unlikely to get any input into this

Andy Jefferson added a comment - 18/Nov/14 08:14 PM
Would have been nice to have the tests that were mentioned in the forum thread ...

Andy Jefferson added a comment - 19/Nov/14 09:21 AM
Can't reproduce anything from the pseudo code in the description. Changes of this nature need clearly defined expectations