Issue Details (XML | Word | Printable)

Key: NUCCORE-361
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Andy Jefferson
Reporter: Renato Garcia
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
DataNucleus Core

Unexpected flush for 1-to-1 + dependent="true" with optimistic lock

Created: 14/Aug/09 11:59 PM   Updated: 04/Oct/12 05:02 PM   Resolved: 21/Aug/09 02:58 PM
Component/s: Persistence
Affects Version/s: 1.1.5, 2.0.0.m1
Fix Version/s: 2.0.0.m2

File Attachments: 1. Text File DependentFieldTest-patch.txt (4 kB)
2. Text File patch.txt (1 kB)
3. Zip Archive TCK-patch-passed.zip (948 kB)
4. Zip Archive test-result.zip (545 kB)
5. Zip Archive testrun.out.zip (33 kB)


Forum Thread URL: http://www.jpox.org/servlet/forum/viewthread_thread,5712
Severity: Production


 Description  « Hide
A flush during causes problem to optimistic lock. See fourm details.

Renato Garcia added a comment - 15/Aug/09 12:01 AM
Test results

Renato Garcia made changes - 15/Aug/09 12:01 AM
Field Original Value New Value
Attachment unit-test-result.zip [ 10991 ]
Attachment TCK-before.zip [ 10992 ]
Attachment TCK-after.zip [ 10993 ]
Renato Garcia made changes - 15/Aug/09 12:02 AM
Attachment patch.txt [ 10994 ]
Andy Jefferson added a comment - 15/Aug/09 08:00 AM
In the thread you say
<thread>Additionally to the flush problem there is a scenario that gives the exception below:
Exception in thread "A" Transient instances cant be deleted.
org.datanucleus.exceptions.NucleusUserException: Transient instances cant be deleted.
at org.datanucleus.ObjectManagerImpl.deleteObjectInternal(ObjectManagerImpl.java:1447)
at org.datanucleus.state.JDOStateManagerImpl.setObjectField(JDOStateManagerImpl.java:2440) </thread>

Please provide a testcase using the current samples that demonstrates this


The JDO TCK should pass fully. It does for me. It does for all other people on Apache JDO.

Andy Jefferson added a comment - 15/Aug/09 09:52 AM
Also, the DN unit tests seem to be run in some strange way. You have "NativeQueryTest", yet this is in a scenario for db4o (also CriteriaQueryTest). There is RelationshipTest in "test.jdo.orm.datastore" and "test.jdo.orm.application" yet this appears once and no idea which it is. You have "ApplicationIdentityTest" yet this works for me ... presumably under "test.jdo.application". Also "DatastoreIdPersistenceTest". Also ApplicationIdPersistenceTest.

Renato Garcia added a comment - 17/Aug/09 10:28 PM
1 - Testcase for "Transient instances cant be deleted"
I've included a new test scenario testNullifyNonPersistent1to1Relation using the existing DependentTestField, which reproduces the problem described. (DependentFieldTest-patch.txt)

2 - Unit Test
I ran a "mvn clean test" for the aggregator project test.maven2.parent(This is probably the "strange way" ;) Didn't know about the run_tests.sh script). I've noticed that ApplicationIdentityTest will fail using this "aggregator approach" with a "Unique constraint violation", probably other stuff won't work either.

So, I found the run_tests.sh script, and I've ran the tests again using it.
The NEW results(test-result.zip) are attached along with the build output(testrun.out.zip). It already includes the new scenario above.

Please, could you check if they look alright now?(There are errors yet, but I assume they are already known).

3 - TCK
I'm still trying to figure it out what I did wrong. I've run the tests using the 2.3-ea branch, I had to modify the project.xml to point to 2.0.0-SNAPSHOT version and force the "eb" version into the repo, by replacing jdo2-api-2.3-ea.jar.

Renato Garcia added a comment - 17/Aug/09 10:29 PM
New scenario included

Renato Garcia made changes - 17/Aug/09 10:29 PM
Attachment DependentFieldTest-patch.txt [ 10997 ]
Renato Garcia added a comment - 17/Aug/09 10:30 PM
Build output for run_tests.sh

Renato Garcia made changes - 17/Aug/09 10:30 PM
Attachment testrun.out.zip [ 10998 ]
Renato Garcia added a comment - 17/Aug/09 10:31 PM
Test result after patches applied.

Renato Garcia made changes - 17/Aug/09 10:31 PM
Attachment test-result.zip [ 10999 ]
Renato Garcia added a comment - 17/Aug/09 10:39 PM
Andy,
I would like a little help in order to understand better the test project structure... I've noticed that for some files/test, they appear into different projects, even thought they are identical. Are they links? Should they be considered the same? I already took a look at http://www.datanucleus.org/development/test/unittests.html, but I could not find anything about this.
i.e. DependentFieldTest appears into test.jdo.datastore and test.jdo.application, and they are identical.

Tks!

Andy Jefferson added a comment - 18/Aug/09 06:30 AM
Whether a test appears in test.jdo.datastore and test.jdo.application is of no relevance. A test scenario is to be run on its own. In that particular case, one tests Dependent fields for datastore id, and one for Application id. They are separate tests (different samples used) even though the JUnit code is the same.

Having a separate M2 project means that M2 project is to be run ... on its own ... separately. We don't use any aggregator project for running tests; that is a superclass for inheritance only.

Andre Luis Iasi Moura added a comment - 19/Aug/09 09:43 PM
Andy,

I work on the same project Renato works, and I've been following this thread and working with him trying to make the tests work here. First, I would like to state and make it clear that we see DN as a great solution. And, because we like so many aspects of it and really believe in the project, we chose to build our own project having DN as one of its supporting foundations. For that, we really would like to give our share of contribution, not only to solve the problems we face, but also to make the platform stronger.

I really understand that developing such a project can be very overwhelming, and I understand it's common sense to limit the time you share to help others you don't know, and, for that sake, don't thrust, even though they say they are trying to help. But I assure you that we have a real interest in help and, more important, we have the skills to do that. Today, we are venturing in a project of our own, but for years we've been working in development of software for the majors companies here where we live (that is São Paulo, Brazil), like Bradesco Bank, Itaú Bank, or Telefonica (in fact, on the third parties that were responsable for their software development). And at the last few years we have been responsable for the architecture and management of our projects, so our knowledge comes not only from years of experience but also from the responsability of making a project stand and going on.

I believe you will hear from us once in a while for the next few years, and so our interest in make the tests work is not only to solve a localized problem. If we have these tests working it would be easy to help, it would be easy to check if our contributions are valid and it would be less work for you and for us. Now, you must know, it is not an easy task to make those tests run smoothly only by our selves. And, as you may imagine, we have also a limited share of time to spend on that, so we do not have the luxury to be able to test all the variables. Renato has been passing our results and commenting what we did, so you know our status. You are the best person to have some idea, for you is the one who knows the project deeply. Maybe there is only a set of tests that are those that matters, maybe you (all of you from the project) have something on your local machine that we must have, maybe there is some information missing you could give us, maybe you could send us some of your configuration files, maybe even you have some doubts about what we are doing that we must clarify first... I don't know. What I'm asking from you is the kindness to be opened to help us make this work. Any clue would be appreciated.

Thanks for your attention. Best Regards.

Andy Jefferson added a comment - 20/Aug/09 06:17 AM
Not sure what you have trouble with running the tests and what these "configuration files" are. Running the tests can't be simpler.
Build test.framework, test.samples (mvn clean install)
Run each scenario (mvn clean test).
That's it! There is no hidden config. They use an in-memory HSQLDB so you don't even set that up

Andre Luis Iasi Moura added a comment - 20/Aug/09 05:30 PM
Ok, we understand that for the DN test. The problem we are having is with the TCK, and is not that we can't run the tests, but is that, here, they are not passing as you said it should (or, at least, that was what we understood) . So we are considering the possibility that it is because of some configuration or something else we are not seeing (when we say about configuration, we are thinking mainly about project.xml - there are several possible combination of versions, and we have tested some of them, in addition, of course, to the direct tests of each versions - from 1.1.4 to 2.0.0-SNAPSHOT).

We are attaching the results from the test with 2.0.0-SNAPSHOT, and if you want, we can send the others as well.

Only one more consideration: the version 1.1.4 does not pass because there is at least one documented error that has been corrected in 1.1.5; but the version 1.1.5 (and the following ones) is jdo-2.3eb dependent, and we only have access to the TCK from 2.3.ea version.

Andre Luis Iasi Moura added a comment - 20/Aug/09 05:31 PM
TCK para 2.0.0-SNAPSHOT

Andre Luis Iasi Moura made changes - 20/Aug/09 05:31 PM
Attachment tck-result-2.0.0-SNAPSHOT.zip [ 11000 ]
Andre Luis Iasi Moura added a comment - 20/Aug/09 05:34 PM
The 1.1.4 error mentioned is reported here:

http://www.jpox.org/servlet/jira/browse/NUCRDBMS-200

Andy Jefferson added a comment - 20/Aug/09 05:47 PM
If you only have access to "the TCK for 2.3ea" why are you running 2.0.0-SNAPSHOT against it? If you use 2.0 you need to use JDO TCK from Apache JDO SVN.

I don't run against 1.1.x, not enough time. That branch is effectively finished for any serious development.

Renato Garcia made changes - 20/Aug/09 10:44 PM
Attachment TCK-after.zip [ 10993 ]
Renato Garcia made changes - 20/Aug/09 10:44 PM
Attachment TCK-before.zip [ 10992 ]
Renato Garcia made changes - 20/Aug/09 10:44 PM
Attachment unit-test-result.zip [ 10991 ]
Andre Luis Iasi Moura made changes - 20/Aug/09 10:46 PM
Attachment tck-result-2.0.0-SNAPSHOT.zip [ 11000 ]
Renato Garcia added a comment - 20/Aug/09 10:50 PM
TCK passed results after patch(patch.txt) applied.

Renato Garcia made changes - 20/Aug/09 10:50 PM
Attachment TCK-patch-passed.zip [ 11001 ]
Andre Luis Iasi Moura added a comment - 20/Aug/09 10:57 PM
See, I knew you could help... ;)

Renato Garcia added a comment - 20/Aug/09 11:18 PM
Here is the latest status I have for this:

- Patch for the issue attached (patch.txt)
- Patch for the "Transient instances cant be deleted" scenario attached (DependentFieldTest-patch.txt)
- Unit test results after patch applied, including the new scenario above attached (test-result.zip,testrun.out.zip)
- TCK passed results after patch applied (TCK-patch-passed.zip)

Andy, please let me know if there is anything else I can do in order to help.

Ps.:
I deleted the old attachments, so we keep only the relevant files avoiding confusion.

Andy Jefferson added a comment - 21/Aug/09 02:58 PM
SVN trunk now has this patch (removing the explicit javax.jdo call). JPA TCK passes. Thx

Andy Jefferson made changes - 21/Aug/09 02:58 PM
Status Open [ 1 ] Resolved [ 5 ]
Assignee Andy Jefferson [ andy ]
Fix Version/s 2.0.0.m2 [ 10702 ]
Resolution Fixed [ 1 ]
Andy Jefferson made changes - 11/Sep/09 08:40 AM
Status Resolved [ 5 ] Closed [ 6 ]
Andy Jefferson made changes - 04/Oct/12 05:02 PM
Component/s Persistence [ 10200 ]
Component/s JDO [ 10201 ]