Issue Details (XML | Word | Printable)

Key: NUCRDBMS-379
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Andy Jefferson
Reporter: Marco Schulze
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
DataNucleus Store RDBMS

Map value is null, even though there is an object in the database

Created: 28/Apr/10 07:43 PM   Updated: 24/May/10 03:43 PM   Resolved: 29/Apr/10 03:18 PM
Component/s: None
Affects Version/s: 2.1.0.m2
Fix Version/s: 2.1.0.m3

Datastore: Apache Derby
Severity: Development


 Description  « Hide
Hello Andy,

I'm still having a problem with the combination DataNucleus 2.1.0-m2 and Derby.

Here the scenario in short:

class VoucherLocalAccountantDelegate extends LocalAccountantDelegate
{
  Map<String, Account> accounts; // The key is the ISO currency code.
  // ...
}

class Anchor {
  // ...
}

class Account extends Anchor
{
  Currency currency;
  LegalEntity owner;
  Set<SummaryAccount> summaryAccounts;
  // ...
}

class SummaryAccount extends Account
{
  Set<Account> summedAccounts;
  // ...
}

class LegalEntity extends Anchor
{
  // ...
}

I create an instance of Account and put it into the Map VoucherLocalAccountantDelegate.accounts. In a following transaction, I want to obtain this Account again, but VoucherLocalAccountantDelegate.accounts.get("EUR") returns null. In the debugger, I see that the backing Map contains one entry "EUR"=>null, but in the database, there definitely is an Account referenced for the key "EUR".

Btw. if I use MySQL, it works fine - the error only occurs with Derby. And with Derby, it works fine, too, if I set the fetch-group ALL.

Here's a test: http://www.nightlabs.de/~marco/datanucleus/2010-04-28.00/

Reproducing this issue outside of JFire was pretty tricky and took me quite a while ;-) Even after copying half the data model, it didn't happen, thus I copied more and more classes from JFire until I was finally able to reproduce it with over 30 classes in my test case :-(

Fortunately, I was then able to remove most of these classes again and there are now only 7 left. Please forgive me, if there are still unnecessary classes in this test. I simplified it a lot, but not knowing exactly what causes the problem made it really hard to remove the right things (and still cause the error).

Best regards, Marco :-)

Sort Order: Ascending order - Click to sort in descending order
Marco Schulze added a comment - 28/Apr/10 08:01 PM
I was able to simplify it even further - I think this is now pretty much the minimal scenario necessary for reproducing the error.

Here's the new test: http://www.nightlabs.de/~marco/datanucleus/2010-04-28.01/

Andy Jefferson added a comment - 29/Apr/10 12:04 PM
Derby is somehow buggy on the following query

SELECT 'org.datanucleus.test.Account ' AS NUCLEUS_TYPE,A2.ANCHORID,A2.ANCHORTYPEID,A2.ORGANISATIONID,A0.BALANCE
FROM JFIRETRADE_ACCOUNT A0
LEFT OUTER JOIN JFIRETRADE_SUMMARYACCOUNT A1 ON A0.ANCHORID = A1.ANCHORID AND A0.ANCHORTYPEID = A1.ANCHORTYPEID AND A0.ORGANISATIONID = A1.ORGANISATIONID
INNER JOIN JFIREVOUCHER_VOUCHERLOCALACCOUNTANTDELEGATE_ACCOUNTS B0 ON A0.ANCHORID = B0.ANCHORID_VID AND A0.ANCHORTYPEID = B0.ANCHORTYPEID_VID AND A0.ORGANISATIONID = B0.ORGANISATIONID_VID
INNER JOIN JFIRETRADE_ANCHOR A2 ON A0.ANCHORID = A2.ANCHORID AND A0.ANCHORTYPEID = A2.ANCHORTYPEID AND A0.ORGANISATIONID = A2.ORGANISATIONID
WHERE A1.ANCHORID IS NULL
AND A1.ANCHORTYPEID IS NULL
AND A1.ORGANISATIONID IS NULL
AND B0.LOCALACCOUNTANTDELEGATEID_OID = <'0'>
AND B0.ORGANISATIONID_OID = <'jfire.my.org'>

UNION

SELECT 'org.datanucleus.test.SummaryAccount' AS NUCLEUS_TYPE,A2.ANCHORID,A2.ANCHORTYPEID,A2.ORGANISATIONID,A0.BALANCE
FROM JFIRETRADE_ACCOUNT A0
INNER JOIN JFIRETRADE_SUMMARYACCOUNT A1 ON A0.ANCHORID = A1.ANCHORID AND A0.ANCHORTYPEID = A1.ANCHORTYPEID AND A0.ORGANISATIONID = A1.ORGANISATIONID
INNER JOIN JFIREVOUCHER_VOUCHERLOCALACCOUNTANTDELEGATE_ACCOUNTS B0 ON A0.ANCHORID = B0.ANCHORID_VID AND A0.ANCHORTYPEID = B0.ANCHORTYPEID_VID AND A0.ORGANISATIONID = B0.ORGANISATIONID_VID
INNER JOIN JFIRETRADE_ANCHOR A2 ON A0.ANCHORID = A2.ANCHORID AND A0.ANCHORTYPEID = A2.ANCHORTYPEID AND A0.ORGANISATIONID = A2.ORGANISATIONID
WHERE B0.LOCALACCOUNTANTDELEGATEID_OID = <'0'>
AND B0.ORGANISATIONID_OID = <'jfire.my.org'>

which returns nothing.

When you execute the same thing with MySQL (or HSQLDB too for that matter)
SELECT 'org.datanucleus.test.Account ' AS NUCLEUS_TYPE,`A2`.`ANCHORID`,`A2`.`ANCHORTYPEID`,`A2`.`ORGANISATIONID`,`A0`.`BALANCE`
FROM `JFIRETRADE_ACCOUNT` `A0`
LEFT OUTER JOIN `JFIRETRADE_SUMMARYACCOUNT` `A1` ON `A0`.`ANCHORID` = `A1`.`ANCHORID` AND `A0`.`ANCHORTYPEID` = `A1`.`ANCHORTYPEID` AND `A0`.`ORGANISATIONID` = `A1`.`ORGANISATIONID`
INNER JOIN `JFIREVOUCHER_VOUCHERLOCALACCOUNTANTDELEGATE_ACCOUNTS` `B0` ON `A0`.`ANCHORID` = `B0`.`ANCHORID_VID` AND `A0`.`ANCHORTYPEID` = `B0`.`ANCHORTYPEID_VID` AND `A0`.`ORGANISATIONID` = `B0`.`ORGANISATIONID_VID`
INNER JOIN `JFIRETRADE_ANCHOR` `A2` ON `A0`.`ANCHORID` = `A2`.`ANCHORID` AND `A0`.`ANCHORTYPEID` = `A2`.`ANCHORTYPEID` AND `A0`.`ORGANISATIONID` = `A2`.`ORGANISATIONID`
WHERE `A1`.`ANCHORID` IS NULL
AND `A1`.`ANCHORTYPEID` IS NULL
AND `A1`.`ORGANISATIONID` IS NULL
AND `B0`.`LOCALACCOUNTANTDELEGATEID_OID` = <'0'>
AND `B0`.`ORGANISATIONID_OID` = <'jfire.my.org'>

UNION

SELECT 'org.datanucleus.test.SummaryAccount' AS NUCLEUS_TYPE,`A2`.`ANCHORID`,`A2`.`ANCHORTYPEID`,`A2`.`ORGANISATIONID`,`A0`.`BALANCE`
FROM `JFIRETRADE_ACCOUNT` `A0`
INNER JOIN `JFIRETRADE_SUMMARYACCOUNT` `A1` ON `A0`.`ANCHORID` = `A1`.`ANCHORID` AND `A0`.`ANCHORTYPEID` = `A1`.`ANCHORTYPEID` AND `A0`.`ORGANISATIONID` = `A1`.`ORGANISATIONID`
INNER JOIN `JFIREVOUCHER_VOUCHERLOCALACCOUNTANTDELEGATE_ACCOUNTS` `B0` ON `A0`.`ANCHORID` = `B0`.`ANCHORID_VID` AND `A0`.`ANCHORTYPEID` = `B0`.`ANCHORTYPEID_VID` AND `A0`.`ORGANISATIONID` = `B0`.`ORGANISATIONID_VID`
INNER JOIN `JFIRETRADE_ANCHOR` `A2` ON `A0`.`ANCHORID` = `A2`.`ANCHORID` AND `A0`.`ANCHORTYPEID` = `A2`.`ANCHORTYPEID` AND `A0`.`ORGANISATIONID` = `A2`.`ORGANISATIONID`
WHERE `B0`.`LOCALACCOUNTANTDELEGATEID_OID` = <'0'>
AND `B0`.`ORGANISATIONID_OID` = <'jfire.my.org'>

(i.e the same query, other than quoted identifiers which MySQL requires) this returns the single entry to the map. Rather than using get() you could use Map.values() and this would go straight to this query.

Try it in a Derby console and see if you can find what it has a problem with

Marco Schulze added a comment - 29/Apr/10 03:06 PM
Hi Andy,

thanks a lot for all your help and support! I decided to workaround this problem via refreshing the object VoucherLocalAccountantDelegate with fetch-group ALL for now (fortunately I found this workaround while writing the test for this issue). I've unfortunately no time to follow this issue further, because I've a new project starting tomorrow (and I have to prepare things today).

For other people suffering the same issue, here's the code I used in VoucherLocalAccountantDelegate:

private final void derbyWorkaround()
{
PersistenceManager pm = JDOHelper.getPersistenceManager(this);
if (pm == null)
return;

FetchPlanBackup fetchPlanBackup = NLJDOHelper.backupFetchPlan(pm.getFetchPlan());
try {
pm.getFetchPlan().setGroup(FetchPlan.ALL);
pm.refresh(this);
this.accounts.values();
} finally {
NLJDOHelper.restoreFetchPlan(pm.getFetchPlan(), fetchPlanBackup);
}
}

The affected class's source code is here: https://svn.jfire.org/svn/main/trunk/JFireVoucher/src/org/nightlabs/jfire/voucher/accounting/VoucherLocalAccountantDelegate.java

Fortunately, JFire seems to work fine now with DataNucleus 2.1.0-m3-SNAPSHOT built 2010-04-29.

Shall I close this issue with resolution "Won't fix"? I think there's not much you can do :-(

IMHO we should open an issue in the Derby JIRA. Did you already do this? Otherwise I could do it (probably this week-end).

Best regards, Marco :-)

Andy Jefferson added a comment - 29/Apr/10 03:18 PM
I put a change in SCOUtils to just take the entries query key/value pairs (which works on Derby), and log a warning if the keys/values query returns a different number of rows from the entries query. This means that the test runs anyway without your extra code; obviously Derby bugs may crop up somewhere else.

If you could raise the issue on Derby please, when you have time, since its your schema and you know what data is in there.