DataNucleus JIRA is now in read-only mode. Raise any new issues in GitHub against the plugin that it applies to. DataNucleus JIRA will remain for the foreseeable future but will eventually be discontinued
Issue Details (XML | Word | Printable)

Key: NUCCORE-1103
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Dan Haywood
Votes: 0
Watchers: 0

If you were logged in you would be able to see more operations.
DataNucleus Core

Eager loading of multi-valued fields fails for collections of type java.util.SortedSet. This would seem to be due to a bug in SCOUtils.

Created: 10/Jan/14 12:22 PM   Updated: 14/Jan/14 11:22 AM   Resolved: 10/Jan/14 01:31 PM
Component/s: Queries
Affects Version/s: 3.2.11
Fix Version/s: 3.2.12

Severity: Development

 Description  « Hide
This is the code that tripped up.

    @javax.jdo.annotations.Persistent(mappedBy = "agreement", defaultFetchGroup="true")
    public SortedSet<AgreementRole> getRoles() {
        return roles;

where AgreementRole implements Comparable<AgreementRole>.

the problem we were getting is a ClassCastException:

Thread [1228951097@qtp-612169113-1] (Suspended (exception ClassCastException))
owns: ForwardQueryResult (id=101)
Lease(Agreement).jdoReplaceField(int) line: not available
Lease.jdoReplaceField(int) line: not available
JDOStateManagerForIsis(JDOStateManager).replaceField(PersistenceCapable, int, Object) line: 2206
JDOStateManagerForIsis(JDOStateManager).replaceField(PersistenceCapable, int, Object, boolean) line: 3360
JDOStateManagerForIsis(JDOStateManager).replaceField(int, Object) line: 3263
JDOStateManagerForIsis.replaceField(int, Object) line: 115
ForwardQueryResult.nextResultSetElement() line: 201
ForwardQueryResult$ line: 403
ForwardQueryResult.processNumberOfResults(int) line: 143
ForwardQueryResult.advanceToEndOfResultSet() line: 164
ForwardQueryResult.get(int) line: 496
ForwardQueryResult(AbstractQueryResult).subList(int, int) line: 380
PersistenceQueryFindUsingApplibQueryProcessor.getResults(PersistenceQueryFindUsingApplibQueryDefault) line: 110

It seems that DN is providing a HashSet for the field, rather than an object that implements SortedSet.

This issue doesn't arise in the previous version we were using, namely datanucleus-core-3.2.7 and datanucleus-rdbms-3.2.10. Now using AccessPlatform 3.3.6 (core 3.2.11, rdbms 3.2.10).

Looking at there now seems to be some new processing for bulk loading.

                            // Register any bulk loaded member resultSets that need loading
                            Map<String, IteratorStatement> scoIterStmts = datastoreCompilation.getSCOIteratorStatements();
                            if (scoIterStmts != null)
                                Iterator<Map.Entry<String, IteratorStatement>> scoStmtIter = scoIterStmts.entrySet().iterator();
                                while (scoStmtIter.hasNext())
                                    Map.Entry<String, IteratorStatement> stmtIterEntry =;
                                    IteratorStatement iterStmt = stmtIterEntry.getValue();
                                    String iterStmtSQL = iterStmt.getSQLStatement().getSelectStatement().toSQL();
                                    NucleusLogger.DATASTORE_RETRIEVE.debug(">> JDOQL Bulk-Fetch of " + iterStmt.getBackingStore().getOwnerMemberMetaData().getFullFieldName());
                                        PreparedStatement psSco = sqlControl.getStatementForQuery(mconn, iterStmtSQL);
                                        if (datastoreCompilation.getStatementParameters() != null)
                                            BulkFetchHelper helper = new BulkFetchHelper(this);
                                            helper.applyParametersToStatement(psSco, datastoreCompilation, iterStmt.getSQLStatement(), parameters);
                                        ResultSet rsSCO = sqlControl.executeStatementQuery(ec, mconn, iterStmtSQL, psSco);
                                        qr.registerMemberBulkResultSet(iterStmt, rsSCO);
                                    catch (SQLException e)
                                        throw new NucleusDataStoreException(LOCALISER.msg("056006", iterStmtSQL), e);

So, although in our domain code we had defaultFetchGroup="true", this was a no-op under 3.2.7, but now is implemented in 3.2.11.

The issue though seems to be that call to AbstractRDBMSQueryResultSet#registerMemberBulkResultSet(...). This calls AbstractRDBMSQueryResultSet#addOwnerMemberValue(...) which lazily populates the #bulkLoadedValueByMemberNumber hashmap. The value of that map is the collection, which is instantiated via SCOUtils#getContainerInstanceType.

And its getContainerInstanceType() that's returning the wrong type.

    public static Class getContainerInstanceType(Class declaredType, Boolean ordered)
        if (declaredType.isInterface())
            // Instantiate as ArrayList/HashSet/HashMap
            if (List.class.isAssignableFrom(declaredType))
                return ArrayList.class;
            else if (Set.class.isAssignableFrom(declaredType))
                return HashSet.class;
            else if (Map.class.isAssignableFrom(declaredType))
                return HashMap.class;
            else if (ordered)
                return ArrayList.class;
                return HashSet.class;
        return declaredType;

See also attached screenshot showing the stacktrace:

The fix, probably, is to fix up SCOUtils.

The workaround until this is fixed is to add:

        jdoQuery.addExtension("datanucleus.multivaluedFetch", "none");

for each query submitted. This basically disables the new functionality.

Andy Jefferson added a comment - 10/Jan/14 01:31 PM
GitHub master SCOUtils.getContainerInstanceType now allows for SortedSet/SortedMap interface container types

Andy Jefferson made changes - 10/Jan/14 01:31 PM
Field Original Value New Value
Status Open [ 1 ] Resolved [ 5 ]
Fix Version/s 3.2.12 [ 12074 ]
Resolution Fixed [ 1 ]
Andy Jefferson made changes - 14/Jan/14 11:22 AM
Status Resolved [ 5 ] Closed [ 6 ]