Issue Details (XML | Word | Printable)

Key: NUCCORE-1103
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Dan Haywood
Votes: 0
Watchers: 0

If you were logged in you would be able to see more operations.
DataNucleus Core

Eager loading of multi-valued fields fails for collections of type java.util.SortedSet. This would seem to be due to a bug in SCOUtils.

Created: 10/Jan/14 12:22 PM   Updated: 14/Jan/14 11:22 AM   Resolved: 10/Jan/14 01:31 PM
Component/s: Queries
Affects Version/s: 3.2.11
Fix Version/s: 3.2.12

Severity: Development

 Description  « Hide
This is the code that tripped up.

    @javax.jdo.annotations.Persistent(mappedBy = "agreement", defaultFetchGroup="true")
    public SortedSet<AgreementRole> getRoles() {
        return roles;

where AgreementRole implements Comparable<AgreementRole>.

the problem we were getting is a ClassCastException:

Thread [1228951097@qtp-612169113-1] (Suspended (exception ClassCastException))
owns: ForwardQueryResult (id=101)
Lease(Agreement).jdoReplaceField(int) line: not available
Lease.jdoReplaceField(int) line: not available
JDOStateManagerForIsis(JDOStateManager).replaceField(PersistenceCapable, int, Object) line: 2206
JDOStateManagerForIsis(JDOStateManager).replaceField(PersistenceCapable, int, Object, boolean) line: 3360
JDOStateManagerForIsis(JDOStateManager).replaceField(int, Object) line: 3263
JDOStateManagerForIsis.replaceField(int, Object) line: 115
ForwardQueryResult.nextResultSetElement() line: 201
ForwardQueryResult$ line: 403
ForwardQueryResult.processNumberOfResults(int) line: 143
ForwardQueryResult.advanceToEndOfResultSet() line: 164
ForwardQueryResult.get(int) line: 496
ForwardQueryResult(AbstractQueryResult).subList(int, int) line: 380
PersistenceQueryFindUsingApplibQueryProcessor.getResults(PersistenceQueryFindUsingApplibQueryDefault) line: 110

It seems that DN is providing a HashSet for the field, rather than an object that implements SortedSet.

This issue doesn't arise in the previous version we were using, namely datanucleus-core-3.2.7 and datanucleus-rdbms-3.2.10. Now using AccessPlatform 3.3.6 (core 3.2.11, rdbms 3.2.10).

Looking at there now seems to be some new processing for bulk loading.

                            // Register any bulk loaded member resultSets that need loading
                            Map<String, IteratorStatement> scoIterStmts = datastoreCompilation.getSCOIteratorStatements();
                            if (scoIterStmts != null)
                                Iterator<Map.Entry<String, IteratorStatement>> scoStmtIter = scoIterStmts.entrySet().iterator();
                                while (scoStmtIter.hasNext())
                                    Map.Entry<String, IteratorStatement> stmtIterEntry =;
                                    IteratorStatement iterStmt = stmtIterEntry.getValue();
                                    String iterStmtSQL = iterStmt.getSQLStatement().getSelectStatement().toSQL();
                                    NucleusLogger.DATASTORE_RETRIEVE.debug(">> JDOQL Bulk-Fetch of " + iterStmt.getBackingStore().getOwnerMemberMetaData().getFullFieldName());
                                        PreparedStatement psSco = sqlControl.getStatementForQuery(mconn, iterStmtSQL);
                                        if (datastoreCompilation.getStatementParameters() != null)
                                            BulkFetchHelper helper = new BulkFetchHelper(this);
                                            helper.applyParametersToStatement(psSco, datastoreCompilation, iterStmt.getSQLStatement(), parameters);
                                        ResultSet rsSCO = sqlControl.executeStatementQuery(ec, mconn, iterStmtSQL, psSco);
                                        qr.registerMemberBulkResultSet(iterStmt, rsSCO);
                                    catch (SQLException e)
                                        throw new NucleusDataStoreException(LOCALISER.msg("056006", iterStmtSQL), e);

So, although in our domain code we had defaultFetchGroup="true", this was a no-op under 3.2.7, but now is implemented in 3.2.11.

The issue though seems to be that call to AbstractRDBMSQueryResultSet#registerMemberBulkResultSet(...). This calls AbstractRDBMSQueryResultSet#addOwnerMemberValue(...) which lazily populates the #bulkLoadedValueByMemberNumber hashmap. The value of that map is the collection, which is instantiated via SCOUtils#getContainerInstanceType.

And its getContainerInstanceType() that's returning the wrong type.

    public static Class getContainerInstanceType(Class declaredType, Boolean ordered)
        if (declaredType.isInterface())
            // Instantiate as ArrayList/HashSet/HashMap
            if (List.class.isAssignableFrom(declaredType))
                return ArrayList.class;
            else if (Set.class.isAssignableFrom(declaredType))
                return HashSet.class;
            else if (Map.class.isAssignableFrom(declaredType))
                return HashMap.class;
            else if (ordered)
                return ArrayList.class;
                return HashSet.class;
        return declaredType;

See also attached screenshot showing the stacktrace:

The fix, probably, is to fix up SCOUtils.

The workaround until this is fixed is to add:

        jdoQuery.addExtension("datanucleus.multivaluedFetch", "none");

for each query submitted. This basically disables the new functionality.

Sort Order: Ascending order - Click to sort in descending order
Andy Jefferson added a comment - 10/Jan/14 01:31 PM
GitHub master SCOUtils.getContainerInstanceType now allows for SortedSet/SortedMap interface container types