Issue Details (XML | Word | Printable)

Key: NUCRDBMS-391
Type: Bug Bug
Status: Closed Closed
Resolution: Won't Fix
Priority: Major Major
Assignee: Unassigned
Reporter: Areg Beketovski
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
DataNucleus Store RDBMS

Scroll insensitive result set throws exception upon iteration when caching is disabled

Created: 14/May/10 05:49 PM   Updated: 10/Dec/10 07:49 AM   Resolved: 29/Nov/10 03:01 PM
Component/s: Queries
Affects Version/s: 2.1.0.m2
Fix Version/s: None

File Attachments: 1. Zip Archive testcase-391.zip (2 kB)


Forum Thread URL: http://www.datanucleus.org/servlet/forum/viewthread_thread,6129
Datastore: Microsoft SQL Server


 Description  « Hide
The following query configuration throws "The result set is closed." exception on the first call of "Iterator.next()" for the returned Collection:

query.addExtension("datanucleus.rdbms.query.resultSetType", "scroll-insensitive");
query.addExtension("datanucleus.query.resultCacheType", "none");


When such a query is not executed in a transaction by default all the returned results are cached upfront and aside consuming lots of memory looks like might get garbage collected if iteration takes a long time. This leads to another "The result set is closed." exception once underlying ScrollableQueryResult,getObjectForIndex() is called.

Areg Beketovski added a comment - 14/May/10 05:55 PM
For this particular case holding on the ManagedConnection instance from org.datanucleus.store.rdbms.query.JDOQL.performExecute(), skipping "finally {mconn.release();}" and delaying release of the connection until closeAll() allows the application to proceed successfully for 10000+ returned objects.

Andy Jefferson added a comment - 17/May/10 09:09 AM
Obviously leaving query connections open is bad practice, and prone to disaster, hence why current non-tx handling will always close them. Yes it impacts on memory utilisation, but then you decided to run it outside a tx (which imposes minimal overhead anyway, and gives other benefits).

Provide a testcase

Areg Beketovski added a comment - 17/May/10 07:03 PM
Please see attached the testcase demonstrating the issue. Testcase was run against 2.1.0-m3-SNAPSHOT jars from 17-May-2010 02:32.

In our case this is used for batch processing during analytics. The input to this process is a large number (unbounded) of Entities like Accounts or Memberships. A Result is generated for each Entity analyzed and is normally persisted immediately upon creation in a very short transaction. The advantage of this approach is that it allows incremental recovery of what could be a very long process. A single transaction is total progress or total failure, which is not working in our case.

Areg Beketovski made changes - 17/May/10 07:03 PM
Field Original Value New Value
Attachment testcase-391.zip [ 11171 ]
Andy Jefferson added a comment - 29/Nov/10 03:01 PM
Use PMF property "datanucleus.connection.nontx.releaseAfterUse" as false if you don't want the connection to be handed back to the pool after the non-xt operation completes. Testcase passes if you do that

Andy Jefferson made changes - 29/Nov/10 03:01 PM
Status Open [ 1 ] Resolved [ 5 ]
Resolution Won't Fix [ 2 ]
Andy Jefferson made changes - 10/Dec/10 07:49 AM
Status Resolved [ 5 ] Closed [ 6 ]