DataNucleus JIRA is now in read-only mode. Raise any new issues in GitHub against the plugin that it applies to. DataNucleus JIRA will remain for the foreseeable future but will eventually be discontinued
Issue Details (XML | Word | Printable)

Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Chris Rued
Votes: 0
Watchers: 0

If you were logged in you would be able to see more operations.
DataNucleus Store MongoDB

Add native support for query range using skip/limit

Created: 14/Jun/12 08:39 PM   Updated: 25/Jun/12 02:44 PM   Resolved: 15/Jun/12 09:21 AM
Component/s: Query
Affects Version/s: None
Fix Version/s: 3.1.0.m4

File Attachments: 1. Text File vcs-diff2495926650834641550.patch (4 kB)

 Description  « Hide
Currently all objects are loaded into memory and then resulting list is trimmed. For small collections, this is not a problem, but for larger result sets this significantly slows things down and leads to OutOfMemoryErrors.

A simple fix is to change getObjectsOfCandidateType() to take a skip/limit paramter and to use these to set the skip and limit on the resulting cursor. A problem with this is that it will not return the correct results for a query with an ORDER BY clause. This can be addressed by modifying and to not pass the skip and limit parameters when there is an ORDER BY clause.

Alternatively (or perhaps in the future) it may be possible to translate simple ORDER BY clauses to a DBObject that can then be passed to the sort() method of the DBCursor if the ordering were passed to the getObjectsOfCandidateType() method in Of course, this can't be used if the sort requires a join, so it would have to fall back to the current in-memory handling for those cases I would think.

Patch to follow...

Sort Order: Ascending order - Click to sort in descending order
Chris Rued added a comment - 14/Jun/12 09:08 PM
Patch to implement skip/limit in MONGODB when there is no ordering specified

Andy Jefferson added a comment - 15/Jun/12 09:01 AM
A side comment to the issue you mention ... and the real solution is to enable use of MongoDBCandidateList, passing the DBCursor to that and it lazily loading any results, which is why it is there, just not enabled.

Andy Jefferson added a comment - 15/Jun/12 09:21 AM
SVN trunk has this patch (after formatting), and TODO for the real work in NUCMONGODB-81. Thx

Andy Jefferson added a comment - 25/Jun/12 02:44 PM
Obviously the implementation of this is flawed where there are multiple candidate types (i.e subclasses which aren't stored in the same MongoDB document).