Currently all objects are loaded into memory and then resulting list is trimmed. For small collections, this is not a problem, but for larger result sets this significantly slows things down and leads to OutOfMemoryErrors.
A simple fix is to change getObjectsOfCandidateType() to take a skip/limit paramter and to use these to set the skip and limit on the resulting cursor. A problem with this is that it will not return the correct results for a query with an ORDER BY clause. This can be addressed by modifying JPQLQuery.java and JDOQLQuery.java to not pass the skip and limit parameters when there is an ORDER BY clause.
Alternatively (or perhaps in the future) it may be possible to translate simple ORDER BY clauses to a DBObject that can then be passed to the sort() method of the DBCursor if the ordering were passed to the getObjectsOfCandidateType() method in MongoDBUtils.java. Of course, this can't be used if the sort requires a join, so it would have to fall back to the current in-memory handling for those cases I would think.
Patch to follow...