Issue Details (XML | Word | Printable)

Key: NUCCORE-545
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Andy Jefferson
Reporter: Andy Jefferson
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
DataNucleus Core

Develop type-safe query mechanism primarily for JDO (3.1)

Created: 19/Jun/10 06:39 AM   Updated: 13/Nov/10 12:17 PM   Resolved: 07/Nov/10 08:47 AM
Component/s: Queries
Affects Version/s: None
Fix Version/s: 2.2.0.m3


 Description  « Hide
JDO requires a type-safe query mechanism, avoiding the need to embody strings in queries. The initial "implementation" can be JDOQL, but we could develop a JPQL "implementation" also. The API should be fluent, like with QueryDSL

Sort Order: Ascending order - Click to sort in descending order
Timo Westkämper added a comment - 19/Jun/10 08:54 AM - edited
For Querydsl we chose the following approach for the core / implementation separation

core
* dynamic expression model
* basic query interfaces
* basic serialization for toString() of expression instances

implementation (e.g. JDOQL, JPQL etc)
* extended query interface
* query implementation
* query conversion / serialization

This could also work for this case. The expression model of Querydsl is currently class based, but could easily be changed into an interface based form.

For both JDOQL and JPQL APT was used for mapping the domain model to a query model.

Concerning the dynamic vs static metamodel approach and JPA 2 Criteria readability see here :
* http://rjewalker.blogspot.com/2010/06/querydsl-vs-jpa-20-criteria-api-first.html
* http://luisfpg.blogspot.com/2010/05/jpa-2-criteria.html

One approach to get started could be to get to know
* Querydsl Core
* Querydsl APT
* Querydsl JDOQL

and then think how the Querydsl JDOQL module could be implemented as part of DN/JDO and how some parts of Querydsl Core could be generalized into maybe a common project.

DN and Querydsl sharing a common query model would be great.


Andy Jefferson added a comment - 22/Jun/10 09:25 AM
An AnnotationProcessor (standard in JDK 1.6) is a requirement certainly, but does not affect the API design (other than defining requirements of what fields are public and what are their names). We already have an implementation of such for JPQL (to JPA2 spec requirements).

As far as the API design, there will need to be an expression model (generic), and basic query interface (like with javax.jdo.Query method names), and then build on top of that a JDOQL query interface. The requirement to have toString() to return the single-string form of the query would also be nice to have (and something that DN JPQL Criteria already does, although not in the JPA spec).

Provisional package names
org.datanucleus.query.typesafe
org.datanucleus.query.typesafe.jdoql


The actual implementation is not part of the JDO spec, so is not the focus here, though DataNucleus would certainly be providing one (in our case we would provide an implementation that generated the org.datanucleus.query.compiler.QueryCompilation generic query compilation. This is then in the form to be executed (converted into a datastore-specific compilation (e.g SQL), and run against the datastore).

Andy Jefferson added a comment - 01/Jul/10 02:22 PM
DataNucleus "core" SVN trunk has a series of expressions and a Query interface. These are not intended to be fully-thought-out expressions catering for all JDOQL syntax, nor the most type-safe yet, just there for discussion.

Particular areas :-

Accessing candidate, parameter, variable
========================================
I've added methods on to Query to access these.
QPerson cand = (QPerson)q.candidate();
QAccount param1 = (QAccount)q.parameter("myAccount", Account.class);
QDepartment var1 = (QDepartment)q.variable("myDeptVar", Department.class);
Don't like the need to cast, but at least it brings the notion of parameters, variables, and candidates ... integral part of JDOQL.


Alternatively could do like QueryDSL
QPerson cand = QPerson.person;
QAccount param1 = new QAccount("myAccount");
QDepartment var1 = new QDepartment("myDeptVar");
but no concept of parameters, variables.


Could add a static method on the generated Q class to return parameter or variable type, like this
QPerson cand = QPerson.candidate;
QAccount param1 = QAccount.parameter("myAccount");
Qdepartment var1 = QDepartment.variable("myDeptVar");
which is my favoured right now.


Need a definition of what methods a "Q" class has.
Is there a QueryDSL document that defines this ?


Expression methods
==================
I've simply put String/Date/Numeric methods under the expression type for now whether or not they are only in JDOQL or not. Will split out into the different query language specifics later on.


Ordering
========
To generate an ordering clause, just call the Expression.asc()/desc() which returns an OrderExpression.
qp.asc();
qp.desc();


Aggregates
==========
Could do as max(), count(), sum(), etc methods on the Expression interface (like the ordering is done), so
qp.max();

Andy Jefferson added a comment - 01/Jul/10 02:40 PM
Something else missing from above about params/vars is where the type is not a persistable type e.g String. Option1 would still work ok, but still the need to cast. Option2 ... presumably would be
StringExpression param1 = new StringExpressionImpl("myParam");
StringExpression var1 = new StringExpressionImpl("myVar");
Rule this out since we cannot have people coding to a particular implementations classes.

Option3 ... would need some way of creating params/vars

Timo Westkämper added a comment - 01/Jul/10 03:58 PM
> Don't like the need to cast, but at least it brings the notion of parameters, variables, and candidates ... integral part of JDOQL.

Get rid of the casts and formalize the approach with parameters, variables and candidates.

You should go further with the approach and see where it goes.

e.g.

Candidate<Person> cand = q.candidate();
Parameter<Account> param1 = q.parameter("myAccount", Account.class);
Variable<Department> var1 = q.variable("myDeptVar", Department.class);

Now you would need to access the properties JPA 2 Criteria style.

If you use a Querydsl-like approach then the factory approach for creation is difficult.

> Could add a static method on the generated Q class to return parameter or variable type, like this
>Need a definition of what methods a "Q" class has.
>Is there a QueryDSL document that defines this ?

A generated Q-class is guaranteed to be a subclass of PEntity (entity path). For Querydsl JDOQL it can act as a candidate and variable.

Beside that a Q-class has the properties of the related entity class and all its mapped superclasses. The properties are available as fields. For types with mapped supertypes there is a field called _super with a reference to the path itself with the supertype query signature.

e.g.
    public class Animal {}

    public class Cat extends Animal{}

    public class QAnimal extends PEntity<Animal> {

    }

    public class QCat extends PEntity<Cat> {
        public final QAnimal _super = new QAnimal(this);

    }

For Q-types related to entity types there is also a field for the default variable.

e.g.

    public class QCat extends PEntity<Cat> {
        public static final QCat cat = new QCat("cat");

    }

The Q-types are kept without any generic type parameters for ease of use. Casts into both directions are possible like this :

    QAnimal animal = QAnimal.animal;
    QCat cat = QCat.cat;

    // Cat to Animal
    animal = cat._super;

    // Animal to Cat
    cat = new QCat(animal);


Timo Westkämper added a comment - 01/Jul/10 04:09 PM
Do parameters need property access which is the main point of Q-types? If not, then drop Q-type usage for parameters.

Do you need different creation styles for candidates and variables? What are the key differences beside there different role in the query?

The Querydsl translation of the following

QPerson cand = QPerson.candidate;
QAccount param1 = QAccount.parameter("myAccount");
QDepartment var1 = QDepartment.variable("myDeptVar");

is

QPerson can = QPerson.person; // person is also the variable name and the result of toString
Param<Account> param1 = new Param<Account>(Account.class, "myAccount"); // a generic approach is ok here, since literal parameters need to be supported as well
QDepartment var1 = new QDepartment("myDeptVar");


Andy Jefferson added a comment - 01/Jul/10 04:25 PM
Parameters (and variables) need property access. JDOQL is Java. A parameter could be a "Person" object. A parameter is something input by the user. A variable is something relative to the candidate (like the components of the FROM in a JPQL query ... e.g an element of a Collection).

Example of a variable
JDOQL : SELECT FROM Person WHERE this.friends.contains(varFriend) && varFriend.name = "Fred"

"varFriend" is an element of the "friends" field of Person.

Timo Westkämper added a comment - 01/Jul/10 04:31 PM
Thanks for the clarifications. At the moment Querydsl doesn't support property access for parameters, but might in the future.

Andy Jefferson added a comment - 11/Jul/10 10:26 AM

Andy Jefferson added a comment - 28/Oct/10 06:54 PM
JDO Query Annotation Processor now moved into separate plugin
"jdo.query" with jar "datanucleus-jdo-query" for convenience of use with Eclipse etc.

Andy Jefferson added a comment - 03/Nov/10 04:32 PM
SVN trunk now has parameters, and variables implemented and tested.
Only remaining aspect to define and implement is subqueries. Since a subquery can have methods invoked on it, or can represent a long, integer, string etc then it has to be any type of expression (including persistable). Consequently we just add another "type" to the expression and a method "isSubquery()" to Expression

Andy Jefferson added a comment - 05/Nov/10 10:22 AM
Actually had rethink on subqueries, and what is likely best is as follows

TypesafeQuery q = pm.newTypesafeQuery(Inventory.class);
q.filter(
    value.lt(
        q.subquery(Product.class, "p").filter(myFilter).result(myResult)
    )
);

In terms of the API, TypesafeQuery.subquery() will return a TypesafeSubquery. This has the methods applicable to a subquery, and then to link it back to the main query the method "result()" defines what type of expression the subquery is (numeric, string, persistable etc) and so that returns an Expression (cast to the relevant type if needing to perform method calls on the subquery expression).

Andy Jefferson added a comment - 07/Nov/10 08:47 AM
Now has basic subqueries too. Marking as complete, and can be used as basis for discussion for getting something into JDO3.1