JDO : Value generation

Fields of a class can either have the values set by you the user, or you can set DataNucleus to generate them for you. This is of particular importance with identity fields where you want unique identities. You can use this value generation process with any field in JDO. There are many different "strategies" for generating values, as defined by the JDO specifications, and also some DataNucleus extensions. Some strategies are specific to a particular datastore, and some are generic. You should choose the strategy that best suits your target datastore. The available strategies for JDO are :-

  • native - this is the default and allows DataNucleus to choose the most suitable for the datastore
  • sequence - this uses a datastore sequence (if supported by the datastore)
  • identity - these use autoincrement/identity/serial features in the datastore (if supported by the datastore)
  • increment - this is datastore neutral and increments a sequence value using a table.
  • uuid-string - this is a UUID in string form
  • uuid-hex - this is a UUID in hexadecimal form
  • uuid - provides a pure UUID utilising the JDK1.5 UUID class
  • auid - provides a pure UUID following the OpenGroup standard
  • timestamp - creates a java.sql.Timestamp of the current time
  • timestamp-value - creates a long (millisecs) of the current time
  • max - uses a max(column)+1 method (only in RDBMS)
  • datastore-uuid-hex - UUID in hexadecimal form using datastore capabilities (only in RDBMS)
  • user-supplied value generators - allows you to hook in your own identity generator

See also :-



Please note that by defining a value-strategy for a field then it will, by default, always generate a value for that field on persist. If the field can store nulls and you only want it to generate the value at persist when it is null (i.e you haven't assigned a value yourself) then you can add the extension "strategy-when-notnull" as false


native

With this strategy DataNucleus will choose the most appropriate strategy for the datastore being used. If you also specify the 'sequence' name attribute and the datastore supports sequences then "sequence" strategy would be used. Otherwise it will always choose "increment" strategy.

sequence

A sequence is a user-defined database function that generates a sequence of unique numeric ids. The unique identifier value returned from the database is translated to a java type: java.lang.Long. DataNucleus supports sequences for the following datastores:

  • Oracle
  • PostgreSQL
  • SAP DB
  • DB2
  • Firebird
  • HSQLDB
  • H2
  • Derby (from v10.6)
  • DB4O

To configure a class to use either of these generation methods with datastore identity you simply add this to the class' Meta-Data 

<class name="myclass" ... >
    <datastore-identity strategy="sequence" sequence="yourseq"/>
    ...
    <sequence name="yourseq" datastore-sequence="YOUR_SEQUENCE_NAME"/>
</class>

or using annotations 

@PersistenceCapable
@DatastoreIdentity(strategy="sequence", sequence="yourseq"/>
@Sequence(name="yourseq", datastore-sequence="YOUR_SEQUENCE_NAME"/>
public class MyClass

You replace "YOUR_SEQUENCE_NAME" with your sequence name. To configure a class to use either of these generation methods using application identity you would add the following to the class' Meta-Data 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="sequence" sequence="yourseq"/>
    ...
    <sequence name="yourseq" datastore-sequence="YOUR_SEQUENCE_NAME"/>
</class>

or using annotations 

@PersistenceCapable
@Sequence(name="yourseq", datastore-sequence="YOUR_SEQUENCE_NAME"/>
public class MyClass
{
    @Persistent(valueStrategy="sequence", sequence="yourseq"/>
    private long myfield;
    ...
}

If the sequence does not yet exist in the database at the time DataNucleus needs a new unique identifier, a new sequence is created in the database based on the JDO Meta-Data configuration. Additional properties for configuring sequences are set in the JDO Meta-Data, see the available properties below. Unsupported properties by a database are silently ignored by DataNucleus.

Property Description Required
key-initial-value the initial value for the sequence. In JDO3.1 this is specified in the standard metadata ( initialValue ) No
key-cache-size number of unique identifiers to cache in the PersistenceManagerFactory instance. Notes:
  1. The keys are pre-allocated, cached and used on demand. If key-cache-size is greater than 1, it may generate holes in the object keys in the database, if not all keys are used.
In JDO3.1 this is specified in the standard metadata ( allocationSize ) 		No.
key-min-value determines the minimum value a sequence can generate No
key-max-value determines the maximum value a sequence can generate No
key-database-cache-size specifies how many sequence numbers are to be preallocated and stored in memory for faster access. This is an optimization feature provided by the database No
sequence-catalog-name Name of the catalog where the sequence is. No.
sequence-schema-name Name of the schema where the sequence is. No.

This value generator will generate values unique across different JVMs



identity

Auto-increment/identity/serial are primary key columns that are populated when a row is inserted in the table. These use the databases own keywords on table creation and so rely on having the table structure either created by DataNucleus or having the column with the necessary keyword.

DataNucleus supports auto-increment/identity/serial keys for many databases including :

  • DB2 (IDENTITY)
  • MySQL (AUTOINCREMENT)
  • MSSQL (IDENTITY)
  • Sybase (IDENTITY)
  • HSQLDB (IDENTITY)
  • H2 (IDENTITY)
  • PostgreSQL (SERIAL)

This generation strategy should only be used if there is a single "root" table for the inheritance tree. If you have more than 1 root table (e.g using subclass-table inheritance) then you should choose a different generation strategy

For a class using datastore identity you need to set the strategy attribute. You can configure the Meta-Data for the class something like this (replacing 'myclass' with your class name) : 

<class name="myclass">
    <datastore-identity strategy="identity"/>
    ...
</class>

For a class using application identity you need to set the value-strategy attribute on the primary key field. You can configure the Meta-Data for the class something like this (replacing 'myclass' and 'myfield' with your class and field names) : 

<class name="myclass" identity-type="application" objectid-class="myprimarykeyclass">
    <field name="myfield" primary-key="true" value-strategy="identity"/<
    ...
</class>

Please be aware that if you have an inheritance tree with the base class defined as using "identity" then the column definition for the PK of the base table will be defined as "AUTO_INCREMENT" or "IDENTITY" or "SERIAL" (dependent on the RDBMS) and all subtables will NOT have this identifier added to their PK column definitions. This is because the identities are assigned in the base table (since all objects will have an entry in the base table).

Please note that if using optimistic transactions, this strategy will mean that the value is only set when the object is actually persisted (i.e at flush() or commit())

This value generator will generate values unique across different JVMs



increment

This method is database neutral and uses a sequence table that holds an incrementing sequence value. The unique identifier value returned from the database is translated to a java type: java.lang.Long. This strategy will work with any datastore. This method require a sequence table in the database and creates one if doesn't exist.

To configure a datastore identity class to use this generation method you simply add this to the classes Meta-Data. 

<class name="myclass" ... >
    <datastore-identity strategy="increment"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' Meta-Data. If your class is in an inheritance tree you should define this for the base class only. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="increment"/>
    ...
</class>>

Additional properties for configuring this generator are set in the JDO Meta-Data, see the available properties below. Unsupported properties are silently ignored by DataNucleus.

Property Description Required
key-initial-value First value to be allocated. No. Defaults to 1
key-cache-size number of unique identifiers to cache. The keys are pre-allocated, cached and used on demand. If key-cache-size is greater than 1, it may generate holes in the object keys in the database, if not all keys are used. Refer to persistence property datanucleus.valuegeneration.increment.allocationSize No. Defaults to 10
sequence-table-basis Whether to define uniqueness on the base class name or the base table name. Since there is no "base table name" when the root class has "subclass-table" this should be set to "class" when the root class has "subclass-table" inheritance No. Defaults to class , but the other option is table
sequence-name name for the sequence (overriding the "sequence-table-basis" above). The row in the table will use this in the PK column No
sequence-table-name Table name for storing the sequence. No. Defaults to SEQUENCE_TABLE
sequence-catalog-name Name of the catalog where the table is. No.
sequence-schema-name Name of the schema where the table is. No.
sequence-name-column-name Name for the column that represent sequence names. No. Defaults to SEQUENCE_NAME
sequence-nextval-column-name Name for the column that represent incremeting sequence values. No. Defaults to NEXT_VAL
table-name Name of the table whose column we are generating the value for (used when we have no previous sequence value and want a start point. No.
column-name Name of the column we are generating the value for (used when we have no previous sequence value and want a start point. No.

This value generator will generate values unique across different JVMs (from DataNucleus 1.1.3)



uuid-string

This generator creates identities with 16 characters in string format. The identity contains the IP address of the local machine where DataNucleus is running, as well as other necessary components to provide uniqueness across time.

This generator can be used in concurrent applications. It is especially useful in situations where large numbers of transactions within a certain amount of time have to be made, and the additional overhead of synchronizing the concurrent creation of unique identifiers through the database would break performance limits. It doesn't require datastore access to generate the identities and so has performance benefits over some of the other generators.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="uuid-string"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="uuid-string"/>
    ...
</class>


uuid-hex

This generator creates identities with 32 characters in hexadecimal format. The identity contains the IP address of the local machine where DataNucleus is running, as well as other necessary components to provide uniqueness across time.

This generator can be used in concurrent applications. It is especially useful in situations where large numbers of transactions within a certain amount of time have to be made, and the additional overhead of synchronizing the concurrent creation of unique identifiers through the database would break performance limits. It doesn't require datastore access to generate the identities and so has performance benefits over some of the other generators.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="uuid-hex"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="uuid-hex"/>
    ...
</class>


datastore-uuid-hex

This method is like the "uuid-hex" option above except that it utilises datastore capabilities to generate the UUIDHEX code. Consequently this only works on some RDBMS (MSSQL, MySQL). The disadvantage of this strategy is that it makes a call to the datastore for each new UUID required. The generated UUID is in the same form as the AUID strategy where identities are generated in memory and so the AUID strategy is the recommended choice relative to this option.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="datastore-uuid-hex"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="datastore-uuid-hex"/>
    ...
</class>


max

This method is database neutral and uses the "select max(column) from table" + 1 strategy to create unique ids. The unique identifier value returned from the database is translated to a java type: java.lang.Long. It is however not recommended by DataNucleus since it makes a DB call for every record to be inserted and hence is inefficient. Each DB call will run a scan in all table contents causing contention and locks in the table. We recommend the use of either Sequence or Identity based value generators (see below) - which you use would depend on your RDBMS.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="max"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="max"/>
    ...
</class>

This value generator will NOT guarantee to generate values unique across different JVMs. This is because it will select the "max+1" and before creating the record another thread may come in and insert one.



uuid

This generator uses the JDK1.5 UUID class to generate values. The values are 128-bit (36 character) of the form "0e400c2c-b3a0-4786-a0c6-f2607bf643eb"

This generator can be used in concurrent applications. It is especially useful in situations where large numbers of transactions within a certain amount of time have to be made, and the additional overhead of synchronizing the concurrent creation of unique identifiers through the database would break performance limits.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="uuid"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="uuid"/>
    ...
</class>

Or using annotations

public class MyClass
{
    @Persistent(customValueStrategy="uuid")
    String myField;
}

This value generator will generate values unique across different JVMs



auid

This generator uses a Java implementation of DCE UUIDs to create unique identifiers without the overhead of additional database transactions or even an open database connection. The identifiers are Strings of the form "LLLLLLLL-MMMM-HHHH-CCCC-NNNNNNNNNNNN" where 'L', 'M', 'H', 'C' and 'N' are the DCE UUID fields named time low, time mid, time high, clock sequence and node.

This generator can be used in concurrent applications. It is especially useful in situations where large numbers of transactions within a certain amount of time have to be made, and the additional overhead of synchronizing the concurrent creation of unique identifiers through the database would break performance limits.

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="auid"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="auid"/>
    ...
</class>

This value generator will generate values unique across different JVMs



timestamp

This method will create a java.sql.Timestamp of the current time (at insertion in the datastore).

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="timestamp"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="timestamp"/>
    ...
</class>


timestamp-value

This method will create a long of the current time in millisecs (at insertion in the datastore).

For a class using datastore identity you need to add metadata something like the following 

<class name="myclass" ... >
    <datastore-identity strategy="timestamp-value"/>
    ...
</class>

To configure an application identity class to use this generation method you simply add this to the class' JDO Meta-Data. 

<class name="myclass" ... >
    <field name="myfield" primary-key="true" value-strategy="timestamp-value"/>
    ...
</class>


Standalone ID generation

This section describes how to use the DataNucleus Value Generator API for generating unique keys for objects outside the DataNucleus (JDO) runtime. DataNucleus defines a framework for identity generation and provides many builtin strategies for identities. You can make use of the same strategies described above but for generating identities manually for your own use. The process is described below

The DataNucleus Value Generator API revolves around 2 classes. The entry point for retrieving generators is the ValueGenerationManager . This manages the appropriate ValueGenerator classes. Value generators maintain a block of cached ids in memory which avoid reading the database each time it needs a new unique id. Caching a block of unique ids provides you the best performance but can cause "holes" in the sequence of ids for the stored objects in the database.

Let's take an example. Here we want to obtain an identity using the TableGenerator ("increment" above). This stores identities in a datastore table. We want to generate an identity using this. Here is what we add to our code 

PersistenceManagerImpl pm = (PersistenceManagerImpl) ... // cast your pm to impl ;

// Obtain a ValueGenerationManager
ValueGenerationManager mgr = new ValueGenerationManager();

// Obtain a ValueGenerator of the required type
Properties properties = new Properties();
properties.setProperty("sequence-name", "GLOBAL"); // Use a global sequence number (for all tables)
ValueGenerator generator = mgr.createValueGenerator("MyGenerator",
    org.datanucleus.store.rdbms.valuegenerator.TableGenerator.class, props, pm.getStoreManager(),
                new ValueGenerationConnectionProvider()
                {
                    RDBMSManager rdbmsManager = null;
                    ManagedConnection con;
                    public ManagedConnection retrieveConnection()
                    {
                        rdbmsManager = (RDBMSManager) pm.getStoreManager();
                        try
                        {
                            // important to use TRANSACTION_NONE like DataNucleus does
                            con = rdbmsManager.getConnection(Connection.TRANSACTION_NONE);;
                            return con;
                        }
                        catch (SQLException e)
                        {
                            logger.error("Failed to obtain new DB connection for identity generation!");
                            throw new RuntimeException(e);
                        }
                    }
                    public void releaseConnection()
                    {
                        try
                        {
                            con.close();
                            con = null;
                        }
                        catch (DataNucleusException e)
                        {
                            logger.error("Failed to close DB connection for identity generation!");
                            throw new RuntimeException(e);
                        }
                        finally
                        {
                            rdbmsManager = null;
                        }
                    }
                });

// Retrieve the next identity using this strategy
Long identifier = (Long)generator.next();

Some ValueGenerators are specific to RDBMS datastores, and some are generic, so bear this in mind when selecting and adding your own.