Issue Details (XML | Word | Printable)

Key: NUCCORE-588
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Andy Jefferson
Reporter: Tom Zurkan
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
DataNucleus Core

Be able to cache relationships.

Created: 21/Oct/10 11:19 PM   Updated: 23/Jan/11 11:57 AM   Resolved: 10/Dec/10 11:08 AM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.2.0.release

File Attachments: 1. Java Source File CacheRelationshipTest.java (4 kB)
2. Java Source File CacheRelationshipTest.java (3 kB)
3. File jdostatemanager.diff (14 kB)
4. Java Source File JDOStateManager2.java (168 kB)
5. Java Source File RelationshipTest.java (3 kB)


Datastore: MySQL


 Description  « Hide
What we would like to do is be able to cache relationship fields so that the first time the relationship is accessed, there will be a join against the owner id. On subsequent calls, the relationship would be cached with the id. The id would be used to look up in the cache and either pass back the object, load it by id, and cache it. The relationship is removed when a field is made dirty.

I have augmented the StateManager to accomplish this but I am not sure I have gone down the best path. I am submitting the enclosed augmented JDOStatemanager to show where I trying to go with this. One thing to look at it that I have to use the OID to get the class of the object id i have so that i can create hallow pms with statemanagers.

it is configurable via the cacheRelationships variable. <persistence-property name="datanucleus.CacheRelationships" value="false"/>

Andy Jefferson made changes - 22/Oct/10 06:19 AM
Field Original Value New Value
Description What we would like to do is be able to cache relationship fields so that the first time the relationship is accessed, there will be a join against the owner id. On subsequent calls, the relationship would be cached with the id. The id would be used to look up in the cache and either pass back the object, load it by id, and cache it. The relationship is removed when a field is made dirty.

I have augmented the StateManager to accomplish this but I am not sure I have gone down the best path. I am submitting the enclosed augmented JDOStatemanager to show where I trying to go with this. One thing to look at it that I have to use the OID to get the class of the object id i have so that i can create hallow pms with statemanagers.

it is configurable via the cacheRelationships variable. <persistence-property name="datanucleus.CacheRelationships" value="false"/>

package com.protrade.common.datanucleus.jdo.state;

import java.io.PrintWriter;
import java.lang.reflect.Array;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.ArrayList;
import java.util.BitSet;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;

import javax.jdo.JDOFatalInternalException;
import javax.jdo.JDOFatalUserException;
import javax.jdo.JDOUserException;
import javax.jdo.PersistenceManager;
import javax.jdo.spi.Detachable;
import javax.jdo.spi.JDOImplHelper;
import javax.jdo.spi.PersistenceCapable;
import javax.jdo.spi.StateManager;

import org.datanucleus.ClassLoaderResolver;
import org.datanucleus.FetchPlan;
import org.datanucleus.FetchPlanForClass;
import org.datanucleus.ObjectManager;
import org.datanucleus.PersistenceConfiguration;
import org.datanucleus.api.ApiAdapter;
import org.datanucleus.cache.CachedPC;
import org.datanucleus.cache.Level2Cache;
import org.datanucleus.exceptions.ClassNotResolvedException;
import org.datanucleus.exceptions.NucleusException;
import org.datanucleus.exceptions.NucleusObjectNotFoundException;
import org.datanucleus.exceptions.NucleusUserException;
import org.datanucleus.identity.OID;
import org.datanucleus.identity.OIDFactory;
import org.datanucleus.identity.OIDImplKodo;
import org.datanucleus.jdo.NucleusJDOHelper;
import org.datanucleus.jdo.state.JDOStateManagerImpl;
import org.datanucleus.metadata.AbstractClassMetaData;
import org.datanucleus.metadata.AbstractMemberMetaData;
import org.datanucleus.metadata.IdentityStrategy;
import org.datanucleus.metadata.IdentityType;
import org.datanucleus.metadata.MetaDataUtils;
import org.datanucleus.metadata.Relation;
import org.datanucleus.state.AbstractStateManager;
import org.datanucleus.state.ActivityState;
import org.datanucleus.state.DetachState;
import org.datanucleus.state.FetchPlanState;
import org.datanucleus.state.LifeCycleState;
import org.datanucleus.state.ObjectProviderImpl;
import org.datanucleus.state.RelationshipManager;
import org.datanucleus.state.StateManagerFactory;
import org.datanucleus.store.FieldValues;
import org.datanucleus.store.ObjectProvider;
import org.datanucleus.store.exceptions.NotYetFlushedException;
import org.datanucleus.store.fieldmanager.AttachFieldManager;
import org.datanucleus.store.fieldmanager.CachePopulateFieldManager;
import org.datanucleus.store.fieldmanager.CacheRetrieveFieldManager;
import org.datanucleus.store.fieldmanager.DeleteFieldManager;
import org.datanucleus.store.fieldmanager.DetachFieldManager;
import org.datanucleus.store.fieldmanager.FieldManager;
import org.datanucleus.store.fieldmanager.LoadFieldManager;
import org.datanucleus.store.fieldmanager.MakeTransientFieldManager;
import org.datanucleus.store.fieldmanager.NullifyRelationFieldManager;
import org.datanucleus.store.fieldmanager.PersistFieldManager;
import org.datanucleus.store.fieldmanager.ReachabilityFieldManager;
import org.datanucleus.store.fieldmanager.SingleTypeFieldManager;
import org.datanucleus.store.fieldmanager.SingleValueFieldManager;
import org.datanucleus.store.fieldmanager.AbstractFetchFieldManager.EndOfFetchPlanGraphException;
import org.datanucleus.store.objectvaluegenerator.ObjectValueGenerator;
import org.datanucleus.store.types.sco.SCO;
import org.datanucleus.store.types.sco.SCOCollection;
import org.datanucleus.store.types.sco.SCOContainer;
import org.datanucleus.store.types.sco.SCOMap;
import org.datanucleus.store.types.sco.SCOUtils;
import org.datanucleus.store.types.sco.UnsetOwners;
import org.datanucleus.util.NucleusLogger;
import org.datanucleus.util.StringUtils;
import org.datanucleus.util.TypeConversionHelper;

public class JDOStateManagerImpl2 extends AbstractStateManager implements StateManager {
private static final JDOImplHelper HELPER;

private static final SingleTypeFieldManager HOLLOWFIELDMANAGER = new SingleTypeFieldManager();

/** Flag for {@link #operationalFlags} whether we are retrieving detached state from the detached object. */
private static final byte MISC_RETRIEVING_DETACHED_STATE = (byte) ( 1 << 7 );

/** Flag for {@link #operationalFlags} whether we are resetting the detached state. */
private static final byte MISC_RESETTING_DETACHED_STATE = (byte) ( 1 << 6 );

/** Flag whether this SM is updating the ownership of its embedded/serialised field(s). */
private static final byte MISC_UPDATING_EMBEDDING_FIELDS_WITH_OWNER = (byte) ( 1 << 5 );

/** Flag for {@link #operationalFlags} whether we are in the process of attaching the object. */
private static final byte MISC_ATTACHING = (byte) ( 1 << 4 );

/** Flag for {@link #operationalFlags} whether we are in the process of detaching the object. */
private static final byte MISC_DETACHING = (byte) ( 1 << 3 );

/** Flag for {@link #operationalFlags} whether we are in the process of making transient the object. */
private static final byte MISC_MAKING_TRANSIENT = (byte) ( 1 << 2 );

/** Flag for {@link #operationalFlags} whether we are in the process of flushing changes to the object. */
private static final byte MISC_FLUSHING = (byte) ( 1 << 1 );

/** Flag for {@link #operationalFlags} whether we are in the process of disconnecting the object. */
private static final byte MISC_DISCONNECTING = (byte) ( 1 << 0 );

/** Bit-packed flags for operational settings. */
private byte operationalFlags;

/** Flags for DFG state stored with the object. */
private byte jdoDfgFlags;

/** Flags of the PersistenceCapable instance when the instance is enlisted in the transaction. */
private byte savedFlags;

/** Image of the PersistenceCapable instance when the instance is enlisted in the transaction. */
private PersistenceCapable savedImage = null;

/** Loaded fields of the PersistenceCapable instance when the instance is enlisted in the transaction. */
private boolean[] savedLoadedFields = null;

/** if the PersistenceCapable instance is new and was flushed to the datastore. */
private boolean flushedNew = false;

/** state for transitions of activities. */
private ActivityState activity = ActivityState.NONE;

private boolean changingState = false;

private boolean postLoadPending = false;

/** Whether the managed object needs the inheritance level validating before loading fields. */
private boolean needsInheritanceValidating = false;

/** Referenced PC object whilst attaching/detaching, for the other object in the process (if any). */
private PersistenceCapable referencedPC = null;

/** List of StateManagers that we must notify when we have completed inserting our record. */
private List<org.datanucleus.StateManager> insertionNotifyList = null;

/** Fields of this object that we must update when notified of the insertion of the related objects. */
private Map<StateManager,FieldContainer> fieldsToBeUpdatedAfterObjectInsertion = null;

/** List of owners when embedded. */
private List<EmbeddedOwnerRelation> embeddedOwners = null;

/** Manager of relationships for the managed object. Will be null if no fields yet to be managed. */
private RelationshipManager relationManager;

/**
* Map of associated values for the object being managed. This can contain anything really and is down to the StoreManager to define. For example RDBMS datastores
* typically put external FK info in here keyed by the mapping of the field to which it pertains.
*/
private HashMap associatedValuesMap = null;

private boolean cacheRelationships = false;

private ObjectProvider objectProvider = new ObjectProviderImpl( this );

static {
HELPER = NucleusJDOHelper.getJDOImplHelper();
}

/**
* Basic constructor. Delegates to the superclass.
*
* @param om The ObjectManager
* @param cmd the metadata for the class.
*/
public JDOStateManagerImpl2( ObjectManager om, AbstractClassMetaData cmd ) {
super( om, cmd );

        PersistenceConfiguration config = om.getOMFContext().getPersistenceConfiguration();


        cacheRelationships = config.getBooleanProperty( "datanucleus.CacheRelationships", false );

}

/**
* Initialises a state manager to manage a hollow instance having the given object ID and the given (optional) field values. This constructor is used for creating new
* instances of existing persistent objects, and consequently shouldnt be used when the StoreManager controls the creation of such objects (such as in an ODBMS).
*
* @param id the JDO identity of the object.
* @param fv the initial field values of the object (optional)
* @param pcClass Class of the object that this will manage the state for
*/
public void initialiseForHollow( Object id, FieldValues fv, Class pcClass ) {
myID = id;
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.HOLLOW );
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
if ( id instanceof OID || id == null ) {
// Create new PC
myPC = HELPER.newInstance( pcClass, this );
} else {
// Create new PC, and copy the key class to fields
myPC = HELPER.newInstance( pcClass, this, myID );
markPKFieldsAsLoaded();
}

if ( fv != null ) {
loadFieldValues( fv );
}
}

/**
* Initialises a state manager to manage a HOLLOW / P_CLEAN instance having the given FieldValues. This constructor is used for creating new instances of existing
* persistent objects using application identity, and consequently shouldnt be used when the StoreManager controls the creation of such objects (such as in an ODBMS).
*
* @param fv the initial field values of the object.
* @param pcClass Class of the object that this will manage the state for
*/
public void initialiseForHollowAppId( FieldValues fv, Class pcClass ) {
if ( cmd.getIdentityType() != IdentityType.APPLICATION ) {
throw new NucleusUserException( "This constructor is only for objects using application identity." ).setFatal();
}

myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.HOLLOW );
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC = HELPER.newInstance( pcClass, this ); // Create new PC
if ( myPC == null ) {
if ( !HELPER.getRegisteredClasses().contains( pcClass ) ) {
// probably never will get here, as JDOImplHelper.newInstance() internally already throws
// JDOFatalUserException when class is not registered
throw new NucleusUserException( LOCALISER.msg( "026018", pcClass.getName() ) ).setFatal();
} else {
// Provide advisory information since we can't create an instance of this class, so maybe they
// have an error in their data ?
throw new NucleusUserException( LOCALISER.msg( "026019", pcClass.getName() ) ).setFatal();
}
}

loadFieldValues( fv ); // as a minimum the PK fields are loaded here

// Create the ID now that we have the PK fields loaded
myID = myPC.jdoNewObjectIdInstance();
if ( !cmd.usesSingleFieldIdentityClass() ) {
myPC.jdoCopyKeyFieldsToObjectId( myID );
}
}

/**
* Initialises a state manager to manage the given hollow instance having the given object ID. Unlike the {@link #initialiseForHollow} method, this method does not
* create a new instance and instead takes a pre-constructed instance.
*
* @param id the identity of the object.
* @param pc the object to be managed.
*/
public void initialiseForHollowPreConstructed( Object id, Object pc ) {
myID = id;
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.HOLLOW );
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC = (PersistenceCapable) pc;

replaceStateManager( myPC, this ); // Assign this StateManager to the PC
myPC.jdoReplaceFlags();

// TODO Add to the cache
}

/**
* Initialises a state manager to manage the passed persistent instance having the given object ID. Used where we have retrieved a PC object from a datastore directly
* (not field-by-field), for example on an object datastore. This initialiser will not add StateManagers to all related PCs. This must be done by any calling process.
* This simply adds the StateManager to the specified object and records the id, setting all fields of the object as loaded.
*
* @param id the identity of the object.
* @param pc The object to be managed
*/
public void initialiseForPersistentClean( Object id, Object pc ) {
myID = id;
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_CLEAN );
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC = (PersistenceCapable) pc;

replaceStateManager( myPC, this ); // Assign this StateManager to the PC
myPC.jdoReplaceFlags();

// Mark all fields as loaded
for ( int i = 0; i < loadedFields.length; ++i ) {
loadedFields[i] = true;
}

// Add the object to the cache
myOM.putObjectIntoCache( this );
}

/**
* Initialises a state manager to manage a PersistenceCapable instance that will be EMBEDDED/SERIALISED into another PersistenceCapable object. The instance will not
* be assigned an identity in the process since it is a SCO.
*
* @param pc The PersistenceCapable to manage (see copyPc also)
* @param copyPc Whether the SM should manage a copy of the passed PC or that one
*/
public void initialiseForEmbedded( Object pc, boolean copyPc ) {
pcObjectType = EMBEDDED_PC; // Default to an embedded PC object
myID = null; // It is embedded at this point so dont need an ID since we're not persisting it
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_NEW );
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;

myPC = (PersistenceCapable) pc;
replaceStateManager( myPC, this ); // Set SM for embedded PC to be this
if ( copyPc ) {
// Create a new PC with the same field values
PersistenceCapable pcCopy = myPC.jdoNewInstance( this );
pcCopy.jdoCopyFields( myPC, getAllFieldNumbers() );

// Swap the managed PC to be the copy and not the input
replaceStateManager( pcCopy, this );
myPC = pcCopy;
disconnectClone( (PersistenceCapable) pc );
}

// Mark all fields as loaded since we are using the passed PersistenceCapable
for ( int i = 0; i < loadedFields.length; i++ ) {
loadedFields[i] = true;
}
}

/**
* Initialises a state manager to manage a transient instance that is becoming newly persistent. A new object ID for the instance is obtained from the store manager
* and the object is inserted in the data store.
* <p>
* This constructor is used for assigning state managers to existing instances that are transitioning to a persistent state.
*
* @param pc the instance being make persistent.
* @param preInsertChanges Any changes to make before inserting
*/
public void initialiseForPersistentNew( Object pc, FieldValues preInsertChanges ) {
myPC = (PersistenceCapable) pc;
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_NEW );
jdoDfgFlags = PersistenceCapable.READ_OK;
for ( int i = 0; i < loadedFields.length; ++i ) {
loadedFields[i] = true;
}

replaceStateManager( myPC, this ); // Assign this StateManager to the PC
myPC.jdoReplaceFlags();

saveFields();

// Populate all fields that have "value-strategy" and are not datastore populated
populateStrategyFields();

if ( preInsertChanges != null ) {
// Apply any pre-insert field updates
preInsertChanges.fetchFields( this );
}

if ( cmd.getIdentityType() == IdentityType.APPLICATION ) {
// load key fields from Application Id instance to PC instance

// if a primary key field is of type PersistenceCapable, it must first be persistent
for ( int fieldNumber = 0; fieldNumber < getAllFieldNumbers().length; fieldNumber++ ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( fmd.isPrimaryKey() ) {
if ( myOM.getMetaDataManager().getMetaDataForClass( fmd.getType(), getObjectManager().getClassLoaderResolver() ) != null ) {
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
try {
currFM = new SingleValueFieldManager();
myPC.jdoProvideField( fieldNumber );
PersistenceCapable pkFieldPC = (PersistenceCapable) ( (SingleValueFieldManager) currFM ).fetchObjectField( fieldNumber );
if ( pkFieldPC == null ) {
throw new NucleusUserException( LOCALISER.msg( "026016", fmd.getFullFieldName() ) );
}
if ( !myOM.getApiAdapter().isPersistent( pkFieldPC ) ) {
// Make sure the PC field is persistent - can cause the insert of our object
// being managed by this SM via flush() when bidir relation
Object persistedFieldPC = myOM.persistObjectInternal( pkFieldPC, null, null, -1, org.datanucleus.StateManager.PC );
replaceField( myPC, fieldNumber, persistedFieldPC, false );
}
}
finally {
currFM = prevFM;
}
}
}
}
}
}

/*
* Set the identity This must come after the above block, because in identifying relationships the PK FK associations must be persisted before, otherwise we don't
* have an id assigned to the PK FK associations
*/
setIdentity( false );

if ( this.getObjectManager().getTransaction().isActive() ) {
myOM.enlistInTransaction( this );
}

// Now in PERSISTENT_NEW so call any callbacks/listeners
getCallbackHandler().postCreate( myPC );

if ( myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.manageRelationships" ) ) {
// Managed Relations : register non-null bidir fields for later processing
ClassLoaderResolver clr = myOM.getClassLoaderResolver();
int[] relationPositions = cmd.getRelationMemberPositions( clr, getMetaDataManager() );
if ( relationPositions != null ) {
for ( int i = 0; i < relationPositions.length; i++ ) {
AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( relationPositions[i] );
int relationType = mmd.getRelationType( clr );
if ( relationType == Relation.ONE_TO_ONE_BI || relationType == Relation.MANY_TO_ONE_BI || relationType == Relation.ONE_TO_MANY_BI
|| relationType == Relation.MANY_TO_MANY_BI ) {
Object value = provideField( relationPositions[i] );
if ( value != null ) {
if ( relationManager == null ) {
relationManager = new RelationshipManager( this );
}
// Store the field with value of null so it gets checked
relationManager.relationChange( relationPositions[i], null, null );
}
}
}
}
}
}

/**
* Initialises a state manager to manage a Transactional Transient instance. A new object ID for the instance is obtained from the store manager and the object is
* inserted in the data store.
* <p>
* This constructor is used for assigning state managers to Transient instances that are transitioning to a transient clean state.
*
* @param pc the instance being make persistent.
*/
public void initialiseForTransactionalTransient( Object pc ) {
myPC = (PersistenceCapable) pc;
myLC = null;
jdoDfgFlags = PersistenceCapable.READ_OK;
for ( int i = 0; i < loadedFields.length; ++i ) {
loadedFields[i] = true;
}
myPC.jdoReplaceFlags();

// Populate all fields that have "value-strategy" and are not datastore populated
populateStrategyFields();

// Set the identity
setIdentity( false );

// for non transactional read, tx might be not active
// TODO add verification if is non transactional read = true
if ( myOM.getTransaction().isActive() ) {
myOM.enlistInTransaction( this );
}
}

/**
* Initialises the StateManager to manage a PersistenceCapable object in detached state.
*
* @param pc the detach object.
* @param id the identity of the object.
* @param version the detached version
*/
public void initialiseForDetached( Object pc, Object id, Object version ) {
this.myID = id;
this.myPC = (PersistenceCapable) pc;
setVersion( version );

// This lifecycle state is not always correct. It is certainly "detached"
// but we dont know if it is CLEAN or DIRTY. We need this setting here since all objects
// have a lifecycle state and other methods e.g isPersistent() depend on it.
this.myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.DETACHED_CLEAN );

this.myPC.jdoReplaceFlags();
replaceStateManager( myPC, this );
}

/**
* Initialises the StateManager to manage a PersistenceCapable object that is not persistent but is about to be deleted.
*
* @param pc the object to delete
*/
public void initialiseForPNewToBeDeleted( Object pc ) {
this.myID = null;
this.myPC = (PersistenceCapable) pc;
this.myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_NEW );
for ( int i = 0; i < loadedFields.length; ++i ) // Mark all fields as loaded
{
loadedFields[i] = true;
}
replaceStateManager( myPC, this );
}

/**
* Look to the database to determine which class this object is. This parameter is a hint. Set false, if it's already determined the correct pcClass for this pc
* "object" in a certain level in the hierarchy. Set to true and it will look to the database.
*
* @param fv the initial field values of the object.
*/
public void checkInheritance( FieldValues fv ) {
// Inheritance case, check the level of the instance
ClassLoaderResolver clr = myOM.getClassLoaderResolver();
String className = getStoreManager().getClassNameForObjectID( myID, clr, myOM.getExecutionContext() );
if ( className == null ) {
// className is null when id class exists, and object has been validated and doesn't exist.
throw new NucleusObjectNotFoundException( LOCALISER.msg( "026013", myOM.getIdentityAsString( myID ) ), myID );
} else if ( !cmd.getFullClassName().equals( className ) ) {
Class pcClass;
try {
// load the class and make sure the class is initialized
pcClass = clr.classForName( className, myID.getClass().getClassLoader(), true );
cmd = myOM.getMetaDataManager().getMetaDataForClass( pcClass, clr );
}
catch ( ClassNotResolvedException e ) {
NucleusLogger.PERSISTENCE.warn( LOCALISER.msg( "026014", myOM.getIdentityAsString( myID ) ) );
throw new NucleusUserException( LOCALISER.msg( "026014", myOM.getIdentityAsString( myID ) ), e );
}
if ( cmd == null ) {
throw new NucleusUserException( LOCALISER.msg( "026012", pcClass ) ).setFatal();
}
if ( cmd.getIdentityType() != IdentityType.APPLICATION ) {
throw new NucleusUserException( "This method should only be used for objects using application identity." ).setFatal();
}
myFP = myOM.getFetchPlan().manageFetchPlanForClass( cmd );

initialiseFieldInformation();

// Create new PC at right inheritance level
myPC = HELPER.newInstance( pcClass, this );
if ( myPC == null ) {
throw new NucleusUserException( LOCALISER.msg( "026018", cmd.getFullClassName() ) ).setFatal();
}

// Note that this will mean the fields are loaded twice (loaded earlier in this method)
// and also that postLoad will be called twice
loadFieldValues( fv );

// Create the id for the new PC
myID = myPC.jdoNewObjectIdInstance();
if ( !cmd.usesSingleFieldIdentityClass() ) {
myPC.jdoCopyKeyFieldsToObjectId( myID );
}
}
}

/**
* Convenience method to populate all fields in the PC object that have "value-strategy" specified and that aren't datastore attributed. This applies not just to PK
* fields (where it is most useful to use value-strategy) but also to any other field. Fields are populated only if they are null This is called once on a PC object,
* when makePersistent is called.
*/
private void populateStrategyFields() {
int totalFieldCount = cmd.getNoOfInheritedManagedMembers() + cmd.getNoOfManagedMembers();

for ( int fieldNumber = 0; fieldNumber < totalFieldCount; fieldNumber++ ) {
AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
IdentityStrategy strategy = mmd.getValueStrategy();

// Check for the strategy, and if it is a datastore attributed strategy
if ( strategy != null && !getStoreManager().isStrategyDatastoreAttributed( strategy, false ) ) {
// Assign the strategy value where required.
// Default JDO2 behaviour is to always provide a strategy value when it is marked as using a strategy
boolean applyStrategy = true;
if ( !mmd.getType().isPrimitive() && strategy != null && mmd.hasExtension( "strategy-when-notnull" )
&& mmd.getValueForExtension( "strategy-when-notnull" ).equalsIgnoreCase( "false" ) && this.provideField( fieldNumber ) != null ) {
// extension to only provide a value-strategy value where the field is null at persistence.
applyStrategy = false;
}

if ( applyStrategy ) {
// Apply a strategy value for this field
Object obj = getStoreManager().getStrategyValue( myOM.getExecutionContext(), cmd, fieldNumber );
this.replaceField( fieldNumber, obj, true );
}
} else if ( mmd.hasExtension( "object-value-generator" ) ) {
// Field has object value-generator so generate value based on this object
String valGenName = mmd.getValueForExtension( "object-value-generator" );
ObjectValueGenerator valGen = getObjectValueGenerator( myOM, valGenName );
Object value = valGen.generate( myOM.getExecutionContext(), myPC, mmd.getExtensions() );
this.replaceField( fieldNumber, value, true );
}
}
}

/** Cache of object-value-generators, keyed by their symbolic name. */
protected static HashMap<String,ObjectValueGenerator> objectValGenerators = null;

/**
* Method to find an object value generator based on its name. Caches the generators once generated.
*
* @param om ObjectManager
* @param genName The generator name
* @return The value generator (if any)
* @throws NucleusException if no generator of that name is found
*/
protected static ObjectValueGenerator getObjectValueGenerator( ObjectManager om, String genName ) {
if ( objectValGenerators != null ) {
ObjectValueGenerator valGen = objectValGenerators.get( genName );
if ( valGen != null ) {
return valGen;
}
}

try {
ObjectValueGenerator valGen = (ObjectValueGenerator) om.getOMFContext().getPluginManager().createExecutableExtension(
"org.datanucleus.store_objectvaluegenerator", new String[] { "name" }, new String[] { genName }, "class-name", null, null );
if ( objectValGenerators == null ) {
objectValGenerators = new HashMap();
}
objectValGenerators.put( genName, valGen );
return valGen;
}
catch ( Exception e ) {
NucleusLogger.VALUEGENERATION.info( "Exception thrown generating value using objectvaluegenerator " + genName, e );
throw new NucleusException( "Exception thrown generating value for object", e );
}
}

/**
* Utility to set the identity for the PersistenceCapable object. Creates the identity instance if the required PK field(s) are all already set (by the user, or by a
* value-strategy). If the identity is set in the datastore (sequence, autoassign, etc) then this will not set the identity.
*
* @param afterPreStore Whether preStore has (just) been invoked
*/
private void setIdentity( boolean afterPreStore ) {
if ( cmd.isEmbeddedOnly() ) {
// Embedded objects don't have an "identity"
return;
}

if ( cmd.getIdentityType() == IdentityType.DATASTORE ) {
if ( cmd.getIdentityMetaData() == null || !getStoreManager().isStrategyDatastoreAttributed( cmd.getIdentityMetaData().getValueStrategy(), true ) ) {
// Assumed to be set
myID = myOM.newObjectId( cmd.getFullClassName(), myPC );
}
} else if ( cmd.getIdentityType() == IdentityType.APPLICATION ) {
boolean idSetInDatastore = false;
int totalFieldCount = cmd.getNoOfInheritedManagedMembers() + cmd.getNoOfManagedMembers();
for ( int fieldNumber = 0; fieldNumber < totalFieldCount; fieldNumber++ ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( fmd.isPrimaryKey() ) {
if ( getStoreManager().isStrategyDatastoreAttributed( fmd.getValueStrategy(), false ) ) {
idSetInDatastore = true;
break;
} else if ( cmd.usesSingleFieldIdentityClass() ) {
if ( this.provideField( fieldNumber ) == null ) {
// PK field has not had its value set (user/value-strategy)
// and must be set for single-field identity
if ( afterPreStore ) {
// Not set even after preStore, so user error
throw new NucleusUserException( LOCALISER.msg( "026017", cmd.getFullClassName(), fmd.getName() ) ).setFatal();
} else {
// Log that the value is not yet set for this field, maybe set later in preStore?
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026017", cmd.getFullClassName(), fmd.getName() ) );
return;
}
}
}
}
}

if ( !idSetInDatastore ) {
// Not generating the identity in the datastore so set it now
myID = myOM.newObjectId( cmd.getFullClassName(), myPC );
}
}

if ( myInternalID != myID && myID != null ) {
// Update the id with the PM if it is changing
myOM.replaceObjectId( myPC, myInternalID, myID );

this.myInternalID = myID;
}
}

/**
* Method that replaces the PC managed by this StateManager to be the supplied object. This happens when we want to get an object for an id and create a Hollow
* object, and then validate against the datastore. This validation can pull in a new object graph from the datastore (e.g for DB4O)
*
* @param pc The PersistenceCapable to use
*/
public void replaceManagedPC( PersistenceCapable pc ) {
if ( pc == null ) {
return;
}

// Swap the StateManager on the objects
replaceStateManager( pc, this );
replaceStateManager( myPC, null );

// Swap our object
myPC = pc;

// Put it in the cache in case the previous object was stored
myOM.putObjectIntoCache( this );
}

/**
* Convenience method to update our object with the field values from the passed object. Objects need to be of the same type, and the other object should not have a
* StateManager.
*
* @param pc The object that we should copy fields from
*/
public void copyFieldsFromObject( PersistenceCapable pc, int[] fieldNumbers ) {
if ( pc == null ) {
return;
}
if ( !pc.getClass().getName().equals( myPC.getClass().getName() ) ) {
return;
}

// Assign the new object to this StateManager temporarily so that we can copy its fields
replaceStateManager( pc, this );
myPC.jdoCopyFields( pc, fieldNumbers );

// Remove the StateManager from the other object
replaceStateManager( pc, null );

// Set the loaded flags now that we have copied
for ( int i = 0; i < fieldNumbers.length; i++ ) {
loadedFields[fieldNumbers[i]] = true;
}
}

/**
* Utility to update the passed object with the passed StateManager (can be null).
*
* @param pc The object to update
* @param sm The new state manager
*/
private void replaceStateManager( final PersistenceCapable pc, final StateManager sm ) {
try {
// Calls to pc.jdoReplaceStateManager must be run privileged
AccessController.doPrivileged( new PrivilegedAction() {
public Object run() {
pc.jdoReplaceStateManager( sm );
return null;
}
} );
}
catch ( SecurityException e ) {
throw new JDOFatalUserException( LOCALISER.msg( "026000" ), e );
}
}

/**
* Method to enlist the managed object in the current transaction.
*/
public void enlistInTransaction() {
if ( !getObjectManager().getTransaction().isActive() ) {
return;
}
myOM.enlistInTransaction( this );

if ( jdoDfgFlags == PersistenceCapable.LOAD_REQUIRED && isDefaultFetchGroupLoaded() ) {
// All DFG fields loaded and object is transactional so it doesnt need to contact us for those fields
// Note that this is the DFG and NOT the current FetchPlan since in the enhancement of classes
// all DFG fields are set to check jdoFlags before relaying back to the StateManager
jdoDfgFlags = PersistenceCapable.READ_OK;
myPC.jdoReplaceFlags();
}
}

/**
* Method to evict the managed object from the current transaction.
*/
public void evictFromTransaction() {
myOM.evictFromTransaction( this );

/*
* A non-transactional object needs to contact us on any field read no matter what fields are loaded.
*/
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC.jdoReplaceFlags();
}

/**
* Method to save all fields of the object.
*/
public void saveFields() {
savedImage = myPC.jdoNewInstance( this );
savedImage.jdoCopyFields( myPC, getAllFieldNumbers() );
savedFlags = jdoDfgFlags;
savedLoadedFields = loadedFields.clone();
}

/**
* Method to restore all fields of the object.
*/
public void restoreFields() {
if ( savedImage != null ) {
loadedFields = savedLoadedFields;
jdoDfgFlags = savedFlags;
myPC.jdoReplaceFlags();
myPC.jdoCopyFields( savedImage, getAllFieldNumbers() );

clearDirtyFlags();
clearSavedFields();
}
}

/**
* Method to clear all fields of the object.
*/
public void clearFields() {
try {
getCallbackHandler().preClear( myPC );
}
finally {
clearFieldsByNumbers( getAllFieldNumbers() );
clearDirtyFlags();

getStoreManager().notifyObjectIsOutdated( this.getObjectProvider() ); // For datastores that manage the object reference
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC.jdoReplaceFlags();

getCallbackHandler().postClear( myPC );
}
}

/**
* Method to clear all fields that are not part of the primary key of the object.
*/
public void clearNonPrimaryKeyFields() {
try {
getCallbackHandler().preClear( myPC );
}
finally {
clearFieldsByNumbers( getNonPrimaryKeyFieldNumbers() );

clearDirtyFlags( getNonPrimaryKeyFieldNumbers() );

getStoreManager().notifyObjectIsOutdated( this.getObjectProvider() ); // For datastores that manage the object reference
jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC.jdoReplaceFlags();

getCallbackHandler().postClear( myPC );
}
}

private void clearFieldsByNumbers( int[] fieldNumbers ) {
replaceFields( fieldNumbers, HOLLOWFIELDMANAGER );
for ( int i = 0; i < fieldNumbers.length; i++ ) {
loadedFields[fieldNumbers[i]] = false;
dirtyFields[fieldNumbers[i]] = false;
}
}

/**
* Method to clear all saved fields on the object.
*/
public void clearSavedFields() {
savedImage = null;
savedFlags = 0;
savedLoadedFields = null;
}

/**
* Method to clear all loaded flags on the object. Note that the contract of this method implies, especially for object database backends, that the memory form of the
* object is outdated. Thus, for features like implicit saving of dirty object subgraphs should be switched off for this PC, even if the object actually looks like
* being dirty (because it is being changed to null values).
*/
public void clearLoadedFlags() {
getStoreManager().notifyObjectIsOutdated( this.getObjectProvider() );

jdoDfgFlags = PersistenceCapable.LOAD_REQUIRED;
myPC.jdoReplaceFlags();
clearFlags( loadedFields );
}

/**
* Marks the given field dirty.
*
* @param field The no of field to mark as dirty.
*/
public void makeDirty( int field ) {
if ( activity != ActivityState.DELETING ) {
// Mark dirty unless in the process of being deleted
boolean wasDirty = preWriteField( field );
postWriteField( wasDirty );

if ( embeddedOwners != null ) {
// Notify any owners that embed this object that it has just changed
for ( EmbeddedOwnerRelation owner : embeddedOwners ) {
JDOStateManagerImpl2 ownerSM = (JDOStateManagerImpl2) owner.sm;

if ( ownerSM == null || ownerSM.cmd == null ) {
// for some reason these are null... raised when running JPA TCK
continue;
}

if ( ( ownerSM.operationalFlags & MISC_UPDATING_EMBEDDING_FIELDS_WITH_OWNER ) == 0 ) {
ownerSM.makeDirty( owner.fieldNumber );
}
}
}
}
}

/**
* Mark the associated PersistenceCapable field dirty.
*
* @param pc the calling PersistenceCapable instance
* @param fieldName the name of the field
*/
public void makeDirty( PersistenceCapable pc, String fieldName ) {
if ( !disconnectClone( pc ) ) {
int fieldNumber = cmd.getAbsolutePositionOfMember( fieldName );
if ( fieldNumber == -1 ) {
throw new JDOUserException( LOCALISER.msg( "026002", fieldName, cmd.getFullClassName() ) );
}

if (cacheRelationships) {
removeFieldFromL2Cache(pc, fieldName);
setAssociatedValue( fieldName, null );
}
makeDirty( fieldNumber );
}
}

// -------------------------- Accessor Methods -----------------------------

private void removeFieldFromL2Cache( PersistenceCapable pc, String fieldName ) {
Level2Cache l2cache = myOM.getObjectManagerFactory().getLevel2Cache();
if ( cacheRelationships && l2cache != null && cmd.isCacheable() && !myOM.isObjectModifiedInTransaction( myID ) ) {
synchronized ( l2cache ) {
CachedPC cachedPC = l2cache.get( pc.jdoGetObjectId() );
if ( cachedPC != null ) {
if (cachedPC.getRelationField( fieldName ) != null) {
NucleusLogger.CACHE.debug( "ASS: removing relation for field " + fieldName + " for "+myID+" from cache" );
cachedPC.setRelationField( fieldName, null );
}
}
}
}
}

/**
* Accessor for the PersistenceManager that owns this instance.
*
* @param pc The PersistenceCapable instance
* @return The PersistenceManager that owns this instance
*/
public javax.jdo.PersistenceManager getPersistenceManager( PersistenceCapable pc ) {
// in identifying relationships, jdoCopyKeyFieldsFromId will call
// this method, and at this moment, myPC in statemanager is null
// Currently AbstractPersistenceManager.java putObjectInCache prevents any identifying relation object being put in L2

// if not identifying relationship, do the default check of disconnectClone:
// "this.disconnectClone(pc)"
if ( myPC != null && this.disconnectClone( pc ) ) {
return null;
} else if ( myOM == null ) {
return null;
} else {
myOM.hereIsStateManager( this, myPC );
return (PersistenceManager) myOM.getOwner();
}
}

/**
* Return the object representing the JDO identity of the calling instance.
*
* According to the JDO specification, if the JDO identity is being changed in the current transaction, this method returns the JDO identify as of the beginning of
* the transaction.
*
* @param pc the calling PersistenceCapable instance
* @return the object representing the JDO identity of the calling instance
*/
public Object getObjectId( PersistenceCapable pc ) {
if ( disconnectClone( pc ) ) {
return null;
} else {
return getExternalObjectId( pc );
}
}

/**
* If the id is obtained after inserting the object into the database, set new a new id for persistent classes (for example, increment).
*
* @param id the id received from the datastore
*/
public void setPostStoreNewObjectId( Object id ) {
if ( cmd.getIdentityType() == IdentityType.DATASTORE ) {
if ( id instanceof OID ) {
// Provided an OID direct
this.myID = id;
} else {
// OID "key" value provided
myID = OIDFactory.getInstance( myOM.getOMFContext(), cmd.getFullClassName(), id );
}
} else if ( cmd.getIdentityType() == IdentityType.APPLICATION ) {
try {
myID = null;

int fieldCount = getHighestFieldNumber();
for ( int fieldNumber = 0; fieldNumber < fieldCount; fieldNumber++ ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( fmd.isPrimaryKey() && getStoreManager().isStrategyDatastoreAttributed( fmd.getValueStrategy(), false ) ) {
// replace the value of the id, but before convert the value to the field type if needed
replaceField( fieldNumber, TypeConversionHelper.convertTo( id, fmd.getType() ), true );
}
}
}
catch ( Exception e ) {
NucleusLogger.PERSISTENCE.error( e );
}
finally {
myID = myOM.getApiAdapter().getNewApplicationIdentityObjectId( getObject(), cmd );
}
}

if ( myInternalID != myID && myID != null ) {
// Update the id with the PM if it is changing
myOM.replaceObjectId( myPC, myInternalID, myID );

this.myInternalID = myID;
}
}

/**
* Return an object id that the user can use.
*
* @param obj the PersistenceCapable object
* @return the object id
*/
public Object getExternalObjectId( Object obj ) {
if ( cmd.getIdentityType() == IdentityType.DATASTORE ) {
if ( ( ( operationalFlags & MISC_FLUSHING ) == 0 ) ) {
// Flush any datastore changes so that myID is set by the time we return
if ( !flushedNew && activity != ActivityState.INSERTING && activity != ActivityState.INSERTING_CALLBACKS && myLC.stateType() == LifeCycleState.P_NEW ) {
if ( getStoreManager().isStrategyDatastoreAttributed( cmd.getIdentityMetaData().getValueStrategy(), true ) ) {
flush();
}
}
}
} else if ( cmd.getIdentityType() == IdentityType.APPLICATION ) {
// Note that we always create a new application identity since it is mutable and we can't allow
// the user to change it. The only drawback of this is that we *must* have the relevant fields
// set when this method is called, so that the identity can be generated.
if ( ( ( operationalFlags & MISC_FLUSHING ) == 0 ) ) {
// Flush any datastore changes so that we have all necessary fields populated
// only if the datastore generates the field numbers
if ( !flushedNew && activity != ActivityState.INSERTING && activity != ActivityState.INSERTING_CALLBACKS && myLC.stateType() == LifeCycleState.P_NEW ) {
int[] pkFieldNumbers = cmd.getPKMemberPositions();
for ( int i = 0; i < pkFieldNumbers.length; i++ ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( pkFieldNumbers[i] );
if ( getStoreManager().isStrategyDatastoreAttributed( fmd.getValueStrategy(), false ) ) {
flush();
break;
}
}
}
}

if ( cmd.usesSingleFieldIdentityClass() ) {
// SingleFieldIdentity classes are immutable.
// Note, the instances of SingleFieldIdentity can be changed by the user using reflection,
// but this is not allowed by the JDO spec
return myID;
}
return myOM.getApiAdapter().getNewApplicationIdentityObjectId( myPC, cmd );
}

return myID;
}

/**
* Replace the current value of jdoStateManager.
*
* <P>
* This method is called by the PersistenceCapable whenever jdoReplaceStateManager is called and there is already an owning StateManager. This is a security
* precaution to ensure that the owning StateManager is the only source of any change to its reference in the PersistenceCapable.
* </p>
*
* @return the new value for the jdoStateManager
* @param pc the calling PersistenceCapable instance
* @param sm the proposed new value for the jdoStateManager
*/
public StateManager replacingStateManager( PersistenceCapable pc, StateManager sm ) {
if ( myLC == null ) {
throw new JDOFatalInternalException( "Null LifeCycleState" );
}

if ( myLC.stateType() == LifeCycleState.DETACHED_CLEAN ) {
return sm;
}

if ( pc == myPC ) {
// TODO check if we are really in transition to a transient instance
if ( sm == null ) {
return null;
}
if ( sm == this ) {
return this;
}

if ( this.myOM == ( (AbstractStateManager) sm ).getObjectManager() ) {
// This is a race condition when makePersistent or
// makeTransactional is called on the same PC instance for the
// same PM. It has been already set to this SM - just
// disconnect the other one. Return this SM so it won't be
// replaced.
( (JDOStateManagerImpl2) sm ).disconnect();
return this;
}

if ( sm != null ) {
throw new JDOUserException( LOCALISER.msg( "026003" ) );
}
if ( ( ( operationalFlags & MISC_DISCONNECTING ) == 0 ) ) {
throw new JDOUserException( LOCALISER.msg( "026004" ) );
}

if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026005", StringUtils.toJVMIDString( pc ) ) );
}

return null;
} else if ( pc == savedImage ) {
return null;
} else {
return sm;
}
}

/**
* Return the object representing the JDO identity of the calling instance. If the JDO identity is being changed in the current transaction, this method returns the
* current identity as changed in the transaction.
*
* @param pc the calling PersistenceCapable instance
* @return the object representing the JDO identity of the calling instance
*/
public Object getTransactionalObjectId( PersistenceCapable pc ) {
return getObjectId( pc );
}

// --------------------------- Load Field Methods --------------------------

/**
* Convenience method to update a Level2 cached version of this object if cacheable and has not been modified during this transaction. If any of the specified fields
* are not loaded in the current L2 cached object this will update the loaded value for that field(s).
*
* @param fieldNumbers Numbers of fields to update
*/
private void updateLevel2CacheForFields( int[] fieldNumbers ) {
Level2Cache l2cache = myOM.getObjectManagerFactory().getLevel2Cache();
if ( l2cache != null && cmd.isCacheable() && !myOM.isObjectModifiedInTransaction( myID ) ) {
synchronized ( l2cache ) {
CachedPC cachedPC = l2cache.get( myID );
if ( cachedPC != null ) {
int[] cacheFieldsToLoad = getFlagsSetTo( cachedPC.getLoadedFields(), fieldNumbers, false );
if ( cacheFieldsToLoad != null && cacheFieldsToLoad.length > 0 ) {
NucleusLogger.CACHE.debug( LOCALISER.msg( "026033", StringUtils.toJVMIDString( myPC ), myID, StringUtils.intArrayToString( cacheFieldsToLoad ) ) );

// Connect a StateManager to the cachedPC
Object cachePC = cachedPC.getPersistableObject();
JDOStateManagerImpl2 cacheSM = new JDOStateManagerImpl2( myOM, cmd );
// TODO Change to use a specific method - lifecycle state is not strictly correct
cacheSM.initialiseForDetached( cachePC, getExternalObjectId( myPC ), getVersion( myPC ) );

// Copy across the specified fields into the cached object
cacheSM.replaceFields( cacheFieldsToLoad, new CachePopulateFieldManager( this.getObjectProvider(), cachedPC ) );

if (cacheRelationships) {
if ( associatedValuesMap != null ) {
for ( Object o : associatedValuesMap.keySet() ) {
String name = (String) o;
if ( cachedPC.getRelationField( name ) == null ) {
NucleusLogger.CACHE.debug( "ASS: updating relations for " + name + " from cache" );
cachedPC.setRelationField( name, getAssociatedValue( name ) );
}
}
}
}
// Disconnect the StateManager
replaceStateManager( ( (PersistenceCapable) cachePC ), null );
}
}
}
}
}

/**
* Fetchs from the database all SCO fields that are not containers that aren't already loaded.
*/
private void loadSCONonContainerFields() {
int[] noncontainerFieldNumbers = cmd.getSCONonContainerMemberPositions();
int[] fieldNumbers = getFlagsSetTo( loadedFields, noncontainerFieldNumbers, false );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
loadFieldsFromDatastore( fieldNumbers );
updateLevel2CacheForFields( fieldNumbers );
// We currently don't call postLoad here since this is only called as part of attaching an object
// and consequently we just read to get the current (attached) values.
// Could add a flag on input to allow postLoad
}
}

/**
* Convenience method to load the specified field if not loaded.
*
* @param fieldNumber Absolute field number
*/
public void loadField( int fieldNumber ) {
if ( loadedFields[fieldNumber] ) {
// Already loaded
return;
}
loadSpecifiedFields( new int[] { fieldNumber } );
}

/**
* Fetch from the database all fields that are not currently loaded regardless of whether they are in the current fetch group or not. Called by lifecycle transitions.
*/
public void loadUnloadedFields() {
int[] fieldNumbers = getFlagsSetTo( loadedFields, getAllFieldNumbers(), false );
if ( fieldNumbers == null || fieldNumbers.length == 0 ) {
// All loaded so return
return;
}

if ( preDeleteLoadedFields != null && ( ( myLC.isDeleted() && myOM.isFlushing() ) || activity == ActivityState.DELETING ) ) {
// During deletion process so we know what is really loaded so only load if necessary
fieldNumbers = getFlagsSetTo( preDeleteLoadedFields, fieldNumbers, false );
}

if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
loadFieldsFromDatastore( fieldNumbers );

int[] secondClassMutableFieldNumbers = getSecondClassMutableFieldNumbers();

// Make sure all SCO lazy-loaded fields have been loaded
for ( int i = 0; i < secondClassMutableFieldNumbers.length; i++ ) {
SingleValueFieldManager sfv = new SingleValueFieldManager();
provideFields( new int[] { secondClassMutableFieldNumbers[i] }, sfv );
Object value = sfv.fetchObjectField( i );
if ( value instanceof SCOContainer ) {
( (SCOContainer) value ).load();
}
}

updateLevel2CacheForFields( fieldNumbers );
if ( callPostLoad ) {
postLoad();
}
}
}

boolean loadingFieldsInFetchPlan = false;

/**
* Method to load all unloaded fields in the FetchPlan. Recurses through the FetchPlan objects and loads fields of sub-objects where needed. Used as a precursor to
* detaching objects at commit since fields can't be loaded during the postCommit phase when the detach actually happens.
*
* @param state The FetchPlan state
*/
public void loadFieldsInFetchPlan( FetchPlanState state ) {
if ( loadingFieldsInFetchPlan ) {
// Already in the process of loading fields in this class so skip
return;
}

// Load unloaded FetchPlan fields of this object
loadingFieldsInFetchPlan = true;
loadUnloadedFieldsInFetchPlan();

// Recurse through all fields and do the same
int[] fieldNumbers = getFlagsSetTo( loadedFields, getAllFieldNumbers(), true );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
// TODO Fix this to just access the fields of the FieldManager yet this actually does a replaceField
replaceFields( fieldNumbers, new LoadFieldManager( this.getObjectProvider(), getSecondClassMutableFields(), myFP, state ) );
updateLevel2CacheForFields( fieldNumbers );
}

loadingFieldsInFetchPlan = false;
}

/**
* Fetchs from the database all fields that are not currently loaded and that are in the current fetch group. Called by lifecycle transitions.
*/
public void loadUnloadedFieldsInFetchPlan() {
int[] fieldNumbers = getFlagsSetTo( loadedFields, myFP.getMemberNumbers(), false );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
loadFieldsFromDatastore( fieldNumbers );
updateLevel2CacheForFields( fieldNumbers );
if ( callPostLoad ) {
postLoad();
}
}
}

/**
* Fetchs from the database all fields in current fetch plan that are not currently loaded as well as the version. Called by lifecycle transitions.
*/
public void loadUnloadedFieldsInFetchPlanAndVersion() {
if ( !cmd.hasVersionStrategy() ) {
loadUnloadedFieldsInFetchPlan();
} else {
int[] fieldNumbers = getFlagsSetTo( loadedFields, myFP.getMemberNumbers(), false );
if ( fieldNumbers == null ) {
fieldNumbers = new int[0];
}

boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
loadFieldsFromDatastore( fieldNumbers );
updateLevel2CacheForFields( fieldNumbers );
if ( callPostLoad && fieldNumbers.length > 0 ) {
postLoad();
}
}
}

/**
* Fetchs from the database all fields in the actual fetch plan. Called by life-cycle transitions.
*/
public void loadUnloadedFieldsOfClassInFetchPlan( FetchPlan fetchPlan ) {
FetchPlanForClass fpc = fetchPlan.manageFetchPlanForClass( this.cmd );
int[] fieldNumbers = getFlagsSetTo( loadedFields, fpc.getMemberNumbers(), false );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
boolean callPostLoad = fpc.isToCallPostLoadFetchPlan( this.loadedFields );
loadFieldsFromDatastore( fieldNumbers );
updateLevel2CacheForFields( fieldNumbers );
if ( callPostLoad ) {
postLoad();
}
}
}

/**
* Convenience method to unload a field/property.
*
* @param fieldName Name of the field/property
* @throws NucleusUserException if the object managed by this StateManager is embedded
*/
public void unloadField( String fieldName ) {
if ( pcObjectType == PC ) {
// Mark as not loaded
AbstractMemberMetaData mmd = getClassMetaData().getMetaDataForMember( fieldName );
loadedFields[mmd.getAbsoluteFieldNumber()] = false;
} else {
throw new NucleusUserException( "Cannot unload field/property of embedded object" );
}
}

/**
* Convenience method to mark PK fields as loaded (if using app id).
*/
protected void markPKFieldsAsLoaded() {
if ( cmd.getIdentityType() == IdentityType.APPLICATION ) {
int[] pkPositions = cmd.getPKMemberPositions();
for ( int i = 0; i < pkPositions.length; i++ ) {
loadedFields[pkPositions[i]] = true;
}
}
}

/**
* Refreshes from the database all fields in fetch plan. Called by life-cycle transitions when the object undergoes a "transitionRefresh".
*/
public void refreshFieldsInFetchPlan() {
int[] fieldNumbers = myFP.getMemberNumbers();
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
clearDirtyFlags( fieldNumbers );
clearFlags( loadedFields, fieldNumbers );
markPKFieldsAsLoaded(); // Can't refresh PK fields!

boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );

// Refresh the fetch plan fields in this object
setTransactionalVersion( null ); // Make sure that the version is reset upon fetch
loadFieldsFromDatastore( fieldNumbers );

if ( cmd.hasRelations( myOM.getClassLoaderResolver(), getMetaDataManager() ) ) {
// Check for cascade refreshes to related objects
for ( int i = 0; i < fieldNumbers.length; i++ ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumbers[i] );
int relationType = fmd.getRelationType( myOM.getClassLoaderResolver() );
if ( relationType != Relation.NONE && fmd.isCascadeRefresh() ) {
// Need to refresh the related field object(s)
Object value = provideField( fieldNumbers[i] );
if ( value != null ) {
if ( value instanceof Collection ) {
// Refresh any PC elements in the collection
// TODO This should replace the SCO wrapper with a new one, or reload the wrapper
SCOUtils.refreshFetchPlanFieldsForCollection( this.getObjectProvider(), ( (Collection) value ).toArray() );
} else if ( value instanceof Map ) {
// Refresh any PC keys/values in the map
// TODO This should replace the SCO wrapper with a new one, or reload the wrapper
SCOUtils.refreshFetchPlanFieldsForMap( this.getObjectProvider(), ( (Map) value ).entrySet() );
} else if ( value instanceof PersistenceCapable ) {
// Refresh any PC fields
myOM.refreshObject( value );
}
}
}
}
}

if ( callPostLoad ) {
postLoad();
}

getCallbackHandler().postRefresh( myPC );
}
}

/**
* Refreshes from the database all fields currently loaded. Called by life-cycle transitions when making transactional or reading fields.
*/
public void refreshLoadedFields() {
int[] fieldNumbers = getFlagsSetTo( loadedFields, myFP.getMemberNumbers(), true );

if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
clearDirtyFlags();
clearFlags( loadedFields );
markPKFieldsAsLoaded(); // Can't refresh PK fields!

boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
loadFieldsFromDatastore( fieldNumbers );
if ( callPostLoad ) {
postLoad();
}
}
}

/**
* Method that will unload all fields that are not in the FetchPlan. This is typically for use when the instance is being refreshed.
*/
public void unloadNonFetchPlanFields() {
int[] fpFieldNumbers = myFP.getMemberNumbers();
int[] nonfpFieldNumbers = null;
if ( fpFieldNumbers == null || fpFieldNumbers.length == 0 ) {
nonfpFieldNumbers = getAllFieldNumbers();
} else {
int fieldCount = getHighestFieldNumber();
if ( fieldCount == fpFieldNumbers.length ) {
// No fields that arent in FetchPlan
return;
}

nonfpFieldNumbers = new int[fieldCount - fpFieldNumbers.length];
int currentFPFieldIndex = 0;
int j = 0;
for ( int i = 0; i < fieldCount; i++ ) {
if ( currentFPFieldIndex >= fpFieldNumbers.length ) {
// Past end of FetchPlan fields
nonfpFieldNumbers[j++ ] = i;
} else {
if ( fpFieldNumbers[currentFPFieldIndex] == i ) {
// FetchPlan field so move to next
currentFPFieldIndex++ ;
} else {
nonfpFieldNumbers[j++ ] = i;
}
}
}
}

// Mark all non-FetchPlan fields as unloaded
for ( int i = 0; i < nonfpFieldNumbers.length; i++ ) {
loadedFields[nonfpFieldNumbers[i]] = false;
}
}

/**
* Convenience method to load a field from the datastore. Used in attaching fields and checking their old values (so we dont want any postLoad method being called).
* TODO Merge this with one of the loadXXXFields methods.
*
* @param fieldNumber The field number.
*/
public void loadFieldFromDatastore( int fieldNumber ) {
if ( needsInheritanceValidating ) // TODO Merge this into fetch object handler
{
validateInheritance();
}

// TODO If the field has "loadFetchGroup" defined, then add it to the fetch plan etc
getStoreManager().getPersistenceHandler().fetchObject( objectProvider, new int[] { fieldNumber } );
}

/**
* Convenience method to load a field from the datastore.
*
* @param fieldNumbers The field numbers.
*/
private void loadFieldsFromDatastore( int[] fieldNumbers ) {
if ( needsInheritanceValidating ) // TODO Merge this into fetch object handler
{
validateInheritance();
}

// TODO If the field has "loadFetchGroup" defined, then add it to the fetch plan etc
getStoreManager().getPersistenceHandler().fetchObject( objectProvider, fieldNumbers );
}

/**
* Method to validate the inheritance of this object. Used in the situation where the user requested an object with an id and didn't want it validating at that point,
* but we must prevent its usage when fields are loaded if it is basically wrong.
*
* @throws NucleusObjectNotFoundException if the object has incorrect inheritance level
*/
private void validateInheritance() {
String className = getStoreManager().getClassNameForObjectID( myID, myOM.getClassLoaderResolver(), myOM.getExecutionContext() );
if ( !myPC.getClass().getName().equals( className ) ) {
myOM.removeObjectFromCache( myPC, myID );
//myOM.removeObjectFromLevel2Cache( myID );
throw new NucleusObjectNotFoundException( "Object with id " + myID + " was created without validating of type " + myPC.getClass().getName()
+ " but is actually of type " + className );
}
needsInheritanceValidating = false;
}

/**
* Return true if the field is cached in the calling instance. In this implementation, isLoaded() will always return true. If the field is not loaded, it will be
* loaded as a side effect of the call to this method. If it is in the default fetch group, the default fetch group, including this field, will be loaded.
*
* @param pc the calling PersistenceCapable instance
* @param field the absolute field number
* @return always returns true (this implementation)
*/
public boolean isLoaded( PersistenceCapable pc, int field ) {
try {
if ( disconnectClone( pc ) ) {
return true;
} else {
boolean checkRead = true;
boolean beingDeleted = false;
if ( ( myLC.isDeleted() && myOM.isFlushing() ) || activity == ActivityState.DELETING ) {
// Bypass "read-field" check when deleting, or when marked for deletion and flushing
checkRead = false;
beingDeleted = true;
}
if ( checkRead ) {
transitionReadField( loadedFields[field] );
}

if ( !loadedFields[field] ) {
// Field not loaded, so load it
if ( pcObjectType != PC ) {
// Embedded object so we assume that all was loaded before (when it was read)
return true;
}

if ( beingDeleted && preDeleteLoadedFields != null && preDeleteLoadedFields[field] ) {
// Field was loaded prior to starting delete so just return true
return true;
} else if ( !beingDeleted && myFP.hasMember( field ) ) {
// Load rest of FetchPlan if this is part of it (and not in the process of deletion)
loadUnloadedFieldsInFetchPlan();
} else {
// Just load this field
loadSpecifiedFields( new int[] { field } );
}
}

return true;
}
}
catch ( NucleusException ne ) {
NucleusLogger.PERSISTENCE.warn( "Exception thrown by StateManager.isLoaded", ne );

// Convert into a JDOException since this is called from a user update of a field
throw NucleusJDOHelper.getJDOExceptionForNucleusException( ne );
}
}

// ---------------------- Field Accessor/Mutator Methods -------------------

/**
* Called by the various setXXXField() methods once it's established that the field really deserves to be written. Makes the state transition and replaces the field
* value in the PC instance.
*/
private void writeField( int field, Object newValue ) {
replaceField( field, newValue, true );
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setBooleanField( PersistenceCapable pc, int field, boolean currentValue, boolean newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, newValue ? Boolean.TRUE : Boolean.FALSE, true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, newValue ? Boolean.TRUE : Boolean.FALSE );
postWriteField( wasDirty );
}
} else {
replaceField( field, newValue ? Boolean.TRUE : Boolean.FALSE, true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setByteField( PersistenceCapable pc, int field, byte currentValue, byte newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Byte.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Byte.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Byte.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setCharField( PersistenceCapable pc, int field, char currentValue, char newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Character.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Character.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Character.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setDoubleField( PersistenceCapable pc, int field, double currentValue, double newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Double.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Double.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Double.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setFloatField( PersistenceCapable pc, int field, float currentValue, float newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Float.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Float.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Float.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setIntField( PersistenceCapable pc, int field, int currentValue, int newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Integer.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Integer.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Integer.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setLongField( PersistenceCapable pc, int field, long currentValue, long newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Long.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Long.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Long.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setShortField( PersistenceCapable pc, int field, short currentValue, short newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, Short.valueOf( newValue ), true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || currentValue != newValue ) {
boolean wasDirty = preWriteField( field );
writeField( field, Short.valueOf( newValue ) );
postWriteField( wasDirty );
}
} else {
replaceField( field, Short.valueOf( newValue ), true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setStringField( PersistenceCapable pc, int field, String currentValue, String newValue ) {
if ( pc != myPC ) {
replaceField( pc, field, newValue, true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

if ( !loadedFields[field] || !equals( currentValue, newValue ) ) {
boolean wasDirty = preWriteField( field );
writeField( field, newValue );
postWriteField( wasDirty );
}
} else {
replaceField( field, newValue, true );
}
}

/**
* This method is called by the associated PersistenceCapable when the corresponding mutator method (setXXX()) is called on the PersistenceCapable.
*
* @param pc the calling PersistenceCapable instance
* @param field the field number
* @param currentValue the current value of the field
* @param newValue the new value for the field
*/
public void setObjectField( PersistenceCapable pc, int field, Object currentValue, Object newValue ) {
if ( currentValue != null && currentValue != newValue && currentValue instanceof PersistenceCapable ) {
// Where the object is embedded, remove the owner from its old value since it is no longer managed by this StateManager
JDOStateManagerImpl2 currentSM = (JDOStateManagerImpl2) myOM.findStateManager( currentValue );
if ( currentSM != null && currentSM.isEmbedded() ) {
currentSM.removeEmbeddedOwner( this, field );
}
}

if ( pc != myPC ) {
// Clone
replaceField( pc, field, newValue, true );
disconnectClone( pc );
} else if ( myLC != null ) {
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
// Not got version but should have
loadUnloadedFieldsInFetchPlanAndVersion();
}

boolean loadedOldValue = false;
Object oldValue = currentValue;
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( field );
ClassLoaderResolver clr = myOM.getClassLoaderResolver();
int relationType = fmd.getRelationType( clr );
if ( !loadedFields[field] && currentValue == null ) {
// Updating value of a field that isnt currently loaded
if ( myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.manageRelationships" )
&& ( relationType == Relation.ONE_TO_ONE_BI || relationType == Relation.MANY_TO_ONE_BI ) ) {
// Managed relation field, so load old value
loadField( field );
loadedOldValue = true;
oldValue = provideField( field );
}

if ( relationType != Relation.NONE && newValue == null && ( fmd.isDependent() || fmd.isCascadeRemoveOrphans() ) ) {
// Field being nulled and is dependent so load the existing value so it can be deleted
loadField( field );
loadedOldValue = true;
oldValue = provideField( field );
}
// TODO When field has relation consider loading it always for managed relations
}

// Check equality of old and new values
boolean equal = false;
if ( oldValue == null && newValue == null ) {
equal = true;
} else if ( oldValue != null && newValue != null ) {
if ( oldValue instanceof PersistenceCapable ) {
// PC object field so compare object equality
// See JDO2 [5.4] "The JDO implementation must not use the application's hashCode and equals methods
// from the persistence-capable classes except as needed to implement the Collections Framework"
if ( oldValue == newValue ) {
equal = true;
}
} else {
// Non-PC object field so compare using equals()
if ( oldValue.equals( newValue ) ) {
equal = true;
}
}
}

// Update the field
if ( !loadedFields[field] || !equal || fmd.hasArray() ) {
// Either field isn't loaded, or has changed, or is an array.
// We include arrays here since we have no way of knowing if the array element has changed
// except if the user sets the array field. See JDO2 [6.3] that the application should
// replace the value with its current value.
boolean wasDirty = preWriteField( field );

if ( oldValue instanceof SCO ) {
if ( oldValue instanceof SCOContainer ) {
// Make sure container values are loaded
( (SCOContainer) oldValue ).load();
}
( (SCO) oldValue ).unsetOwner();
}
if ( newValue instanceof SCO ) {
SCO sco = (SCO) newValue;
Object owner = sco.getOwner();
if ( owner != null ) {
throw new JDOUserException( LOCALISER.msg( "026007", sco.getFieldName(), owner ) );
}
}

writeField( field, newValue );
postWriteField( wasDirty );
} else if ( loadedOldValue ) {
// We've updated the value with the old value (when retrieving it above), so put the new value back again
boolean wasDirty = preWriteField( field );
writeField( field, newValue );
postWriteField( wasDirty );
}

if ( !equal && myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.manageRelationships" ) ) {
// Managed Relations : register updated bidir fields for later processing
if ( relationType == Relation.ONE_TO_ONE_BI || relationType == Relation.MANY_TO_ONE_BI || relationType == Relation.ONE_TO_MANY_BI
|| relationType == Relation.MANY_TO_MANY_BI ) {
// Managed Relationships - add the field to be managed so we can analyse its value at flush
if ( relationManager == null ) {
relationManager = new RelationshipManager( this );
}
relationManager.relationChange( field, oldValue, newValue );
}
}

if ( oldValue != null && newValue == null && oldValue instanceof PersistenceCapable ) {
if ( fmd.isDependent() || fmd.isCascadeRemoveOrphans() ) {
if ( myOM.getApiAdapter().isPersistent( oldValue ) ) {
// PC field being nulled, so delete previous PC value
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026026", oldValue, fmd.getFullFieldName() ) );
myOM.deleteObjectInternal( oldValue );
}
}
}
} else {
replaceField( field, newValue, true );
}
}

/**
* Accessor for the relationship manager, and create if not existing and we are managing relations.
*
* @return The Relationship manager
*/
public RelationshipManager getRelationshipManager() {
if ( relationManager == null && myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.manageRelationships" )
&& !myOM.isManagingRelations() ) {
relationManager = new RelationshipManager( this );
}
return relationManager;
}

/**
* Method to check all updated managed relations in this object.
*/
public void checkManagedRelations() {
if ( myLC == null || myLC.isDeleted() ) {
// Has been deleted so ignore all relationship changes
return;
}
if ( relationManager == null ) {
return;
}
relationManager.checkConsistency();
}

/**
* Method to process all updated managed relations in this object.
*/
public void processManagedRelations() {
if ( myLC == null || myLC.isDeleted() ) {
// Has been deleted so ignore all relationship changes
return;
}
if ( relationManager == null ) {
return;
}
relationManager.process();
}

/**
* Method to clear all initial values for bidirectional fields involved in "managed relationships".
*/
public void clearManagedRelations() {
if ( relationManager != null ) {
relationManager.clearFields();
relationManager = null;
}
}

/**
* Convenience method to change the value of a field that is assumed loaded. Will mark the object/field as dirty if it isnt previously. If the object is deleted then
* does nothing. Only for use in management of relations.
*
* @param fieldNumber Number of field
* @param newValue The new value
*/
public void replaceFieldValue( int fieldNumber, Object newValue ) {
if ( myLC.isDeleted() ) {
// Object is deleted so do nothing
return;
}
boolean currentWasDirty = preWriteField( fieldNumber );
writeField( fieldNumber, newValue );
postWriteField( currentWasDirty );
}

/**
* The StateManager uses this method to supply the value of jdoFlags to the associated PersistenceCapable instance.
*
* @param pc the calling PersistenceCapable instance
* @return the value of jdoFlags to be stored in the PersistenceCapable instance
*/
public byte replacingFlags( PersistenceCapable pc ) {
// If this is a clone, return READ_WRITE_OK.
if ( pc != myPC ) {
return PersistenceCapable.READ_WRITE_OK;
} else {
return jdoDfgFlags;
}
}

/**
* Method to return the current value of a particular field.
*
* @param fieldNumber Number of field
* @return The value of the field
*/
public Object provideField( int fieldNumber ) {
return provideField( myPC, fieldNumber );
}

/**
* Method to change the value of a particular field.
*
* @param fieldNumber Number of field
* @param value New value
* @param makeDirty Whether to make the field dirty when replacing it
*/
public void replaceField( int fieldNumber, Object value, boolean makeDirty ) {
replaceField( myPC, fieldNumber, value, makeDirty );
}

/**
* Method to retrieve the value of a field from the PC object. Assumes that it is loaded.
*
* @param pc The PC object
* @param fieldNumber Number of field
* @return The value of the field
*/
private Object provideField( PersistenceCapable pc, int fieldNumber ) {
Object obj;
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
currFM = new SingleValueFieldManager();
try {
pc.jdoProvideField( fieldNumber );
obj = currFM.fetchObjectField( fieldNumber );
}
finally {
currFM = prevFM;
}
}

return obj;
}

/**
* Method to change the value of a field in the PC object.
*
* @param pc The PC object
* @param fieldNumber Number of field
* @param value The new value of the field
* @param makeDirty Whether to make the field dirty while replacing its value (in embedded owners)
*/
private void replaceField( PersistenceCapable pc, int fieldNumber, Object value, boolean makeDirty ) {
if ( embeddedOwners != null ) {
// Notify any owners that embed this object that it has just changed
// We do this before we actually change the object so we can compare with the old value
Iterator<EmbeddedOwnerRelation> ownerIter = embeddedOwners.iterator();
while ( ownerIter.hasNext() ) {
EmbeddedOwnerRelation owner = ownerIter.next();
JDOStateManagerImpl2 ownerSM = (JDOStateManagerImpl2) owner.sm;

if ( ownerSM == null || ownerSM.cmd == null ) {
// for some reason these are null... raised when running JPA TCK
continue;
}
AbstractMemberMetaData ownerFmd = ownerSM.cmd.getMetaDataForManagedMemberAtAbsolutePosition( owner.fieldNumber );
if ( ownerFmd.getCollection() != null ) {
// PC Object embedded in collection
Object ownerField = ownerSM.provideField( owner.fieldNumber );
if ( ownerField instanceof SCOCollection ) {
( (SCOCollection) ownerField ).updateEmbeddedElement( myPC, fieldNumber, value );
}
} else if ( ownerFmd.getMap() != null ) {
// PC Object embedded in map
Object ownerField = ownerSM.provideField( owner.fieldNumber );
if ( ownerField instanceof SCOMap ) {
if ( pcObjectType == EMBEDDED_MAP_KEY_PC ) {
( (SCOMap) ownerField ).updateEmbeddedKey( myPC, fieldNumber, value );
}
if ( pcObjectType == EMBEDDED_MAP_VALUE_PC ) {
( (SCOMap) ownerField ).updateEmbeddedValue( myPC, fieldNumber, value );
}
}
} else {
// PC Object embedded in PC object
if ( ( ownerSM.operationalFlags & MISC_UPDATING_EMBEDDING_FIELDS_WITH_OWNER ) == 0 ) {
// Update the owner when one of our fields have changed, EXCEPT when they have just
// notified us of our owner field!
if ( ownerSM.isEmbedded() ) {
// Owner is embedded so just update its field
ownerSM.replaceField( owner.fieldNumber, pc, makeDirty );
} else {
if ( makeDirty ) {
// Owner is not embedded so mark the field as dirty too
boolean wasDirty = ownerSM.preWriteField( owner.fieldNumber );
ownerSM.replaceField( owner.fieldNumber, pc, true );
ownerSM.postWriteField( wasDirty );
} else {
ownerSM.replaceField( owner.fieldNumber, pc, false );
}
}
}
}
}
}

// Update the field in our PC object
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
currFM = new SingleValueFieldManager();

try {
currFM.storeObjectField( fieldNumber, value );
pc.jdoReplaceField( fieldNumber );
}
finally {
currFM = prevFM;
}
}
}

/**
* Called from the StoreManager after StoreManager.update() is called to obtain updated values from the PersistenceCapable associated with this StateManager.
*
* @param fieldNumbers An array of field numbers to be updated by the Store
* @param fm The updated values are stored in this object. This object is only valid for the duration of this call.
*/
public void provideFields( int fieldNumbers[], FieldManager fm ) {
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
currFM = fm;

try {
// This will respond by calling this.providedXXXFields() with the value of the field
myPC.jdoProvideFields( fieldNumbers );
}
finally {
currFM = prevFM;
}
}
}

/**
* Called from the StoreManager to refresh data in the PersistenceCapable object associated with this StateManager.
*
* @param fieldNumbers An array of field numbers to be refreshed by the Store
* @param fm The updated values are stored in this object. This object is only valid for the duration of this call.
* @param replaceWhenDirty Whether to replace the fields when they are dirty here
*/
public void replaceFields( int fieldNumbers[], FieldManager fm, boolean replaceWhenDirty ) {
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
currFM = fm;

try {
int[] fieldsToReplace = fieldNumbers;
if ( !replaceWhenDirty ) {
int numberToReplace = fieldNumbers.length;
for ( int i = 0; i < fieldNumbers.length; i++ ) {
if ( dirtyFields[fieldNumbers[i]] ) {
numberToReplace-- ;
}
}
if ( numberToReplace > 0 && numberToReplace != fieldNumbers.length ) {
fieldsToReplace = new int[numberToReplace];
int n = 0;
for ( int i = 0; i < fieldNumbers.length; i++ ) {
if ( !dirtyFields[fieldNumbers[i]] ) {
fieldsToReplace[n++ ] = fieldNumbers[i];
}
}
} else if ( numberToReplace == 0 ) {
fieldsToReplace = null;
}
}

if ( fieldsToReplace != null ) {
myPC.jdoReplaceFields( fieldsToReplace );
}
}
finally {
currFM = prevFM;
}
}
}

/**
* Called from the StoreManager to refresh data in the PersistenceCapable object associated with this StateManager.
*
* @param fieldNumbers An array of field numbers to be refreshed by the Store
* @param fm The updated values are stored in this object. This object is only valid for the duration of this call.
*/
public void replaceFields( int fieldNumbers[], FieldManager fm ) {
replaceFields( fieldNumbers, fm, true );
}

/**
* Called from the StoreManager to refresh data in the PersistenceCapable object associated with this StateManager. Only not loaded fields are refreshed
*
* @param fieldNumbers An array of field numbers to be refreshed by the Store
* @param fm The updated values are stored in this object. This object is only valid for the duration of this call.
*/
public void replaceNonLoadedFields( int fieldNumbers[], FieldManager fm ) {
synchronized ( currFMmonitor ) {
FieldManager prevFM = currFM;
currFM = fm;

boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
try {
int[] fieldsToReplace = getFlagsSetTo( loadedFields, fieldNumbers, false );
if ( fieldsToReplace != null && fieldsToReplace.length > 0 ) {
myPC.jdoReplaceFields( fieldsToReplace );
}
}
finally {
currFM = prevFM;
}
if ( callPostLoad && isFetchPlanLoaded() ) {
// The fetch plan is now loaded so fire off any necessary post load
postLoad();
}
}
}

/**
* Method to register an owner StateManager with this embedded/serialised object.
*
* @param ownerSM The owning State Manager.
* @param ownerFieldNumber The field number in the owner that the embedded/serialised object is stored as
*/
public void addEmbeddedOwner( org.datanucleus.StateManager ownerSM, int ownerFieldNumber ) {
if ( ownerSM == null ) {
return;
}

if ( embeddedOwners == null ) {
embeddedOwners = new ArrayList();
}
embeddedOwners.add( new EmbeddedOwnerRelation( ownerSM, ownerFieldNumber ) );
}

/**
* Method to remove an owner StateManager from this embedded/serialised objects owners list.
*
* @param ownerSM The owner to remove
* @param ownerFieldNumber The field in the owner where this object is stored
*/
public void removeEmbeddedOwner( StateManager ownerSM, int ownerFieldNumber ) {
if ( embeddedOwners != null ) {
Iterator<EmbeddedOwnerRelation> iter = embeddedOwners.iterator();
while ( iter.hasNext() ) {
EmbeddedOwnerRelation relation = iter.next();
if ( relation.sm == ownerSM && relation.fieldNumber == ownerFieldNumber ) {
iter.remove();
break;
}
}
}
}

/**
* Accessor for the owning StateManagers for the managed object when stored embedded. Should really only have a single owner but users could, in principle, assign it
* to multiple.
*
* @return StateManagers owning this embedded object.
*/
public org.datanucleus.StateManager[] getEmbeddedOwners() {
if ( embeddedOwners == null ) {
return null;
}
org.datanucleus.StateManager[] owners = new org.datanucleus.StateManager[embeddedOwners.size()];
for ( int i = 0; i < owners.length; i++ ) {
EmbeddedOwnerRelation relation = embeddedOwners.get( i );
owners[i] = relation.sm;
}
return owners;
}



/**
* Wrapper class storing the owning state manager, and the field of the PC managed by the owning state manager where this object is embedded/serialised.
*/
private class EmbeddedOwnerRelation {
private org.datanucleus.StateManager sm;
private int fieldNumber;

/**
*
* @param ownerSM the owner StateManager
* @param ownerFieldNumber the absolute owner field number
*/
public EmbeddedOwnerRelation( org.datanucleus.StateManager ownerSM, int ownerFieldNumber ) {
this.sm = ownerSM;
this.fieldNumber = ownerFieldNumber;
}
}

/**
* Method to replace all loaded SCO fields with wrappers. If the loaded field already uses a SCO wrapper nothing happens to that field.
*/
public void replaceAllLoadedSCOFieldsWithWrappers() {
boolean[] scoMutableFieldFlags = cmd.getSCOMutableMemberFlags();
for ( int i = 0; i < scoMutableFieldFlags.length; i++ ) {
if ( scoMutableFieldFlags[i] && loadedFields[i] ) {
Object value = provideField( i );
if ( ! ( value instanceof SCO ) ) {
wrapSCOField( i, value, false, false, true );
}
}
}
}

/**
* Method to replace all loaded SCO fields that have wrappers with their value. If the loaded field doesnt have a SCO wrapper nothing happens to that field.
*/
public void replaceAllLoadedSCOFieldsWithValues() {
boolean[] scoMutableFieldFlags = cmd.getSCOMutableMemberFlags();
for ( int i = 0; i < scoMutableFieldFlags.length; i++ ) {
if ( scoMutableFieldFlags[i] && loadedFields[i] ) {
Object value = provideField( i );
if ( value instanceof SCO ) {
unwrapSCOField( i, value, true );
}
}
}
}

/**
* Method to unwrap a SCO field (if it is wrapped currently). If the field is not a SCO field will just return the value. If "replaceFieldIfChanged" is set, we
* replace the value in the object with the unwrapped value.
*
* @param fieldNumber The field number
* @param value The value for the field
* @param replaceFieldIfChanged Whether to replace the field value in the object if unwrapping the value
* @return The unwrapped field value
*/
public Object unwrapSCOField( int fieldNumber, Object value, boolean replaceFieldIfChanged ) {
if ( value == null ) {
return value;
}
if ( getSecondClassMutableFields()[fieldNumber] && value instanceof SCO ) {
SCO sco = (SCO) value;

// Not a SCO wrapper, or is a SCO wrapper but not owned by this object
Object unwrappedValue = sco.getValue();
if ( replaceFieldIfChanged ) {
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026030", StringUtils.toJVMIDString( myPC ), myOM.getIdentityAsString( myID ), fmd.getName() ) );
}
replaceField( fieldNumber, unwrappedValue, false );
}
return unwrappedValue;
}
return value;
}

/**
* Method to create a new SCO wrapper for the specified field. If the field is not a SCO field will just return the value.
*
* @param fieldNumber The field number
* @param value The value to initialise the wrapper with (if any)
* @param forInsert Whether the creation of any wrapper should insert this value into the datastore
* @param forUpdate Whether the creation of any wrapper should update the datastore with this value
* @param replaceFieldIfChanged Whether to replace the field in the object if wrapping the value
* @return The wrapper (or original value if not wrappable)
*/
public Object wrapSCOField( int fieldNumber, Object value, boolean forInsert, boolean forUpdate, boolean replaceFieldIfChanged ) {
if ( value == null ) {
// We don't wrap null objects currently
return value;
}

if ( value instanceof PersistenceCapable ) {
// Special case of SCO that we should split into a separate method for clarity, nothing to do with wrapping
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( fmd.getEmbeddedMetaData() != null && fmd.getEmbeddedMetaData().getOwnerMember() != null ) {
// Embedded field, so assign the embedded/serialised object "owner-field" if specified
JDOStateManagerImpl2 subSM = (JDOStateManagerImpl2) myOM.findStateManager( value );
int ownerAbsFieldNum = subSM.cmd.getAbsolutePositionOfMember( fmd.getEmbeddedMetaData().getOwnerMember() );
if ( ownerAbsFieldNum >= 0 ) {
operationalFlags |= MISC_UPDATING_EMBEDDING_FIELDS_WITH_OWNER;
subSM.replaceField( ownerAbsFieldNum, myPC, true );
operationalFlags &= ~MISC_UPDATING_EMBEDDING_FIELDS_WITH_OWNER;
}
}
}

if ( getSecondClassMutableFields()[fieldNumber] ) {
if ( ! ( value instanceof SCO ) || myPC != ( (SCO) value ).getOwner() ) {
// Not a SCO wrapper, or is a SCO wrapper but not owned by this object
AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( replaceFieldIfChanged ) {
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026029", StringUtils.toJVMIDString( myPC ), myOM != null ? myOM.getIdentityAsString( myID )
: myID, fmd.getName() ) );
}
}
return SCOUtils.newSCOInstance( this.getObjectProvider(), fmd, fmd.getType(), ( value != null ? value.getClass() : null ), value, forInsert, forUpdate,
replaceFieldIfChanged );
}
}

return value;
}

// ------------------------- Lifecycle Methods -----------------------------

/**
* Method to mark an object for reachability. Provides the basis for "persistence-by-reachability", but run at commit time only. The reachability algorithm is also
* run at makePersistent, but directly via InsertRequest.
*
* @param reachables List of object ids currently logged as reachable
*/
public void runReachability( Set reachables ) {
if ( reachables == null ) {
return;
}
if ( !reachables.contains( getInternalObjectId() ) ) {
// Make sure all changes are persisted
flush();

if ( isDeleted( myPC ) ) {
// This object is deleted so nothing further will be reachable
return;
}

// This object was enlisted so make sure all of its fields are loaded before continuing
if ( getObjectManager().isEnlistedInTransaction( getInternalObjectId() ) ) {
loadUnloadedFields();
}

if ( NucleusLogger.REACHABILITY.isDebugEnabled() ) {
NucleusLogger.REACHABILITY.debug( LOCALISER.msg( "007000", StringUtils.toJVMIDString( myPC ), getObjectId( myPC ), myLC ) );
}
// Add this object id since not yet reached
reachables.add( getInternalObjectId() );

// Go through all (loaded FetchPlan) fields for reachability using ReachabilityFieldManager
int[] loadedFieldNumbers = getFlagsSetTo( loadedFields, getAllFieldNumbers(), true );
if ( loadedFieldNumbers != null && loadedFieldNumbers.length > 0 ) {
provideFields( loadedFieldNumbers, new ReachabilityFieldManager( this.getObjectProvider(), reachables ) );
}
}
}

/**
* Method to make the object persistent.
*/
public void makePersistent() {
if ( myLC.isDeleted() && !myOM.getOMFContext().getApiAdapter().allowPersistOfDeletedObject() ) {
// API doesnt allow repersist of deleted objects
return;
}

if ( dirty && !myLC.isDeleted() && myLC.isTransactional() && myOM.isDelayDatastoreOperationsEnabled() ) {
// Already provisionally persistent, but delaying til commit so just re-run reachability
// to bring in any new objects that are now reachable
provideFields( cmd.getAllMemberPositions(), new PersistFieldManager( this.getObjectProvider(), false ) );
return;
}

getCallbackHandler().prePersist( myPC );

if ( flushedNew ) {
// With CompoundIdentity bidir relations when the SM is created for this object ("initialiseForPersistentNew") the persist
// of the PK PC fields can cause the flush of this object, and so it is already persisted by the time we ge here
registerTransactional();
return;
}

if ( cmd.isEmbeddedOnly() ) {
// Cant persist an object of this type since can only be embedded
return;
}

// If this is an embedded/serialised object becoming persistent in its own right, assign an identity.
if ( myID == null ) {
setIdentity( false );
}

dirty = true;

if ( myOM.isDelayDatastoreOperationsEnabled() ) {
// Delaying datastore flush til later
myOM.markDirty( this, false );
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026028", StringUtils.toJVMIDString( myPC ) ) );
}
registerTransactional();

if ( myLC.isTransactional() && myLC.isDeleted() ) {
// Re-persist of a previously deleted object
myLC = myLC.transitionMakePersistent( this );
}

// Run reachability on all fields of this PC - JDO2 [12.6.7]
provideFields( cmd.getAllMemberPositions(), new PersistFieldManager( this.getObjectProvider(), false ) );
} else {
// Persist the object and all reachables
internalMakePersistent();
registerTransactional();
}
}

/**
* Method to persist the object to the datastore.
*/
private void internalMakePersistent() {
activity = ActivityState.INSERTING;
boolean[] tmpDirtyFields = dirtyFields.clone();
try {
getCallbackHandler().preStore( myPC ); // This comes after setting the INSERTING flag so we know we are inserting it now
if ( myID == null ) {
setIdentity( true ); // Just in case user is setting it in preStore
}

// in InstanceLifecycleEvents this object could get dirty if a field is changed in preStore or
// postCreate, we clear dirty flags to make sure this object will not be flushed again
clearDirtyFlags();

getStoreManager().getPersistenceHandler().insertObject( objectProvider );

flushedNew = true;

getCallbackHandler().postStore( myPC );

if ( !isEmbedded() ) {
// Update the object in the cache(s) - has version set etc now
myOM.putObjectIntoCache( this );
}
}
catch ( NotYetFlushedException ex ) {
// happening on cyclic relationships
// if not yet flushed error, we rollback dirty fields, so we can retry inserting
dirtyFields = tmpDirtyFields;
myOM.markDirty( this, false );
dirty = true;
// we throw exception, so the owning relationship will mark it's foreign key to update later
throw ex;
}
finally {
activity = ActivityState.NONE;
}
}

/**
* Tests whether this object is being inserted.
*
* @return true if this instance is inserting.
*/
public boolean isInserting() {
return ( activity == ActivityState.INSERTING );
}

/**
* Tests whether this object is being deleted.
*
* @return true if this instance is being deleted.
*/
public boolean isDeleting() {
return ( activity == ActivityState.DELETING );
}

/**
* Tests whether this object is being detached.
*
* @return true if this instance is detaching.
*/
public boolean isDetaching() {
return ( ( operationalFlags & MISC_DETACHING ) != 0 );
}

/**
* Accessor for whether the instance is newly persistent yet hasnt yet been flushed to the datastore.
*
* @return Whether not yet flushed to the datastore
*/
public boolean isWaitingToBeFlushedToDatastore() {
// Return true if object is new and not yet flushed to datastore
return myLC.stateType() == LifeCycleState.P_NEW && !flushedNew;
}

/**
* Change the activity state.
*
* @param activityState the new state
*/
public void changeActivityState( ActivityState activityState ) {
activity = activityState;
if ( activityState == ActivityState.INSERTING_CALLBACKS && insertionNotifyList != null ) {
// Full insertion has just completed so notify all interested parties
synchronized ( insertionNotifyList ) {
Iterator<org.datanucleus.StateManager> notifyIter = insertionNotifyList.iterator();
while ( notifyIter.hasNext() ) {
org.datanucleus.StateManager notifySM = notifyIter.next();
( (JDOStateManagerImpl2) notifySM ).insertionCompleted( this );
}
insertionNotifyList.clear();
insertionNotifyList = null;
}
}
}

/**
* Method to add a notifier that we must contact when we have finished our insertion.
*
* @param sm the state manager
* @param activityState the ActivityState (unused)
*/
public void addInsertionNotifier( org.datanucleus.StateManager sm, ActivityState activityState ) {
// TODO Use the second param to add the StateManager to other lists for other events
if ( insertionNotifyList == null ) {
insertionNotifyList = new ArrayList();
}
insertionNotifyList.add( sm );
}

/**
* Marks the given field as being required to be updated when the specified object has been inserted.
*
* @param pc The Persistable object
* @param fieldNumber Number of the field.
*/
public void updateFieldAfterInsert( Object pc, int fieldNumber ) {
JDOStateManagerImpl2 otherSM = (JDOStateManagerImpl2) myOM.findStateManager( pc );

// Register the other SM to update us when it is inserted
otherSM.addInsertionNotifier( this, ActivityState.INSERTING_CALLBACKS );

// Register that we should update this field when the other SM informs us
if ( fieldsToBeUpdatedAfterObjectInsertion == null ) {
fieldsToBeUpdatedAfterObjectInsertion = new HashMap();
}
FieldContainer cont = fieldsToBeUpdatedAfterObjectInsertion.get( otherSM );
if ( cont == null ) {
cont = new FieldContainer( fieldNumber );
} else {
cont.set( fieldNumber );
}
fieldsToBeUpdatedAfterObjectInsertion.put( otherSM, cont );

if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026021", cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber ).getFullFieldName(), StringUtils
.toJVMIDString( myPC ), StringUtils.toJVMIDString( pc ) ) );
}
}

private boolean updatingForPostInsert = false;

/**
* Method called by another StateManager when this object has registered that it needs to know when the other object has been inserted.
*
* @param sm State Manager of the other object that has just been inserted
*/
void insertionCompleted( org.datanucleus.StateManager sm ) {
if ( fieldsToBeUpdatedAfterObjectInsertion == null ) {
return;
}

// Go through our insertion update list and mark all required fields as dirty
FieldContainer fldCont = fieldsToBeUpdatedAfterObjectInsertion.get( sm );
if ( fldCont != null ) {
dirty = true;
int[] fieldsToUpdate = fldCont.getFields();
for ( int i = 0; i < fieldsToUpdate.length; i++ ) {
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026022", cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldsToUpdate[i] ).getFullFieldName(),
myOM.getIdentityAsString( myID ), StringUtils.toJVMIDString( sm.getObject() ) ) );
}
dirtyFields[fieldsToUpdate[i]] = true;
}
fieldsToBeUpdatedAfterObjectInsertion.remove( sm );
if ( fieldsToBeUpdatedAfterObjectInsertion.size() == 0 ) {
fieldsToBeUpdatedAfterObjectInsertion = null;
}

try {
updatingForPostInsert = true;

// Perform our update
flush();
}
finally {
updatingForPostInsert = false;
}
}
}

/**
* Convenience method to return if we are in the phase of performing postInsert updates due to related objects having been inserted.
*
* @return Whether we are updating for postInsert
*/
public boolean isUpdatingFieldForPostInsert() {
return updatingForPostInsert;
}

/**
* Method to set an associated value stored with this object. This is for a situation such as in ORM where this object can have an "external" foreign-key provided by
* an owning object (e.g 1-N uni relation and this is the element with no knowledge of the owner, so the associated value is the FK value).
*
* @param key Key for the value
* @param value The associated value
*/
public void setAssociatedValue( Object key, Object value ) {
if ( associatedValuesMap == null ) {
associatedValuesMap = new HashMap();
}
if (value == null) {
associatedValuesMap.remove( key );
} else {
associatedValuesMap.put( key, value );
}
}

/**
* Accessor for an associated value stored with this object. This is for a situation such as in ORM where this object can have an "external" foreign-key provided by
* an owning object (e.g 1-N uni relation and this is the element with no knowledge of the owner, so the associated value is the FK value).
*
* @param key Key for the value
* @return The associated value
*/
public Object getAssociatedValue( Object key ) {
if ( associatedValuesMap == null ) {
return null;
}
return associatedValuesMap.get( key );
}

public void clearAssociatedValues( ) {
if ( associatedValuesMap != null ) {
associatedValuesMap.clear();
}
}


/** Private class storing the fields to be updated for a StateManager, when it is inserted */
private class FieldContainer {
boolean[] fieldsToUpdate = new boolean[getAllFieldNumbers().length];

/**
* Constructor
*
* @param fieldNumber the absolute field number to flag true
*/
public FieldContainer( int fieldNumber ) {
fieldsToUpdate[fieldNumber] = true;
}

/**
* Flag to true the <code>fieldNumber</code>
*
* @param fieldNumber the absolute field number to flag true
*/
public void set( int fieldNumber ) {
fieldsToUpdate[fieldNumber] = true;
}

/**
* Array with absolute field numbers with true flag
*
* @return array of absolute field numbers
*/
public int[] getFields() {
return getFlagsSetTo( fieldsToUpdate, true );
}
}

/**
* Method to change the object state to transactional.
*/
public void makeTransactional() {
preStateChange();
try {
if ( myLC == null ) {
initializeSM( LifeCycleState.T_CLEAN );
setRestoreValues( true );
} else {
myLC = myLC.transitionMakeTransactional( this, true );
}
}
finally {
postStateChange();
}
}

/**
* Method to change the object state to transient.
*
* @param state Object containing the state of any fetchplan processing
*/
public void makeTransient( FetchPlanState state ) {
if ( ( ( operationalFlags & MISC_MAKING_TRANSIENT ) != 0 ) ) {
return; // In the process of becoming transient
}

try {
operationalFlags |= MISC_MAKING_TRANSIENT;
if ( state == null ) {
// No FetchPlan in use so just unset the owner of all loaded SCO fields
int[] fieldNumbers = getFlagsSetTo( loadedFields, getSecondClassMutableFieldNumbers(), true );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
provideFields( fieldNumbers, new UnsetOwners() );
}
} else {
// Make all loaded SCO fields transient appropriate to this fetch plan
loadUnloadedFieldsInFetchPlan();
int[] fieldNumbers = getFlagsSetTo( loadedFields, getAllFieldNumbers(), true );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
// TODO Fix this to just access the fields of the FieldManager yet this actually does a replaceField
replaceFields( fieldNumbers, new MakeTransientFieldManager( this.getObjectProvider(), getSecondClassMutableFields(), myFP, state ) );
}
}

preStateChange();
try {
myLC = myLC.transitionMakeTransient( this, state != null, myOM.isRunningDetachAllOnCommit() );
}
finally {
postStateChange();
}
}
finally {
operationalFlags &= ~MISC_MAKING_TRANSIENT;
}
}

/**
* Method to detach this object. If the object is detachable then it will be migrated to DETACHED state, otherwise will migrate to TRANSIENT. Used by
* "DetachAllOnCommit"/"DetachAllOnRollback"
*
* @param state State for the detachment process
*/
public void detach( FetchPlanState state ) {
if ( myOM == null ) {
return;
}

ApiAdapter api = myOM.getApiAdapter();
if ( myLC.isDeleted() || api.isDetached( myPC ) || ( ( operationalFlags & MISC_DETACHING ) != 0 ) ) {
// Already deleted, detached or being detached
return;
}

// Check if detachable ... if so then we detach a copy, otherwise we return a transient copy
boolean detachable = api.isDetachable( myPC );
if ( detachable ) {
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "010009", StringUtils.toJVMIDString( myPC ), "" + state.getCurrentFetchDepth() ) );
}

// Call any "pre-detach" listeners
getCallbackHandler().preDetach( myPC );
}

try {
operationalFlags |= MISC_DETACHING;

// Handle any field loading/unloading before the detach
if ( ( myOM.getFetchPlan().getDetachmentOptions() & FetchPlan.DETACH_LOAD_FIELDS ) != 0 ) {
// Load any unloaded fetch-plan fields
loadUnloadedFieldsInFetchPlan();
}
if ( ( myOM.getFetchPlan().getDetachmentOptions() & FetchPlan.DETACH_UNLOAD_FIELDS ) != 0 ) {
// Unload any loaded fetch-plan fields that aren't in the current fetch plan
unloadNonFetchPlanFields();

// Remove the values from the detached object - not required by the spec
int[] unloadedFields = getFlagsSetTo( loadedFields, getAllFieldNumbers(), false );
if ( unloadedFields != null && unloadedFields.length > 0 ) {
PersistenceCapable dummyPC = myPC.jdoNewInstance( this );
myPC.jdoCopyFields( dummyPC, unloadedFields );
replaceStateManager( dummyPC, null );
}
}

// Detach all (loaded) fields in the FetchPlan
FieldManager detachFieldManager = new DetachFieldManager( this.getObjectProvider(), getSecondClassMutableFields(), myFP, state, false );
for ( int i = 0; i < loadedFields.length; i++ ) {
if ( loadedFields[i] ) {
try {
// Just fetch the field since we are usually called in postCommit() so dont want to update it
detachFieldManager.fetchObjectField( i );
}
catch ( EndOfFetchPlanGraphException eofpge ) {
Object value = provideField( i );
if ( api.isPersistable( value ) ) {
// PC field beyond end of graph
org.datanucleus.StateManager valueSM = myOM.findStateManager( value );
if ( !api.isDetached( value ) && ! ( valueSM != null && valueSM.isDetaching() ) ) {
// Field value is not detached or being detached so unload it
String fieldName = cmd.getMetaDataForManagedMemberAtAbsolutePosition( i ).getName();
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026032", StringUtils.toJVMIDString( myPC ), myOM.getIdentityAsString( myID ),
fieldName ) );
}
unloadField( fieldName );
}
}
// TODO What if we have collection/map that includes some objects that are not detached?
// Currently we just leave as persistent etc but should we????
// The problem is that with 1-N bidir fields we could unload the field incorrectly
}
}
}

if ( detachable ) {
// Migrate the lifecycle state to DETACHED_CLEAN
myLC = myLC.transitionDetach( this );

// Update the object with its detached state
myPC.jdoReplaceFlags();
( (Detachable) myPC ).jdoReplaceDetachedState();

// Call any "post-detach" listeners
getCallbackHandler().postDetach( myPC, myPC ); // there is no copy, so give the same object

PersistenceCapable toCheckPC = myPC;
Object toCheckID = myID;
disconnect();

if ( !toCheckPC.jdoIsDetached() ) {
// Sanity check on the objects detached state
throw new NucleusUserException( LOCALISER.msg( "026025", toCheckPC.getClass().getName(), toCheckID ) );
}
} else {
// Warn the user since they selected detachAllOnCommit
NucleusLogger.PERSISTENCE.warn( LOCALISER.msg( "026031", myPC.getClass().getName(), myOM.getIdentityAsString( myID ) ) );

// Make the object transient
makeTransient( null );
}
}
finally {
operationalFlags &= ~MISC_DETACHING;
}
}

/**
* Method to make detached copy of this instance If the object is detachable then the copy will be migrated to DETACHED state, otherwise will migrate the copy to
* TRANSIENT. Used by "ObjectManager.detachObjectCopy()".
*
* @param state State for the detachment process
* @return the detached PersistenceCapable instance
*/
public Object detachCopy( FetchPlanState state ) {
if ( myLC.isDeleted() ) {
throw new NucleusUserException( LOCALISER.msg( "026023", myPC.getClass().getName(), myID ) );
}
if ( myOM.getApiAdapter().isDetached( myPC ) ) {
throw new NucleusUserException( LOCALISER.msg( "026024", myPC.getClass().getName(), myID ) );
}
if ( dirty ) {
myOM.flushInternal( false );
}
if ( ( ( operationalFlags & MISC_DETACHING ) != 0 ) ) {
// Object in the process of detaching (recursive) so return the object which will be the detached object
return referencedPC;
}

// Look for an existing detached copy
DetachState detachState = (DetachState) state;
DetachState.Entry existingDetached = detachState.getDetachedCopyEntry( myPC );

PersistenceCapable detachedPC;
if ( existingDetached == null ) {
// No existing detached copy - create new one
detachedPC = myPC.jdoNewInstance( this );
detachState.setDetachedCopyEntry( myPC, detachedPC );
} else {
// Found one - if it's sufficient for current FetchPlanState, return it immediately
detachedPC = (PersistenceCapable) existingDetached.getDetachedCopyObject();
if ( existingDetached.checkCurrentState() ) {
return detachedPC;
}

// Need to process the detached copy using current FetchPlanState
}

referencedPC = detachedPC;

// Check if detachable ... if so then we detach a copy, otherwise we return a transient copy
boolean detachable = myOM.getApiAdapter().isDetachable( myPC );

// make sure a detaching PC is not read by another thread while we are detaching
synchronized ( referencedPC ) {
if ( detachable ) {
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "010010", StringUtils.toJVMIDString( myPC ), "" + state.getCurrentFetchDepth(), StringUtils
.toJVMIDString( detachedPC ) ) );
}

// Call any "pre-detach" listeners
getCallbackHandler().preDetach( myPC );
}
try {
operationalFlags |= MISC_DETACHING;

// Handle any field loading/unloading before the detach
if ( ( myOM.getFetchPlan().getDetachmentOptions() & FetchPlan.DETACH_LOAD_FIELDS ) != 0 ) {
// Load any unloaded fetch-plan fields
loadUnloadedFieldsInFetchPlan();
}

if ( myLC == myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.HOLLOW )
|| myLC == myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_NONTRANS ) ) {
// Migrate any HOLLOW/P_NONTRANS to P_CLEAN etc
myLC = myLC.transitionReadField( this, true );
}

// Create a SM for our copy object
JDOStateManagerImpl2 smDetachedPC = new JDOStateManagerImpl2( myOM, cmd );
smDetachedPC.initialiseForDetached( detachedPC, getExternalObjectId( myPC ), getVersion( myPC ) );
smDetachedPC.referencedPC = myPC;

// If detached copy already existed, take note of fields previously loaded
if ( existingDetached != null ) {
smDetachedPC.retrieveDetachState( smDetachedPC );
}

smDetachedPC.replaceFields( getFieldsNumbersToDetach(), new DetachFieldManager( this.getObjectProvider(), getSecondClassMutableFields(), myFP, state,
true ) );

smDetachedPC.referencedPC = null;
if ( detachable ) {
// Update the object with its detached state - not to be confused with the "state" object above
detachedPC.jdoReplaceFlags();
( (Detachable) detachedPC ).jdoReplaceDetachedState();
} else {
smDetachedPC.makeTransient( null );
}

// Remove its StateManager since now detached or transient
replaceStateManager( detachedPC, null );
}
catch ( Exception e ) {
// What could possible be thrown here ?
NucleusLogger.PERSISTENCE.debug( "DETACH ERROR : Error thrown while detaching " + StringUtils.toJVMIDString( myPC ) + " (id=" + myID + ")", e );
}
finally {
operationalFlags &= ~MISC_DETACHING;
referencedPC = null;
}

if ( detachable && !myOM.getApiAdapter().isDetached( detachedPC ) ) {
// Sanity check on the objects detached state
throw new NucleusUserException( LOCALISER.msg( "026025", detachedPC.getClass().getName(), myID ) );
}

if ( detachable ) {
// Call any "post-detach" listeners
getCallbackHandler().postDetach( myPC, detachedPC );
}
}
return detachedPC;
}

/**
* Return an array of field numbers that must be included in the detached object
*
* @return the field numbers array
*/
private int[] getFieldsNumbersToDetach() {
// This will cause the detach of any other fields in the FetchPlan.
int[] fieldsToDetach = myFP.getMemberNumbers();
if ( ( myOM.getFetchPlan().getDetachmentOptions() & FetchPlan.DETACH_UNLOAD_FIELDS ) == 0 ) {
// Detach fetch-plan fields plus any other loaded fields
int[] allFieldNumbers = getAllFieldNumbers();
int[] loadedFieldNumbers = getFlagsSetTo( loadedFields, allFieldNumbers, true );
if ( loadedFieldNumbers != null && loadedFieldNumbers.length > 0 ) {
boolean[] flds = new boolean[allFieldNumbers.length];
for ( int i = 0; i < fieldsToDetach.length; i++ ) {
flds[fieldsToDetach[i]] = true;
}
for ( int i = 0; i < loadedFieldNumbers.length; i++ ) {
flds[loadedFieldNumbers[i]] = true;
}
fieldsToDetach = getFlagsSetTo( flds, true );
}
}
return fieldsToDetach;
}

/**
* Accessor for the referenced PC object when we are attaching or detaching. When attaching and this is the detached object this returns the newly attached object.
* When attaching and this is the newly attached object this returns the detached object. When detaching and this is the newly detached object this returns the
* attached object. When detaching and this is the attached object this returns the newly detached object.
*
* @return The referenced object (or null).
*/
public Object getReferencedPC() {
return referencedPC;
}

/**
* Method to attach the object managed by this StateManager.
*
* @param embedded Whether it is embedded
*/
public void attach( boolean embedded ) {
if ( ( ( operationalFlags & MISC_ATTACHING ) != 0 ) ) {
return;
}

operationalFlags |= MISC_ATTACHING;
try {
// Check if the object is already persisted
boolean persistent = false;
if ( embedded ) {
persistent = true;
} else {
if ( !myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.attachSameDatastore" ) ) {
// We cant assume that this object was detached from this datastore so we check it
try {
locate();
persistent = true;
}
catch ( NucleusObjectNotFoundException onfe ) {
// Not currently present!
}
} else {
// Assumed detached from this datastore
persistent = true;
}
}

// Call any "pre-attach" listeners
getCallbackHandler().preAttach( myPC );

// Retrieve the updated values from the detached object
replaceStateManager( myPC, this );
retrieveDetachState( this );

if ( !persistent ) {
// Persist the object into this datastore first
makePersistent();
}

// Migrate the lifecycle state to persistent
myLC = myLC.transitionAttach( this );

// Make sure the attached object goes in the cache
// [would not get cached when not changed if we didnt do this here]
myOM.putObjectIntoCache( this );

int[] attachFieldNumbers = getFieldNumbersOfLoadedOrDirtyFields( loadedFields, dirtyFields );
if ( attachFieldNumbers != null ) {
// Only update the fields that were detached, and only update them if there are any to update
provideFields( attachFieldNumbers, new AttachFieldManager( this.getObjectProvider(), getSecondClassMutableFields(), dirtyFields, persistent, true, false ) );
}

// Call any "post-attach" listeners
getCallbackHandler().postAttach( myPC, myPC );
}
finally {
operationalFlags &= ~MISC_ATTACHING;
}
}

/**
* Method to attach a copy of the detached persistable instance and return the (attached) copy.
*
* @param obj the detached persistable instance to be attached
* @param embedded Whether the object is stored embedded/serialised in another object
* @return The attached copy
*/
public Object attachCopy( Object obj, boolean embedded ) {
if ( ( ( operationalFlags & MISC_ATTACHING ) != 0 ) ) {
return myPC;
}
operationalFlags |= MISC_ATTACHING;

PersistenceCapable detachedPC = (PersistenceCapable) obj;
try {
// Check if the object is already persisted
boolean persistent = false;
if ( embedded ) {
persistent = true;
} else {
if ( !myOM.getOMFContext().getPersistenceConfiguration().getBooleanProperty( "datanucleus.attachSameDatastore" ) ) {
// We cant assume that this object was detached from this datastore so we check it
try {
locate();
persistent = true;
}
catch ( NucleusObjectNotFoundException onfe ) {
// Not currently present!
}
} else {
// Assumed detached from this datastore
persistent = true;
}
}

// Call any "pre-attach" listeners
getCallbackHandler().preAttach( detachedPC );

if ( myOM.getApiAdapter().isDeleted( detachedPC ) ) {
// The detached object has been deleted
myLC = myLC.transitionDeletePersistent( this );
}

if ( !myOM.getTransaction().getOptimistic()
&& ( myLC == myOM.getApiAdapter().getLifeCycleState( LifeCycleState.HOLLOW ) || myLC == myOM.getApiAdapter().getLifeCycleState(
LifeCycleState.P_NONTRANS ) ) ) {
// Pessimistic txns and in HOLLOW/P_NONTRANS, so move to P_CLEAN
// TODO Move this into the lifecycle state classes as a "transitionAttach"
myLC = myLC.transitionMakeTransactional( this, persistent );
}

if ( persistent ) {
// Make sure that all non-container SCO fields are loaded so we can make valid dirty checks
// for whether these fields have been updated whilst detached. The detached object doesnt know if the contents
// have been changed.
loadSCONonContainerFields();
}

// Add a state manager to the detached PC so that we can retrieve its detached state
JDOStateManagerImpl2 smDetachedPC = new JDOStateManagerImpl2( myOM, cmd );
smDetachedPC.initialiseForDetached( detachedPC, getExternalObjectId( detachedPC ), null );

// Cross-reference the attached and detached objects for the attach process
smDetachedPC.referencedPC = myPC;
this.referencedPC = detachedPC;

// Retrieve the updated values from the detached object
retrieveDetachState( smDetachedPC );

if ( !persistent ) {
// Object is not yet persisted! so make it persistent

// Make sure all field values in the attach object are ready for inserts (but dont trigger any cascade attaches)
internalAttachCopy( this, smDetachedPC, smDetachedPC.loadedFields, smDetachedPC.dirtyFields, persistent, smDetachedPC.myVersion, false );

makePersistent();
}

// Go through all related fields and attach them (including relationships)
internalAttachCopy( this, smDetachedPC, smDetachedPC.loadedFields, smDetachedPC.dirtyFields, persistent, smDetachedPC.myVersion, true );

// Remove the state manager from the detached PC
replaceStateManager( detachedPC, null );

// Remove the corss-referencing now we have finished the attach process
smDetachedPC.referencedPC = null;
this.referencedPC = null;

// Call any "post-attach" listeners
getCallbackHandler().postAttach( myPC, detachedPC );
}
catch ( NucleusException ne ) {
// Log any errors in the attach
NucleusLogger.PERSISTENCE.debug( "Unexpected exception thrown in attach", ne );
throw ne;
}
finally {
operationalFlags &= ~MISC_ATTACHING;
}
return myPC;
}

/**
* Attach the fields of a persistent object.
*
* @param sm StateManager for the attached object.
* @param smDetached StateManager for the detached object.
* @param loadedFields Fields that were detached with the object
* @param dirtyFields Fields that have been modified while detached
* @param persistent whether the object is already persistent
* @param version the version
* @param cascade Whether to cascade the attach to related fields
*/
private void internalAttachCopy( org.datanucleus.StateManager sm, org.datanucleus.StateManager smDetached, boolean[] loadedFields, boolean[] dirtyFields,
boolean persistent, Object version, boolean cascade ) {
// Need to take all loaded fields plus all modified fields
// (maybe some werent detached but have been modified) and attach them
int[] attachFieldNumbers = getFieldNumbersOfLoadedOrDirtyFields( loadedFields, dirtyFields );
sm.setVersion( version );
if ( attachFieldNumbers != null ) {
// Only update the fields that were detached, and only update them if there are any to update
boolean[] fieldsToAttach = dirtyFields;
String attachPolicy = myOM.getAttachPolicy();
if ( attachPolicy.equals( "attach-all" ) ) {
// Attach all loaded+dirty fields
fieldsToAttach = new boolean[dirtyFields.length];
for ( int i = 0; i < dirtyFields.length; i++ ) {
fieldsToAttach[i] = ( loadedFields[i] || dirtyFields[i] );
}
smDetached.provideFields( attachFieldNumbers, new AttachFieldManager( sm.getObjectProvider(), getSecondClassMutableFields(), fieldsToAttach, persistent,
cascade, true ) );
} else {
// Attach all dirty fields
smDetached.provideFields( attachFieldNumbers, new AttachFieldManager( sm.getObjectProvider(), getSecondClassMutableFields(), dirtyFields, persistent,
cascade, true ) );
}
}
}

/**
* Convenience accessor to return the field numbers for the input loaded and dirty field arrays.
*
* @param loadedFields Fields that were detached with the object
* @param dirtyFields Fields that have been modified while detached
*/
private int[] getFieldNumbersOfLoadedOrDirtyFields( boolean[] loadedFields, boolean[] dirtyFields ) {
// Find the number of fields that are loaded or dirty
int numFields = 0;
for ( int i = 0; i < loadedFields.length; i++ ) {
if ( loadedFields[i] || dirtyFields[i] ) {
numFields++ ;
}
}

int[] fieldNumbers = new int[numFields];
int n = 0;
for ( int i = 0; i < loadedFields.length; i++ ) {
if ( loadedFields[i] || dirtyFields[i] ) {
fieldNumbers[n++ ] = getAllFieldNumbers()[i];
}
}
return fieldNumbers;
}

boolean becomingDeleted = false;

/**
* Method to delete the object from persistence.
*/
public void deletePersistent() {
if ( !myLC.isDeleted() ) {
if ( myOM.isDelayDatastoreOperationsEnabled() ) {
// Optimistic transactions, with all updates delayed til flush/commit

// Call any lifecycle listeners waiting for this event
getCallbackHandler().preDelete( myPC );

// Delay deletion until flush/commit so run reachability now to tag all reachable instances as necessary
myOM.markDirty( this, false );

// Reachability
if ( myLC.stateType() == LifeCycleState.P_CLEAN || myLC.stateType() == LifeCycleState.P_DIRTY || myLC.stateType() == LifeCycleState.HOLLOW
|| myLC.stateType() == LifeCycleState.P_NONTRANS || myLC.stateType() == LifeCycleState.P_NONTRANS_DIRTY ) {
// Make sure all fields are loaded so we can perform reachability
loadUnloadedFields();
}
becomingDeleted = true;
provideFields( getAllFieldNumbers(), new DeleteFieldManager( this.getObjectProvider() ) );

// Update lifecycle state (after running reachability since it will unload all fields)
dirty = true;
preStateChange();
try {
// Keep "loadedFields" settings til after delete is complete to save reloading
preDeleteLoadedFields = new boolean[loadedFields.length];
for ( int i = 0; i < preDeleteLoadedFields.length; i++ ) {
preDeleteLoadedFields[i] = loadedFields[i];
}

myLC = myLC.transitionDeletePersistent( this );
}
finally {
becomingDeleted = false;
postStateChange();
}
} else {
// Datastore transactions, with all updates processed now

// Call any lifecycle listeners waiting for this event.
getCallbackHandler().preDelete( myPC );

// Update lifecycle state
dirty = true;
preStateChange();
try {
// Keep "loadedFields" settings til after delete is complete to save reloading
preDeleteLoadedFields = new boolean[loadedFields.length];
for ( int i = 0; i < preDeleteLoadedFields.length; i++ ) {
preDeleteLoadedFields[i] = loadedFields[i];
}

myLC = myLC.transitionDeletePersistent( this );
}
finally {
postStateChange();
}

// Delete the object from the datastore (includes reachability)
internalDeletePersistent();

// Call any lifecycle listeners waiting for this event.
getCallbackHandler().postDelete( myPC );
}
}
}

/**
* Whether this object is moving to a deleted state.
*
* @return Whether the object will be moved into a deleted state during this operation
*/
public boolean becomingDeleted() {
return becomingDeleted;
}

/** Copy of the "loadedFields" just before delete was started to avoid reload during delete */
boolean[] preDeleteLoadedFields = null;

/**
* Method to delete the object from the datastore.
*/
private void internalDeletePersistent() {
if ( isDeleting() ) {
throw new NucleusUserException( LOCALISER.msg( "026008" ) );
}

activity = ActivityState.DELETING;
try {
if ( dirty ) {
clearDirtyFlags();

// Clear the PM's knowledge of our being dirty. This calls flush() which does nothing
myOM.flushInternal( false );
}

getStoreManager().getPersistenceHandler().deleteObject( objectProvider );

preDeleteLoadedFields = null;
}
finally {
activity = ActivityState.NONE;
}
}

/**
* Locate the object in the datastore.
*
* @throws NucleusObjectNotFoundException if the object doesnt exist.
*/
public void locate() {
// Validate the object existence
getStoreManager().getPersistenceHandler().locateObject( objectProvider );
}


/**
* Nullify fields with reference to PersistenceCapable or SCO instances
*/
public void nullifyFields() {
if ( !myLC.isDeleted() && !myOM.getApiAdapter().isDetached( myPC ) ) {
// Update any relationships for fields of this object that aren't dependent
replaceFields( getNonPrimaryKeyFieldNumbers(), new NullifyRelationFieldManager( this.getObjectProvider() ) );
flush();
}
}

/*
* (non-Javadoc)
*
* @see org.datanucleus.StateManager#markForValidation()
*/
public void markForInheritanceValidation() {
this.needsInheritanceValidating = true;
}

/**
* Validates whether the persistence capable instance exists in the datastore. If the instance doesn't exist in the datastore, this method will fail raising a
* NucleusObjectNotFoundException. If the object is transactional then does nothing. If the object has unloaded (non-SCO, non-PK) fetch plan fields then fetches them.
* Else it checks the existence of the object in the datastore.
*/
public void validate() {
if ( !myLC.isTransactional() ) {
// Find all FetchPlan fields that are not PK, not SCO and still not loaded
int[] fieldNumbers = getFlagsSetTo( loadedFields, myFP.getMemberNumbers(), false );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
fieldNumbers = getFlagsSetTo( getNonPrimaryKeyFields(), fieldNumbers, true );
}
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
fieldNumbers = getFlagsSetTo( getSecondClassMutableFields(), fieldNumbers, false );
}

boolean versionNeedsLoading = false;
if ( cmd.hasVersionStrategy() && transactionalVersion == null ) {
versionNeedsLoading = true;
}
if ( ( fieldNumbers != null && fieldNumbers.length > 0 ) || versionNeedsLoading ) {
// Some fetch plan fields, or the version are not loaded so try to load them, and this by itself
// validates the existence. Loads the fields in the current FetchPlan (JDO2 spec 12.6.5)
transitionReadField( false );

fieldNumbers = myFP.getMemberNumbers();
if ( fieldNumbers != null || versionNeedsLoading ) {
boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
setTransactionalVersion( null ); // Make sure we get the latest version
loadFieldsFromDatastore( fieldNumbers );
if ( callPostLoad ) {
postLoad();
}
}
} else {
// Validate the object existence
locate();
transitionReadField( false );
}
}
}

// --------------------------- Process Methods -----------------------------

/**
* Method called before a change in state.
*/
protected void preStateChange() {
changingState = true;
}

/**
* Method called after a change in state.
*/
protected void postStateChange() {
changingState = false;
if ( postLoadPending && isFetchPlanLoaded() ) {
// Only call postLoad when all FetchPlan fields are loaded
postLoadPending = false;
postLoad();
}
}

/**
* Method called before a write of the specified field.
*
* @param field The field to write
* @return true if the field was already dirty before
*/
private boolean preWriteField( int field ) {
boolean wasDirty = dirty;
/*
* If we're writing a field in the process of inserting it must be due to jdoPreStore(). We haven't actually done the INSERT yet so we don't want to mark anything
* as dirty, which would make us want to do an UPDATE later.
*/
if ( activity != ActivityState.INSERTING && activity != ActivityState.INSERTING_CALLBACKS ) {
// TODO dirty already??? this is not correct, only gets dirty after state transition
// dirty = true;
if ( !wasDirty ) // (only do it for first dirty event).
{
// Call any lifecycle listeners waiting for this event
getCallbackHandler().preDirty( myPC );
}

transitionWriteField();

dirty = true;
dirtyFields[field] = true;
loadedFields[field] = true;
}
return wasDirty;
}

/**
* Method called after the write of a field.
*
* @param wasDirty whether before writing this field the pc was dirty
*/
private void postWriteField( boolean wasDirty ) {
if ( dirty && !wasDirty ) // (only do it for first dirty event).
{
// Call any lifecycle listeners waiting for this event
getCallbackHandler().postDirty( myPC );
}

if ( activity == ActivityState.NONE && ( ( operationalFlags & MISC_FLUSHING ) == 0 ) && ! ( myLC.isTransactional() && !myLC.isPersistent() ) ) {
if ( ( ( operationalFlags & MISC_DETACHING ) != 0 ) && referencedPC == null ) {
// detachAllOnCommit caused a field to be dirty so ignore it
return;
} else {
// Not during flush, and not transactional-transient, and not inserting - so mark as dirty
myOM.markDirty( this, true );
}
}
}

/**
* Called whenever the default fetch group fields have all been loaded. Updates jdoFlags and calls jdoPostLoad() as appropriate.
* <p>
* If it's called in the midst of a life-cycle transition both actions will be deferred until the transition is complete. <em>This deferral is important</em>. Without
* it, we could enter user code (jdoPostLoad()) while still making a state transition, and that way lies madness.
* <p>
* As an example, consider a jdoPostLoad() that calls other enhanced methods that read fields (jdoPostLoad() itself is not enhanced). A P_NONTRANS object accessed
* within a transaction would produce the following infinite loop:
* <p>
* <blockquote>
*
* <pre>
*
* isLoaded()
* transitionReadField()
* refreshLoadedFields()
* jdoPostLoad()
* isLoaded()
* ...
*
* </pre>
*
* </blockquote>
* <p>
* because the transition from P_NONTRANS to P_CLEAN can never be completed.
*/
private void postLoad() {
if ( changingState ) {
postLoadPending = true;
} else {
/*
* A transactional object whose DFG fields are loaded does not need to contact us in order to read those fields, so we can safely set READ_OK.
*
* A non-transactional object needs to notify us on all field reads so that we can decide whether or not any transition should occur, so we leave the flags at
* LOAD_REQUIRED.
*/
if ( jdoDfgFlags == PersistenceCapable.LOAD_REQUIRED && myLC.isTransactional() ) {
jdoDfgFlags = PersistenceCapable.READ_OK;
myPC.jdoReplaceFlags();
}

getCallbackHandler().postLoad( myPC );
}
}

/** Flag to signify that we are currently storing the PC object, so we dont detach it on any serialisation. */
private boolean storingPC = false;

/**
* Method to set the storing PC flag.
*/
public void setStoringPC() {
storingPC = true;
}

/**
* Method to unset the storing PC flag.
*/
public void unsetStoringPC() {
storingPC = false;
}

/**
* Guarantee that the serializable transactional and persistent fields are loaded into the instance. This method is called by the generated jdoPreSerialize method
* prior to serialization of the instance.
*
* @param pc the calling PersistenceCapable instance
*/
public void preSerialize( PersistenceCapable pc ) {
if ( disconnectClone( pc ) ) {
return;
}

// Retrieve all fields prior to serialisation
retrieve( false );

myLC = myLC.transitionSerialize( this );

if ( !storingPC && pc instanceof Detachable ) {
if ( !myLC.isDeleted() && myLC.isPersistent() ) {
if ( myLC.isDirty() ) {
flush();
}

// Normal PC Detachable object being serialised so load up the detached state into the instance
// JDO2 spec "For Detachable classes, the jdoPreSerialize method must also initialize the jdoDetachedState
// instance so that the detached state is serialized along with the instance."
( (Detachable) pc ).jdoReplaceDetachedState();
}
}
}

/**
* Flushes any outstanding changes to the object to the datastore. This will process :-
* <ul>
* <li>Any objects that have been marked as provisionally persistent yet havent been flushed to the datastore.</li>
* <li>Any objects that have been marked as provisionally deleted yet havent been flushed to the datastore.</li>
* <li>Any fields that have been updated.</li>
* </ul>
*/
public void flush() {
if ( dirty ) {
if ( ( ( operationalFlags & MISC_FLUSHING ) != 0 ) ) {
// In the case of persisting a new object using autoincrement id within an optimistic
// transaction, flush() will initially be called at the point of recognising that the
// id is generated in the datastore, and will then be called again at the point of doing
// the InsertRequest for the object itself. Just return since we are flushing right now
return;
}
if ( activity == ActivityState.INSERTING || activity == ActivityState.INSERTING_CALLBACKS ) {
return;
}

operationalFlags |= MISC_FLUSHING;
try {
if ( myLC.stateType() == LifeCycleState.P_NEW && !flushedNew ) {
// Newly persisted object but not yet flushed to datastore (e.g optimistic transactions)
if ( !isEmbedded() ) {
// internalMakePersistent does preStore, postStore
internalMakePersistent();
} else {
getCallbackHandler().preStore( myPC );
if ( myID == null ) {
setIdentity( true ); // Just in case user is setting it in preStore
}

if (cacheRelationships) {
// remove the deleted object from level 2 cache.
myOM.removeObjectFromLevel2Cache( myID );

clearAssociatedValues();
}

getCallbackHandler().postStore( myPC );
}
dirty = false;
} else if ( myLC.stateType() == LifeCycleState.P_DELETED ) {
// Object marked as deleted but not yet deleted from datastore
getCallbackHandler().preDelete( myPC );
if ( !isEmbedded() ) {
internalDeletePersistent();
if (cacheRelationships) {
// remove the deleted object from level 2 cache.
clearAssociatedValues();

myOM.removeObjectFromLevel2Cache( myID );
}
}
getCallbackHandler().postDelete( myPC );
} else if ( myLC.stateType() == LifeCycleState.P_NEW_DELETED ) {
// Newly persisted object marked as deleted but not yet deleted from datastore
if ( flushedNew ) {
// Only delete it if it was actually persisted into the datastore
getCallbackHandler().preDelete( myPC );
if ( !isEmbedded() ) {
internalDeletePersistent();
if (cacheRelationships) {
// remove the deleted object from level 2 cache.
myOM.removeObjectFromLevel2Cache( myID );
clearAssociatedValues();
}
}
flushedNew = false; // No longer newly persisted flushed object since has been deleted
getCallbackHandler().postDelete( myPC );
} else {
// Was never persisted to the datastore so nothing to do
dirty = false;
}
} else {
// Updated object with changes to flush to datastore
if ( !isDeleting() ) {
getCallbackHandler().preStore( myPC );
if ( myID == null ) {
setIdentity( true ); // Just in case user is setting it in preStore
}
}

int[] dirtyFieldNumbers = getFlagsSetTo( dirtyFields, true );
if ( dirtyFieldNumbers == null ) {
throw new NucleusException( LOCALISER.msg( "026010" ) ).setFatal();
}
if ( !isEmbedded() ) {
getStoreManager().getPersistenceHandler().updateObject( objectProvider, dirtyFieldNumbers );
if (cacheRelationships) {
// since there was an update remove it from cache in order to avoid optimistic lock exceptions.
myOM.removeObjectFromLevel2Cache( myID );
clearAssociatedValues();
}
// Update the object in the cache(s)
myOM.putObjectIntoCache( this );
}

clearDirtyFlags();

getCallbackHandler().postStore( myPC );
}
}
finally {
operationalFlags &= ~MISC_FLUSHING;
}
}
}

/**
* Initialize SM reference in PC and Oid
*
* @param newState The new StateManager state
*/
private void initializeSM( int newState ) {
final JDOStateManagerImpl2 thisSM = this;
myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( newState );

try {
if ( myLC.isPersistent() ) {
myOM.addStateManager( this );
}

// Everything OK so far. Now we can set SM reference in PC
// It can be done only after myLC is set to deligate validation
// to the LC and objectId verified for uniqueness
replaceStateManager( myPC, thisSM );
}
catch ( SecurityException e ) {
throw new NucleusUserException( e.getMessage() );
}
catch ( NucleusException ne ) {
if ( myOM.getStateManagerById( myID ) == this ) {
myOM.removeStateManager( this );
}
throw ne;
}
}

/**
* Method to disconnect any cloned persistence capable objects from their StateManager.
*
* @param pc The PersistenceCapable object
* @return Whether the object was disconnected.
*/
protected boolean disconnectClone( PersistenceCapable pc ) {
if ( ( ( operationalFlags & MISC_DETACHING ) != 0 ) ) {
return false;
}
if ( pc != myPC ) {
if ( NucleusLogger.PERSISTENCE.isDebugEnabled() ) {
NucleusLogger.PERSISTENCE.debug( LOCALISER.msg( "026001", StringUtils.toJVMIDString( pc ), this ) );
}

// Reset jdoFlags in the clone to PersistenceCapable.READ_WRITE_OK
// and clear its state manager.
pc.jdoReplaceFlags();
replaceStateManager( pc, null );
return true;
} else {
return false;
}
}

/**
* Convenience method to unset the owners of all SCO fields in the PC object.
*/
private void unsetOwnerInSCOFields() {
// Call unsetOwner() on all loaded SCO fields.
int[] fieldNumbers = getFlagsSetTo( loadedFields, getSecondClassMutableFieldNumbers(), true );
if ( fieldNumbers != null && fieldNumbers.length > 0 ) {
provideFields( fieldNumbers, new UnsetOwners() );
}
}

/**
* Disconnect the StateManager from the PersistenceManager and PC object.
*/
public void disconnect()
    {
        if (NucleusLogger.PERSISTENCE.isDebugEnabled())
        {
            NucleusLogger.PERSISTENCE.debug(LOCALISER.msg("026011", StringUtils.toJVMIDString(myPC), this));
        }

        if (cacheRelationships && myOM.getObjectManagerFactory().hasLevel2Cache()) {
         Level2Cache l2cache = myOM.getObjectManagerFactory().getLevel2Cache();
synchronized(l2cache) {
CachedPC cachedpc = l2cache.get( myID );
if (cachedpc == null) {
CachedPC newcachepc = cache();
l2cache.put( myID, newcachepc );
}
else {
if ( associatedValuesMap != null ) {
for ( Object obj : associatedValuesMap.keySet() ) {
NucleusLogger.CACHE.debug( "ASS: setting association for cache " + obj );
cachedpc.setRelationField( (String) obj, getAssociatedValue( obj ) );
}
}

}
}
        }
        //we are transitioning to TRANSIENT state, so if any postLoad
        //action is pending we do it before. This usually happens when
        //we make transient instances using the fetch plan and some
        //fields were loaded during this action which triggered a jdoPostLoad event
        if (postLoadPending)
        {
            changingState = false; //hack to make sure postLoad does not return without processing
            postLoadPending = false;
            postLoad();
        }
        
        unsetOwnerInSCOFields();

        myOM.removeStateManager(this);
        jdoDfgFlags = PersistenceCapable.READ_WRITE_OK;
        myPC.jdoReplaceFlags();

        operationalFlags |= MISC_DISCONNECTING;
        try
        {
            replaceStateManager(myPC, null);
        }
        finally
        {
            operationalFlags &= ~MISC_DISCONNECTING;
        }

        if (associatedValuesMap != null)
        {
            associatedValuesMap.clear();
            associatedValuesMap = null;
        }
        clearSavedFields();
        myOM = null;
        myFP = null;
        myPC = null;
        myID = null;
        myLC = null;
        cmd = null;
    }

/**
* Registers the pc class in the cache
*/
public void registerTransactional() {
myOM.addStateManager( this );
}

// ------------------------------ Detach Methods ---------------------------

/**
* Convenience method to retrieve the detach state from the passed State Manager's object.
*
* @param sm The State Manager
*/
public void retrieveDetachState( org.datanucleus.StateManager sm ) {
if ( sm.getObject() instanceof Detachable ) {
( (JDOStateManagerImpl2) sm ).operationalFlags |= MISC_RETRIEVING_DETACHED_STATE;
( (Detachable) sm.getObject() ).jdoReplaceDetachedState();
( (JDOStateManagerImpl2) sm ).operationalFlags &= ~MISC_RETRIEVING_DETACHED_STATE;
}
}

/**
* Convenience method to reset the detached state in the current object.
*/
public void resetDetachState() {
if ( getObject() instanceof Detachable ) {
operationalFlags |= MISC_RESETTING_DETACHED_STATE;
try {
( (Detachable) getObject() ).jdoReplaceDetachedState();
}
finally {
operationalFlags &= ~MISC_RESETTING_DETACHED_STATE;
}
}
}

/**
* Method to update the "detached state" in the detached object to obtain the "detached state" from the detached object, or to reset it (to null).
*
* @param pc The PersistenceCapable beind updated
* @param currentState The current state values
* @return The detached state to assign to the object
*/
public Object[] replacingDetachedState( Detachable pc, Object[] currentState ) {
if ( ( operationalFlags & MISC_RESETTING_DETACHED_STATE ) != 0 ) {
return null;
} else if ( ( operationalFlags & MISC_RETRIEVING_DETACHED_STATE ) != 0 ) {
// Retrieving the detached state from the detached object
// Don't need the id or version since they can't change
BitSet jdoLoadedFields = (BitSet) currentState[2];
for ( int i = 0; i < this.loadedFields.length; i++ ) {
this.loadedFields[i] = jdoLoadedFields.get( i );
}

BitSet jdoModifiedFields = (BitSet) currentState[3];
for ( int i = 0; i < dirtyFields.length; i++ ) {
dirtyFields[i] = jdoModifiedFields.get( i );
}
setVersion( currentState[1] );
return currentState;
} else {
// Updating the detached state in the detached object with our state
Object[] state = new Object[4];
state[0] = myID;
state[1] = getVersion( myPC );

// Loaded fields
BitSet loadedState = new BitSet();
for ( int i = 0; i < loadedFields.length; i++ ) {
if ( loadedFields[i] ) {
loadedState.set( i );
} else {
loadedState.clear( i );
}
}
state[2] = loadedState;

// Modified fields
BitSet modifiedState = new BitSet();
for ( int i = 0; i < dirtyFields.length; i++ ) {
if ( dirtyFields[i] ) {
modifiedState.set( i );
} else {
modifiedState.clear( i );
}
}
state[3] = modifiedState;

return state;
}
}

// ------------------------------ Helper Methods ---------------------------

/**
* Method to dump a PersistenceCapable object to the specified PrintWriter.
*
* @param pc The PersistenceCapable object
* @param out The PrintWriter
*/
private static void dumpPC( PersistenceCapable pc, PrintWriter out ) {
out.println( StringUtils.toJVMIDString( pc ) );

if ( pc == null ) {
return;
}

out.print( "jdoStateManager = " + peekField( pc, "jdoStateManager" ) );
out.print( "jdoFlags = " );
Object flagsObj = peekField( pc, "jdoFlags" );
if ( flagsObj instanceof Byte ) {
out.println( jdoFlagsToString( ( (Byte) flagsObj ).byteValue() ) );
} else {
out.println( flagsObj );
}

Class c = pc.getClass();
do {
String[] fieldNames = HELPER.getFieldNames( c );
for ( int i = 0; i < fieldNames.length; ++i ) {
out.print( fieldNames[i] );
out.print( " = " );
out.println( peekField( pc, fieldNames[i] ) );
}
c = c.getSuperclass();
} while ( c != null && PersistenceCapable.class.isAssignableFrom( c ) );
}

/**
* Utility to dump the contents of the StateManager.
*
* @param out PrintWriter to dump to
*/
public void dump( PrintWriter out ) {
out.println( "myPM = " + myOM );
out.println( "myID = " + myID );
out.println( "myLC = " + myLC );
out.println( "cmd = " + cmd );
out.println( "srm = " + getStoreManager() );
out.println( "fieldCount = " + getHighestFieldNumber() );
out.println( "dirty = " + dirty );
out.println( "flushing = " + ( ( operationalFlags & MISC_FLUSHING ) != 0 ) );
out.println( "changingState = " + changingState );
out.println( "postLoadPending = " + postLoadPending );
out.println( "disconnecting = " + ( ( operationalFlags & MISC_DISCONNECTING ) != 0 ) );
out.println( "dirtyFields = " + StringUtils.booleanArrayToString( dirtyFields ) );
out.println( "getSecondClassMutableFields() = " + StringUtils.booleanArrayToString( getSecondClassMutableFields() ) );
out.println( "getAllFieldNumbers() = " + StringUtils.intArrayToString( getAllFieldNumbers() ) );
out.println( "secondClassMutableFieldNumbers = " + StringUtils.intArrayToString( getSecondClassMutableFieldNumbers() ) );

out.println();
out.println( "jdoFlags = " + jdoFlagsToString( jdoDfgFlags ) );
out.println( "loadedFields = " + StringUtils.booleanArrayToString( loadedFields ) );
out.print( "myPC = " );
dumpPC( myPC, out );

out.println();
out.println( "savedFlags = " + jdoFlagsToString( savedFlags ) );
out.println( "savedLoadedFields = " + StringUtils.booleanArrayToString( savedLoadedFields ) );

out.print( "savedImage = " );
dumpPC( savedImage, out );
}

/**
* Utility to convert JDO specific flags to a String.
*
* @param flags The JDO flags
* @return String version
*/
private static String jdoFlagsToString( byte flags ) {
switch ( flags ) {
case PersistenceCapable.LOAD_REQUIRED:
return "LOAD_REQUIRED";
case PersistenceCapable.READ_OK:
return "READ_OK";
case PersistenceCapable.READ_WRITE_OK:
return "READ_WRITE_OK";
default:
return "???";
}
}

public ObjectProvider getObjectProvider() {
return objectProvider;
}

/**
* Fetch the specified fields from the database.
*
* @param fieldNumbers the numbers of the field(s) to fetch.
*/
protected void loadSpecifiedFields( int[] fieldNumbers ) {
if ( myOM.getApiAdapter().isDetached( myPC ) ) {
// Nothing to do since we're detached
return;
}

if ( isEmbedded() ) {
// Should never happen since embedded will always retrieve all fields in one go.
} else {

if (cacheRelationships) {
useCacheRelationships( fieldNumbers );
}
else {
getStoreManager().getPersistenceHandler().fetchObject( getObjectProvider(), fieldNumbers );

}
}
}

private void useCacheRelationships(int[] fieldNumbers) {
boolean isFK = false;
String relName = null;
ArrayList<Integer> fNums = new ArrayList<Integer>();

for ( int i = 0; i < fieldNumbers.length; i++ ) {

if ( checkForRelationship( fieldNumbers[i] ) ) {
relName = getClassMetaData().getMetaDataForManagedMemberAtAbsolutePosition( fieldNumbers[i] ).getName();

Object val = getAssociatedValue( relName );

if ( val != null ) {

if (val instanceof Collection || val instanceof Map ) {
replaceCollectionField( fieldNumbers[i], val, false );
continue;
}

NucleusLogger.CACHE.debug( "ASS: " + relName + " = " + val );
// we will get it and populate through replaceFields with a FieldManager

Object fieldValue = myOM.getExecutionContext().getObjectFromCache( val );
NucleusLogger.CACHE.debug( "ASS: value from cache " + relName + " = " + fieldValue );
// get from cache or load from db by id.
if ( fieldValue == null ) {
fieldValue = this.myOM.findObject( val, false, false, this.getObject().getClass().getName() );
NucleusLogger.CACHE.debug( "ASS: value from findObject " + relName + " = " + fieldValue );
}

if ( fieldValue != null ) {
replaceField( fieldNumbers[i], fieldValue, false );
// this.replaceFieldValue( fieldNumbers[0], fieldValue );
// this.loadedFields[fieldNumbers[0]] = true;

}
else {
fNums.add( fieldNumbers[i] );
}
// return;
}
else {
fNums.add( fieldNumbers[i] );
}
} else {
fNums.add( fieldNumbers[i] );
}

}
int[] fieldNums = new int[fNums.size()];

for ( int i = 0; i < fNums.size(); i++ ) {
fieldNums[i] = fNums.get( i );
}

if (fieldNums.length != 0) {
getStoreManager().getPersistenceHandler().fetchObject( getObjectProvider(), fieldNums );

for ( int i = 0; i < fieldNums.length; i++ ) {
if ( checkForRelationship( fieldNums[i] ) ) {
Object o = this.provideField( fieldNums[i] );

setAssocation( fieldNums[i], o );
}
}
}

}

private void replaceCollectionField( int fieldNumber, Object value, boolean makeDirty ) {
AbstractMemberMetaData mmd = getClassMetaData().getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );

AbstractMemberMetaData fmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );

/**
 * so, what we need to do here is change this back to the delegate type with hollow objects or maybe out of cache.
 * then, call scoutils.newscoinstace.
 */
if (value instanceof Collection) {
Collection collValue = (Collection) value;
Iterator collIter = collValue.iterator();
Collection returnColl = null;
try {
if ( value.getClass().isInterface() ) {
if ( List.class.isAssignableFrom( value.getClass() ) || mmd.getOrderMetaData() != null ) {
// List based
returnColl = new ArrayList();
} else {
// Set based
returnColl = new HashSet();
}
} else {
if ( value instanceof SCO ) {
returnColl = (Collection) ( (SCO) value ).getValue().getClass().newInstance();
} else {
returnColl = (Collection) value.getClass().newInstance();
}
}

// Recurse through elements, and put ids of elements in return value
String clazzName = null;
Class cls = null;
while ( collIter.hasNext() ) {
Object elem = collIter.next();
if (elem instanceof OIDImplKodo) {
clazzName = ( (OIDImplKodo) elem ).getPcClass();
cls = myOM.getExecutionContext().getClassLoaderResolver().classForName( clazzName );
}
org.datanucleus.StateManager sm = StateManagerFactory.newStateManagerForHollow( myOM.getExecutionContext(), cls, elem );
returnColl.add( sm.getObject() );
}
SCOUtils.newSCOInstance( this.getObjectProvider(), fmd, fmd.getType(), ( returnColl != null ? returnColl.getClass() : null ), returnColl, false, false, true );
//replaceField( fieldNumber, returnColl, false );
}
catch ( Exception e ) {
NucleusLogger.CACHE.warn( "Unable to hydrate object of type " + value.getClass().getName() + " from L2 caching : " + e.getMessage() );

// Contents not loaded so just mark as unloaded
return;
}
}
else if ( value instanceof Map ) {
// 1-N, M-N Map
if ( MetaDataUtils.getInstance().storesPersistable( mmd, myOM.getExecutionContext() ) ) {
try {
Map returnMap = null;
if ( value.getClass().isInterface() ) {
returnMap = new HashMap();
} else {
if ( value instanceof SCO ) {
returnMap = (Map) ( (SCO) value ).getValue().getClass().newInstance();
} else {
returnMap = (Map) value.getClass().newInstance();
}
}
Iterator mapIter = ( (Map) value ).entrySet().iterator();
while ( mapIter.hasNext() ) {
Map.Entry entry = (Map.Entry) mapIter.next();
Object mapKey = null;
Object mapValue = null;
if ( mmd.getMap().keyIsPersistent() ) {
mapKey = myOM.getApiAdapter().getIdForObject( entry.getKey() );
} else {
mapKey = entry.getKey();
}
String clazzName = null;
Class cls = null;
if ( mmd.getMap().valueIsPersistent() ) {
if (mapValue instanceof OIDImplKodo) {
clazzName = ( (OIDImplKodo) mapValue ).getPcClass();
cls = myOM.getExecutionContext().getClassLoaderResolver().classForName( clazzName );
}

org.datanucleus.StateManager sm = StateManagerFactory.newStateManagerForHollow( myOM.getExecutionContext(), cls, entry.getValue() );
mapValue = sm.getObject();
} else {
mapValue = entry.getValue();
}
returnMap.put( mapKey, mapValue );
}
SCOUtils.newSCOInstance( this.getObjectProvider(), fmd, fmd.getType(), ( returnMap != null ? returnMap.getClass() : null ), returnMap, false, false, true );
//replaceField( fieldNumber, returnMap, false );
}
catch ( Exception e ) {
NucleusLogger.CACHE.warn( "Unable to create object of type " + value.getClass().getName() + " for L2 caching : " + e.getMessage() );

// Contents not loaded so just mark as unloaded
return;
}
}
}
}

private void setAssocation( int fieldNumber, Object value ) {
AbstractMemberMetaData mmd = getClassMetaData().getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
if ( this.myOM.getApiAdapter().isPersistable( value ) ) {
// 1-1, N-1 PC field
if ( mmd.isSerialized() || mmd.isEmbedded() ) {
// TODO Support serialised/embedded PC fields
return;
}

// Put field OID in CachedPC "relationFields" and store null in this field
setAssociatedValue( mmd.getName(), myOM.getApiAdapter().getIdForObject( value ) );
return;
} else if ( value instanceof Collection ) {
// 1-N, M-N Collection
if ( MetaDataUtils.getInstance().storesPersistable( mmd, myOM.getExecutionContext() ) ) {
if ( value instanceof List && mmd.getOrderMetaData() != null && !mmd.getOrderMetaData().isIndexedList() ) {
// Ordered list so don't cache since dependent on datastore-retrieve order
return;
}
if ( mmd.isSerialized() || mmd.isEmbedded() || mmd.getCollection().isSerializedElement() || mmd.getCollection().isEmbeddedElement() ) {
// TODO Support serialised/embedded elements
return;
}
Collection collValue = (Collection) value;
if ( collValue instanceof SCO && ! ( (SCOContainer) value ).isLoaded() ) {
// Contents not loaded so just mark as unloaded
((SCOContainer) value ).load();
//return;
}

Iterator collIter = collValue.iterator();
Collection returnColl = null;
try {
if ( value.getClass().isInterface() ) {
if ( List.class.isAssignableFrom( value.getClass() ) || mmd.getOrderMetaData() != null ) {
// List based
returnColl = new ArrayList();
} else {
// Set based
returnColl = new HashSet();
}
} else {
if ( value instanceof SCO ) {
returnColl = (Collection) ( (SCO) value ).getValue().getClass().newInstance();
} else {
returnColl = (Collection) value.getClass().newInstance();
}
}

// Recurse through elements, and put ids of elements in return value
while ( collIter.hasNext() ) {
Object elem = collIter.next();
returnColl.add( myOM.getApiAdapter().getIdForObject( elem ) );
}

// Put Collection<OID> in CachedPC "relationFields" and store null in this field
setAssociatedValue( mmd.getName(), returnColl );
return;
}
catch ( Exception e ) {
NucleusLogger.CACHE.warn( "Unable to create object of type " + value.getClass().getName() + " for L2 caching : " + e.getMessage() );

// Contents not loaded so just mark as unloaded
return;
}
} else {
// Collection<Non-PC> so just return it
if ( value instanceof SCOContainer ) {
if ( ( (SCOContainer) value ).isLoaded() ) {
// Return unwrapped collection
// return ((SCO)value).getValue();
return;
} else {
// Contents not loaded so just mark as unloaded
return;
}
} else {
return;
}
}
} else if ( value instanceof Map ) {
// 1-N, M-N Map
if ( MetaDataUtils.getInstance().storesPersistable( mmd, myOM.getExecutionContext() ) ) {
if ( mmd.isSerialized() || mmd.isEmbedded() || mmd.getMap().isSerializedKey() || mmd.getMap().isEmbeddedKey() || mmd.getMap().isSerializedValue()
|| mmd.getMap().isEmbeddedValue() ) {
// TODO Support serialised/embedded keys/values
return;
}

if ( value instanceof SCO && ! ( (SCOContainer) value ).isLoaded() ) {
// Contents not loaded so just mark as unloaded
((SCOContainer) value ).load();
return;
}

try {
Map returnMap = null;
if ( value.getClass().isInterface() ) {
returnMap = new HashMap();
} else {
if ( value instanceof SCO ) {
returnMap = (Map) ( (SCO) value ).getValue().getClass().newInstance();
} else {
returnMap = (Map) value.getClass().newInstance();
}
}
Iterator mapIter = ( (Map) value ).entrySet().iterator();
while ( mapIter.hasNext() ) {
Map.Entry entry = (Map.Entry) mapIter.next();
Object mapKey = null;
Object mapValue = null;
if ( mmd.getMap().keyIsPersistent() ) {
mapKey = myOM.getApiAdapter().getIdForObject( entry.getKey() );
} else {
mapKey = entry.getKey();
}
if ( mmd.getMap().valueIsPersistent() ) {
mapValue = myOM.getApiAdapter().getIdForObject( entry.getValue() );
} else {
mapValue = entry.getValue();
}
returnMap.put( mapKey, mapValue );
}

// Put Map<X, Y> in CachedPC "relationFields" and store null in this field
// where X, Y can be OID if they are persistable objects
setAssociatedValue( mmd.getName(), returnMap );
return;
}
catch ( Exception e ) {
NucleusLogger.CACHE.warn( "Unable to create object of type " + value.getClass().getName() + " for L2 caching : " + e.getMessage() );

// Contents not loaded so just mark as unloaded
return;
}
} else {
// Map<Non-PC, Non-PC> so just return it
if ( value instanceof SCOContainer ) {
if ( ( (SCOContainer) value ).isLoaded() ) {
// Return unwrapped map
// setAssociatedValue(mmd.getName(), ((SCO)value).getValue());
return;
} else {
// Contents not loaded so just mark as unloaded
return;
}
} else {
return;
}
}
} else if ( value instanceof Object[] ) {
// Array, maybe of Persistable objects
if ( MetaDataUtils.getInstance().storesPersistable( mmd, myOM.getExecutionContext() ) ) {
if ( mmd.isSerialized() || mmd.isEmbedded() || mmd.getArray().isSerializedElement() || mmd.getArray().isEmbeddedElement() ) {
// TODO Support serialised/embedded elements
return;
}

Object[] returnArr = new Object[Array.getLength( value )];
for ( int i = 0; i < Array.getLength( value ); i++ ) {
Object element = Array.get( value, i );
returnArr[i] = myOM.getApiAdapter().getIdForObject( element );
}

// Put OID[] in CachedPC "relationFields" and store null in this field
setAssociatedValue( mmd.getName(), returnArr );
return;
} else {
// Array element type is not persistable so just return the value
return;
}
} else if ( value instanceof StringBuffer ) {
// Make a copy of a StringBuffer for the cache since it is mutable but final
return; // new StringBuffer(((StringBuffer)value).toString());
} else if ( value instanceof SCO ) {
// SCO wrapper - so replace with unwrapped
if ( NucleusLogger.CACHE.isDebugEnabled() ) {
// TODO Localise this
NucleusLogger.CACHE.debug( "JDOStaeManager this=" + this.toString() + " field=" + fieldNumber
+ " is having its SCO wrapper replaced prior to L2 caching." );
}
// TODO Need to take copy of the object see NUCCORE-52
return;// ((SCO)value).getValue();
} else {
return;
}
}

private int getRelationshipType(int fieldNumber) {
AbstractClassMetaData cmd = getClassMetaData();
AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition( fieldNumber );
int relationType = mmd.getRelationType( myOM.getExecutionContext().getClassLoaderResolver() );

return relationType;
}

private boolean checkForRelationship( int fieldNumber ) {

int relationType = getRelationshipType(fieldNumber);

if ( relationType != Relation.NONE ) {
return true;
}

return false;

}

private int getFieldNumberForName( String name ) {
AbstractClassMetaData cmd = getClassMetaData();
return cmd.getAbsolutePositionOfMember( name );
}

/**
* Method to return an L2 cacheable form of the managed object.
*
* @return The object suitable for L2 caching
*/

public CachedPC cache() {

int[] loadedFieldNumbers = getLoadedFieldNumbers();
if ( loadedFieldNumbers == null || loadedFieldNumbers.length == 0 ) {
// No point caching an object with no loaded fields!
return null;
}

// Create the cacheable object and add to cache state
Object cachePC = myPC.jdoNewInstance( this, myPC.jdoGetObjectId() );
CachedPC cachedPC = new CachedPC( cachePC, getLoadedFields(), getTransactionalVersion( myPC ) );

// Connect a StateManager - the lifecycle state is unimportant
JDOStateManagerImpl2 cacheSM = new JDOStateManagerImpl2( myOM, cmd );
cacheSM.initialiseForDetached( cachePC, getExternalObjectId( myPC ), getVersion( myPC ) );

// Copy across all of the loaded fields that are allowed to be cached
cacheSM.replaceFields( getLoadedFieldNumbers(), new CachePopulateFieldManager( this.getObjectProvider(), cachedPC ) );

if (cacheRelationships) {
if ( associatedValuesMap != null ) {
for ( Object obj : associatedValuesMap.keySet() ) {
// this is to get around where the associated map added a javatypemapping for inserts.
// probably need to create a new relationship map for this stuff.
if (!(obj instanceof String)) {
continue;
}
NucleusLogger.CACHE.debug( "ASS: setting association for cache " + obj );
cachedPC.setRelationField( (String) obj, getAssociatedValue( obj ) );
}
}
}
// Disconnect the StateManager
replaceStateManager( (PersistenceCapable) cachePC, null );

return cachedPC;
}

/**
* Convenience method to load the passed field values. Loads the fields using any required fetch plan and calls jdoPostLoad() as appropriate.
*
* @param fv Field Values to load (including any fetch plan to use when loading)
*/

public void loadFieldValues( FieldValues fv ) {
// Fetch the required fields using any defined fetch plan
FetchPlanForClass origFetchPlan = myFP;
FetchPlan loadFetchPlan = fv.getFetchPlanForLoading();
if ( loadFetchPlan != null ) {
myFP = loadFetchPlan.manageFetchPlanForClass( cmd );
}

boolean callPostLoad = myFP.isToCallPostLoadFetchPlan( this.loadedFields );
if ( loadedFields.length == 0 ) {
// Class has no fields so since we are loading from scratch just call postLoad
callPostLoad = true;
}

fv.fetchFields( this );

if ( callPostLoad && isFetchPlanLoaded() && myOM.getOMFContext().getApi().equalsIgnoreCase( "JDOProtrade" ) ) {
postLoad();
}

// Reinstate the original (PM) fetch plan
myFP = origFetchPlan;
}


/**
* Initialise to create a StateManager for a PersistenceCapable object, assigning the specified id to the object. This is used when getting objects out of the L2
* Cache, where they have no StateManager assigned, and returning them as associated with a particular PM.
*
* @param cachedPC The cached PC object
* @param id Id to assign to the PersistenceCapable object
* @param pcClass Class of the object that this will manage the state for
*/
public void initialiseForCachedPC( CachedPC cachedPC, Object id, Class pcClass ) {
// Create a new copy of the input object type, performing the majority of the initialisation
initialiseForHollow( id, null, pcClass );

myLC = myOM.getOMFContext().getApiAdapter().getLifeCycleState( LifeCycleState.P_CLEAN );
jdoDfgFlags = PersistenceCapable.READ_OK;

// Synchronise the L2 cached object while we grab its fields
Object cachePC = cachedPC.getPersistableObject();
synchronized ( cachePC ) {
if ( cacheRelationships && cachedPC.getRelationFieldNames() != null ) {
for ( String names : cachedPC.getRelationFieldNames() ) {
NucleusLogger.CACHE.debug( "ASS: added association from cache " + names );
this.setAssociatedValue( names, cachedPC.getRelationField( names ) );
}
}
// Load all fields that are cached that are in the fetch plan
// TODO We could just load all fields that are cached but could lead to graph depth problems
int[] fieldsToLoad = getFlagsSetTo( cachedPC.getLoadedFields(), myFP.getMemberNumbers(), true );
if ( fieldsToLoad != null ) {
// Connect a StateManager - the lifecycle state is unimportant
JDOStateManagerImpl2 cacheSM = new JDOStateManagerImpl2( myOM, cmd );
cacheSM.initialiseForDetached( cachePC, getExternalObjectId( myPC ), getVersion( myPC ) );

// Put in L1 cache for easy referencing (case of bi-dir relations needed by the next step)
myOM.putObjectIntoCache( this );

// Copy the field values in
this.replaceFields( fieldsToLoad, new CacheRetrieveFieldManager( cacheSM.getObjectProvider(), cachedPC ) );

// Disconnect the cached object
disconnectClone( (PersistenceCapable) cachePC );
}

// Set the loaded fields to match what was just loaded
if ( fieldsToLoad != null ) {
for ( int i = 0; i < fieldsToLoad.length; i++ ) {
loadedFields[fieldsToLoad[i]] = true;
}
}

if ( cachedPC.getVersion() != null ) {
// Make sure we start from the same version as was cached
setVersion( cachedPC.getVersion() );
}

// Make sure any SCO fields are wrapped
replaceAllLoadedSCOFieldsWithWrappers();

}

if ( myOM.getTransaction().isActive() ) {
myOM.enlistInTransaction( this );
}

if ( isFetchPlanLoaded() ) {
// Should we call postLoad when getting the object out of the L2 cache ? Seems incorrect IMHO
postLoad();
}
}


}


What we would like to do is be able to cache relationship fields so that the first time the relationship is accessed, there will be a join against the owner id. On subsequent calls, the relationship would be cached with the id. The id would be used to look up in the cache and either pass back the object, load it by id, and cache it. The relationship is removed when a field is made dirty.

I have augmented the StateManager to accomplish this but I am not sure I have gone down the best path. I am submitting the enclosed augmented JDOStatemanager to show where I trying to go with this. One thing to look at it that I have to use the OID to get the class of the object id i have so that i can create hallow pms with statemanagers.

it is configurable via the cacheRelationships variable. <persistence-property name="datanucleus.CacheRelationships" value="false"/>
Andy Jefferson added a comment - 22/Oct/10 06:20 AM
Attach what was previously in the "description"!

Andy Jefferson made changes - 22/Oct/10 06:20 AM
Attachment JDOStateManager2.java [ 11283 ]
Andy Jefferson added a comment - 22/Oct/10 06:21 AM
DataNucleus already caches "relationships". How is this different ? Also "patch" format is far easier to read to decide what has changed

Tom Zurkan added a comment - 22/Oct/10 07:10 PM - edited
sorry, should have attached it the first time.

i don't see relationships in level 2 cache at all. please explain. how do i turn that on and use it? where would it be implemented? we are seeing that anytime a relationship is accessed, it goes through the datastore. is it just not in a release yet?

Andy Jefferson added a comment - 22/Oct/10 07:30 PM
JDO defines annotations and XML metadata to mark fields as cacheable (or not).
http://www.datanucleus.org/products/accessplatform_2_2/jdo/cache.html#level2

Look at org.datanucleus.cache.CachedPC and the code already in JDOStateManagerImpl for putting objects in the L2 cache and getting them out. Has been in releases for a very long time, so if you don't see relationships being cached then *define a testcase that demonstrates it*

Tom Zurkan added a comment - 22/Oct/10 07:44 PM
So, collections and individual 1-1 relationships are cachable by default and you are saying that they should be in CachePC relationship? I can easly reproduce that as nothing ever goes to CachePC relationshipsFields map in our app. I will put together a test case today. thanks.

Tom Zurkan added a comment - 23/Oct/10 02:22 AM
ok, i created a junit test that showed that my pc object was put to level2 but the relations field was never updated. here is my class definition:


<class name="FTeam" persistence-capable-superclass="com.protrade.common.persistence.HBaseIdCreateTime">
<field name="league" null-value="exception">
</field>
<field name="owner" persistence-modifier="persistent" null-value="exception">
</field>
<field name="name" persistence-modifier="persistent">
<index />
</field>
<field name="rosterEntries" table="FTEAM_ROSTERENTRIES">
<collection element-type="FScorer" />
<join column="JDOID" />
<element column="ROSTERENTRIES_JDOID" />
</field>
</class>

The league is a 1-1 (M-1) here and the rosterEntries is a 1-M. I just had a simple junit test that went through these relationships as follows:

public void testTeamCache() {
try {
FLeague league = getFLeagueDao().loadByFieldNCS( FLeague.class, "name", "Bugbash 4" );

FootballLeagueRoster roster = league.getRoster();

FTeamId id = null;

for (FTeam team : league.getTeams()) {
if ("Your momma's so old".equals( team.getName() )) {
Set<FScorer> scorers = team.getRosterEntries();

team.getLeague();

id = team.getFTeamId();
}
}

FTeam team = getFLeagueDao().loadNCS( id );

team.getLeague();

team.getRosterEntries();
}
catch ( PersistenceException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}


with an output from logging looking something like this:

2010-10-22 18:15:14,808 [main] DEBUG DataNucleus.Cache - CachePopulateFM.fetchObjectField this=com.protrade.ffootball.data.entities.team.FTeam@125b750 field=1 is having its SCO wrapper replaced prior to L2 caching.
2010-10-22 18:15:14,808 [main] DEBUG DataNucleus.Cache - Object "com.protrade.ffootball.data.entities.team.FTeam@125b750" (id="com.protrade.ffootball.data.entities.team.FTeam-12035695") added to Level 2 cache (loadedFlags="[YYNYNNY]", relationFields="null")


am i missing something?

thanks

 

Andy Jefferson added a comment - 23/Oct/10 08:12 AM
> loadedFlags="[YYNYNNY]", relationFields="null"
> am i missing something?

You mean the fact that half of the fields (field 2, 4, 5) aren't loaded, so consequently can't be cached ?


Why not just run the DN end-to-end tests "test.jdo.datastore" and look at the log, and you see log entries like

08:07:44,910 (main) DEBUG [DataNucleus.Cache] - Object "org.jpox.samples.types.container.ContainerItem@10fd7f6" (id="2[OID]org.jpox.samples.types.container.ContainerItem") added to Level 2 cache (loadedFlags="[YYYYY]", relationFields="null")
08:07:44,911 (main) DEBUG [DataNucleus.Cache] - Object "org.jpox.samples.types.linkedhashset.LinkedHashSet1@15f7107" (id="1[OID]org.jpox.samples.types.linkedhashset.LinkedHashSet1") added to Level 2 cache (loadedFlags="[YYY]", relationFields="[items]")
08:07:45,272 (main) DEBUG [DataNucleus.Cache] - Object "org.jpox.samples.types.linkedhashset.LinkedHashSet1@126d3df" (id="3[OID]org.jpox.samples.types.linkedhashset.LinkedHashSet1") updated in Level 2 cache (loadedFlags="[YYY]", relationFields="[items]")
08:07:45,476 (main) DEBUG [DataNucleus.Cache] - Object with id="6[OID]org.jpox.samples.types.linkedhashset.LinkedHashSet1" taken from Level 2 cache (loadedFlags="[YYY]", relationFields="[items]") [cache size = 6] - extracting into managed persistable object

08:07:50,425 (main) DEBUG [DataNucleus.Cache] - Object "org.jpox.samples.one_many.collection.ListHolder@10c6cfc" (id="10[OID]org.jpox.samples.one_many.collection.ListHolder") added to Level 2 cache (loadedFlags="[YYNYYYYYNYYYYYYY]", relationFields="[joinListPCShared2, joinListPCShared1, fkListPC2, fkListPCShared1, fkListPCShared2, fkListPC, joinListPC]")

Tom Zurkan added a comment - 25/Oct/10 07:36 PM
ok, so you are saying that all fields have to be loaded in order for relationship fields to be cached? if that is the case, then my request/bug is that i would like to be able to cache fields that are loaded when some fields are not.

Andy Jefferson added a comment - 26/Oct/10 08:01 AM
I'm saying that to cache a field (whether relation or not) it has to be loaded. This seems an obvious requirement, otherwise you force loading of a field just to cache it.

Tom Zurkan added a comment - 26/Oct/10 07:28 PM
yes, of course it has to be loaded and i understand what you are asking/saying. let me clarify. i used a bad example.

using the same jdo description look at it like this:
// i have an id and use it to load a object.
FTeamId id = new FTeamId(12035736L);

FTeam team = getFLeagueDao().loadNCS(id);
// at this point the team is obtained and added to level 1 and 2 cache. the relationship
// fields are not loaded but all primitives are:
// added to Level 2 cache (loadedFlags="[YYNYNNY]", relationFields="null")

System.out.println(team.getLeague().getName());
// at this point the league object is loaded using a join with the team oid.
// the league is added to level 1 cache and the team level 1 cache now has the league as well.
team = getFLeagueDao().loadNCS( team.getFTeamId() );

System.out.println(team.getLeague().getName());

What I am asking is that when the league is loaded, the L2 cache for the team->league relationship is updated and the league is added to level 2 cache. this means that subsequent calls after L1 has been cleared (such as after a request cycle) will use the relationship to find the OID and then look in L2 for that object or load it from the datastore directly without the join.

the example above does not put league in level2 cache and it does not update the relationship in CachePC

does that make sense?


Andy Jefferson added a comment - 26/Oct/10 08:05 PM
I see no transactions. The L2 cache is only current set-up to update at commit.
Define a testcase that actually demonstrates something, otherwise it is futile going back and forth.

Andy Jefferson made changes - 26/Oct/10 08:06 PM
Project DataNucleus Cache [ 10142 ] DataNucleus Core [ 10143 ]
Key NUCCACHE-17 NUCCORE-588
Affects Version/s 2.1.2 [ 11040 ]
Tom Zurkan added a comment - 26/Oct/10 08:54 PM
will do. but, what i am asking for is essentially L2 being used outside a transaction like the example above.

Fernando Padilla added a comment - 26/Oct/10 09:02 PM
Hey, I'm also following this bug. :)

I think this is the best way to summarize it.

If DN goes through the trouble of executing an SQL query to get some data, that data should be captured into L2 so that it doesn't have to run that query again.


Huge performance gains up and down the stack. Every time you save a db query it's huge both on the webserver's cpu/response time, but also all of the database resources you save.. really really good stuff. :)

Tom Zurkan added a comment - 26/Oct/10 11:04 PM
yes, fernando has said it succinctly. what i am looking for is that all accessed data should be cached.

Tom Zurkan added a comment - 28/Oct/10 09:16 PM
The state manager that I provided does this. But, I am not sure that the direction is correct. Essentially, the state manager holds the relationships from cache and when a field is requested and not loaded, it gets the id from the relationship, and then loads the field from the datastore or from cache.

Andy Jefferson added a comment - 29/Oct/10 07:32 AM
The JDO spec defines adequately the L2 cache's role. What I need, as said earlier, is a testcase that demonstrates that it isn't caching particular data in some situation. As also mentioned earlier, the provided StateManager doesn't tell me what has changed, which is why patch format is the only thing that makes sense ... going hand in hand with the provided testcase ...

Fernando Padilla added a comment - 29/Oct/10 08:02 AM
Even if the spec says something like that, it's definitely not the most efficient/effetive behavior.

My bet is that it says that you MUST update the L2 cache whenever an object is updated through transaction commit. But I would take a bet that does not say that you MUST NOT update the L2 cache otherwise.



Can you point me to where in the spec it talks about the L2 cache only being populated during transactions? I just downloaded the full spec and can't seem to find any prescription to how the L2 cache is "supposed to work".

( I was searching through the latest: http://jcp.org/aboutJava/communityprocess/mrel/jsr243/index3.html )

Andy Jefferson added a comment - 29/Oct/10 08:07 AM
What? I said the JDO spec defines what the L2 cache is for so there is no need to repeat things here. I also said I want to see a testcase that demonstrates some point where it is not putting fields in the L2 cache, because I don't have one. I also said I wrote the L2 cache code around testcases that are transactional (since that is what JPOX used to be, solely) ... i.e we have no testcases that are non-tx and test the L2 cache (we have non-tx tests but for other things). Nowhere have I said it isn't supposed to work non-tx.

So I return to my first question. I need a testcase that demonstrates something ... the prerequisite for *any* issue

Fernando Padilla added a comment - 29/Oct/10 08:16 AM
Ah Ok. so very sorry. :)

I got confused between you saying "The L2 cache is only current set-up to update at commit" and then later saying "JDO spec defines adequately the L2 cache's role" when we kept pushing about how we wanted it work..

sorry I see my mistake now.

I can see how you want to push back for us to give you a clear "unit test". That's rational, so I guess that's the compromise.



Can we give it to you in pseudo code? Or could you give us a hint, as to a particular unit test we could adapt for this? (sorry i haven't spent time with the unit test code yet).

Andy Jefferson added a comment - 29/Oct/10 08:25 AM
No problem. A testcase can be some sample classes (with some particular relations), some metadata (or annotations), and then sample persistence code (makePersistent, query) and something that says with this persistence operation field X of object Y is not being cached even though it was just loaded/accessed

[It wouldn't need necessarily to be a JUnit since to work out if a field is actually cached in the L2 cache could involve some work]

Fernando Padilla added a comment - 29/Oct/10 08:35 PM
So, do you want running code? like within DN's unit tests? Or just pseudo-like code that you will get running somehow?

ps - Either way, is there an easy way to print out a PC's loaded field array? So we can check it within the test?

Tom Zurkan added a comment - 30/Oct/10 01:00 AM
so, i svn diffed the jdostatemanager with what i had originally checked in as a pretty much straight copy. i'm hoping this gives you a good view of what i changed in the statemanager. let me know if it is helpful at all.

Tom Zurkan made changes - 30/Oct/10 01:00 AM
Attachment jdostatemanager.diff [ 11290 ]
Andy Jefferson added a comment - 30/Oct/10 08:15 AM
Running tests ? As per
http://www.datanucleus.org/project/problem_jdo_testcase.html
is the preference, so either JUnit that uses our existing samples data from the test suites, if not that then a Main.java and the sample classes/metadata, and if not that then pseudo code.

Loaded/dirty fields, as per the foot of
http://www.datanucleus.org/products/accessplatform_2_2/jdo/object_lifecycle.html

Tom Zurkan added a comment - 03/Nov/10 02:56 AM - edited
ok, so i took your existing junit test for 1-1 relationships. i think it is pretty self-explanatory. i removed the transactions and load the preexisting loginAccount. then i access the login. then, i load the loginAccount again and get the login. you will notice that it does a join both times to get the login. please, let me know if you want further details. i did not include the datanucleus.properties. thanks.

Tom Zurkan made changes - 03/Nov/10 02:56 AM
Attachment RelationshipTest.java [ 11291 ]
Tom Zurkan added a comment - 08/Nov/10 08:15 PM
hey andy, i haven't heard anything since i posted the unit test. is this sufficient information for you?

Andy Jefferson added a comment - 08/Nov/10 10:01 PM - edited
Tom, not had much time recently. You don't say which test scenario you took that test from. test.jdo.datastore ? i.e does the sample have to use datastore id for it to work ? It doesn't persist any objects either so don't see how anything can work without knowing where it comes from

Tom Zurkan added a comment - 08/Nov/10 10:51 PM
yes, it is from test.jdo.datastore and yes, you would need to use a datastore id for it to work. alternatively, i can put the objects into the db as per the original unit test and then evict it from l2 cache before looking them up (and use a different pm so that it is not in L1). the point being that there will be a time where the insert happened at some other time, but, we need to read the data in without a transaction and have it go to L2 with relationships cached as well. i will adapt the original junit and send it as a patch since it sounds like that would help.

Andy Jefferson added a comment - 09/Nov/10 10:40 AM - edited
Ok, but I still don't know what is "DNLoginAccount", "DNLogin" ... presumably some copies of the "test.samples" "org.jpox.samples.one_one.unidir" classes. There is no "RelationshipTest" in "test.jdo.datastore" btw; it is in "test.jdo.orm.datastore".

Is this test a real JUnit test that will fail if the relations are not L2 cached ? If so it ought to go under "test.jdo.general" in CacheTest, since it is testing the cache. If not and just to create the situation in which log entries will show a problem then it doesn't matter what it's called.

Also, if the test is not going to "fail" in the JUnit sense, then put some statements to the log (using NucleusLogger.GENERAL.debug(...)) just before and just after where you say the field is not being L2 cached. This makes it clear what you're referring to

Tom Zurkan added a comment - 10/Nov/10 12:44 AM - edited
Ok Andy, I think I finally have it. This is a JUnit test that you can just drop into test.jdo.general. It shows the problem and fails as a JUnit test should. But, of course, no one would code something up like this except as a test. :) Let me know how you feel about this one. Thanks! CacheRelationshipTest.java

Tom Zurkan made changes - 10/Nov/10 12:44 AM
Attachment CacheRelationshipTest.java [ 11300 ]
Tom Zurkan added a comment - 12/Nov/10 08:15 PM
hi andy, wondering if this last unit test highlights the issue. i know it only really tests if the relationship is in cache and not that it is used when the field is accessed. so, just wondering what your feedback is. thanks.

Tom Zurkan added a comment - 16/Nov/10 02:41 AM
ok, so now i am running off of your datanucleus trunk and i see that my junit test passes which is great news. is the relationship map being used when fetching a field? i am going to test that next and hopefully come up with a unit test that shows it. thanks for step one. :)

Tom Zurkan added a comment - 18/Nov/10 03:21 AM
ok, so, now i have added another unit test that will fail because the loginaccount.login call causes a datastore query. of course, i had to add the following line to my datanucleus.properties file: datanucleus.managedRuntime=true
Also added the plugin to the project so that i can get the query count:

<dependency>
   <groupId>org.datanucleus</groupId>
   <artifactId>datanucleus-management</artifactId>
   <version>1.0.2</version>
</dependency>

this covers both the adding of the field to cache and retrieving it. thanks!

Tom Zurkan made changes - 18/Nov/10 03:21 AM
Attachment CacheRelationshipTest.java [ 11312 ]
Andy Jefferson added a comment - 10/Dec/10 11:08 AM
Presumed fixed in 2.2.0.release. All provided tests pass. If any other situation exists then please provide a further test

Andy Jefferson made changes - 10/Dec/10 11:08 AM
Status Open [ 1 ] Resolved [ 5 ]
Assignee Andy Jefferson [ andy ]
Fix Version/s 2.2.0.release [ 10931 ]
Resolution Fixed [ 1 ]
Andy Jefferson made changes - 23/Jan/11 11:57 AM
Status Resolved [ 5 ] Closed [ 6 ]