JPOX and Tapestry : The VLib example

Document version 0.92.

2004 David Ezzio. All rights reserved.

This tutorial explains how to use JPOX with Tapestry, using the VLib example as the template. JPOX provides a downloadable sample with the code described here. While this sample is for JPOX, it can be converted to DataNucleus easily.



What is the VLib example ?

This example, called VLib, is a port of the Virtual Library example distributed with Apache Tapestry. It shows a Tapestry application using JDO.

The original Virtual Library example is an ambitious example intended to show off the capabilities of Tapestry, a Web application framework. The fictitious business case for the Virtual Library application is the notion of a book club, perhaps at work, where people can borrow each other's books. The Virtual Library keeps track of what books are available, who owns them, and who has borrowed them. There are three levels of access: anonymous, logged-in, and administrative. The example looks sharp with a heavy emphasis on graphics in the presentation. Howard Lewis Ship, the architect of Tapestry, gives a two chapter tutorial of the Virtual Library example in his book Tapestry in Action , published by Manning.

As written, the VLib port runs with JPOX 1.0.2 (and later) and JPOX 1.1.0.A2 and later. As of these JPOX versions, the same VLib example works with both versions. As JPOX 1.1 evolves, it is expected that the VLib version for JPOX 1.1 will evolve to make use of some of the new features in JDO 2.0.



Who is most likely to benefit from this example ?

Developers who are familiar with or learning Tapestry will benefit the most from this example.



What is Tapestry ?

Tapestry is a Web application framework from Apache. It can be found at http://jakarta.apache.org/tapestry.

Web applications within Tapestry follow the model-view-controller (MVC) pattern. The view consists of HTML templates, and specifications in XML for JavaScript, as well as style sheets and graphic resources. The controller consists of Java code with page and component wiring in XML. Tapestry does not provide a model, making it an ideal partner for JDO.

Tapestry abstracts from the multi-threadedness of the Servlet API and provides tight coupling between the HTTP request and response handling code. The same components that participate in the rendering of page also accept the user's response to the page. The handshaking between HTTP responses and the Java code is handled by the framework, so that most dynamic information presented on the page and most user input to the page are represented as properties of the Tapestry page or its components. The user's next request action results in an invocation of a listener method on the page or one of its components. Every Tapestry page and component is single-threaded by design, and contrary to what you might be inclined to believe, it performs quite well because of object pooling.



How does the VLib example differ from the original?

The VLib example is a port (or fork) of the Virtual Library example that is distributed with Apache Tapestry.

Unlike the Virtual Library example which uses Enterprise JavaBeans to implement the model, the VLib example uses JDO.

In general, most of the presentation files are unchanged, and those that are changed, are changed very little. The controller code varies class-by-class from no change, to one or two little changes, to many small changes. The model code has been, for the most part, completely rewritten, although there is a family resemblance to the original code.



Where can I see VLib example working?

It would be useful to put the VLib example on the Web, but in the meantime, setting it up on your own machine is not hard.



How do I set up VLib in my development environment?

This tutorial assumes that you want to start by just deploying the example and seeing it work, before you dive into the source code and Ant build.

Install Tapestry 3.0. If you are just learning Tapestry, you should get the Hangman example working. The book Tapestry in Action from Manning Publications can be quite useful.

Install JPOX 1.1.0-alpha-2 (or later). If you are new to JDO, you should work through the JPOX tutorial. The book Using and Understanding Java Data Objects from Apress can be quite useful for understanding both the details of JDO and the big picture.

With the preliminaries taken care of, let's get started

  1. Determine which JDBC driver JPOX needs to talk to your favorite (and JPOX supported) database. Put this driver into the appropriate place to make your favorite Web app container happy. For example, Tomcat wants it in the <tomcat>/common/lib directory.
  2. Start your Web app container. For example, start up Tomcat.
  3. Unzip the example binary distribution, and deploy the vlib.war file to your Web app container. For example, copy it to the <tomcat>/webapps directory. If the container does not unzip the War file for you, unzip it to a vlib directory under the appropriate directory for your container. For example, unzip it to <tomcat>/webapps/vlib .
  4. Start your database server.
  5. You may want to create the database where you want the VLib data to be stored. For example, I use MySQL, and I have a database named vlib where the VLib data is stored. If you have a general test database that you use for lots of stuff, that should work as well.
  6. You may want to create a user account for the VLib application. You can also use an existing account.
  7. Find the jdo.properties file located in the unwrapped War file directory, <webapps-root>\vlib\WEB-INF\classes directory. Edit the four connection properties to connect to your database using the parameters for your JDBC driver. These connection properties provide the information that JPOX requires when it uses JDBC to connect to your database.
  8. At this point, there is no schema in your database to hold the persistent state of the VLib's data objects, and there is no data for the objects either. So the next step, which should be done only once, will create the schema and add the testing data. If you initialize the data more than once, you'll see duplicate books. To remove the duplicates, drop all the tables that the initialization creates, and try again.
  9. To initialize the database, use your Web browser to hit a URL that looks like the following:

    http://localhost:8000/vlib/app?service=external/InitDB

    You may have to alter the host name and the port to conform to your Web container's configuration.
  10. If you are successful, you will see a page that looks like the following:

From this point on, you can just use the application. The book Tapestry in Action contains a tutorial for the user interface. There are four user accounts already defined. Suzy Queue is squeue@bug.org . Frodo Baggins is ringbearer@bagend.shire , Dilbert is dilbert@bigco.com , and Grace Hopper is ghop@cobol.gov . All of them have the password 'secret' (What can I say, they are trusting souls!)

How to build from source

If you use Ant to build, begin by examining and modifying the build.xml file where all the build properties are defined at the top. Most of the Jar files that you will need are contained in the binary distribution, so you should be able to grab the miscellaneous files there and copy into the appropriate directories in your source distribution. The build has a default target called war-file and a target called clean . Begin with the clean target. This will invoke the property verification targets that can detect some types of errors in your build configuration. Once you correct the errors and get the clean target working, you should be ready to build the War file. The War file that you build should look very similar to the War file in the binary distribution. The clean target must always be invoked manually. All of the other targets are incremental build steps.



Is the VLib example dependent on JPOX?

Most of the features of JDO that the VLib example uses are required features in every JDO 1.0 compatible implementation. In one case, the VLib example uses a JDO optional feature, nontransactional-read, which is implemented by JPOX and nearly every JDO implementation.

The VLib example also uses three vendor features in the JPOX implementation. Vendor features are features that are found in the vendor's implementation of JDO but are not specified by the JDO standard. When the same feature is found in another implementation, it generally has a different interface. I'll list the vendor features and say a few words about each.

First, VLib uses JPOX's ability to automatically generate, upon first encounter, the schema for data objects. This is a feature that is ready-made for an example application that will be deployed in many different development environments. Most JDO implementations have ways to create the schema automatically, but so far as I know, there are no plans to make automatic schema generation a standard feature of JDO.

Second, the VLib example uses the toUpperCase method in JDOQL (JDO's query language). This vendor feature is implemented in many JDO implementations and will be required in JDO 2.0.

Lastly, the VLib example uses JPOX's support of SQL to obtain a count of the query results for setting up the paging of search results. In an example program with a small test data set, it is obviously not necessary to spend much time worrying about how to obtain the count of query results, but I used this vendor feature for several reasons. One, the business logic to page results was already present in the original Virtual Library example. Two, paging results is a problem for many real-world applications. Three, many O/R mapping implementation of JDO support SQL queries. Finally, support for aggregates is part of the JDO 2.0.

In short, the VLib example as written will work only with JPOX, but the work to port it to any other JDO implementation is minimal. As this example evolves to use JDO 2.0, its dependency on JPOX will diminish.



What are the design choices in this example?

The VLib example demonstrates a Tapestry Web application using JDO as its model. In this sense, the example is a proof-of-concept. The integration of Tapestry and JDO can be done!

The example has other important goals as well. Since working example code is often used as a template for real-world applications, the VLib example strives to avoid bad code. There is no reason for you to learn bad lessons from an example. It also attempts to show good architectural design. In other words, not only does the example work, but it works in a good way. No doubt there are bugs in this code, and the bugs may go deeper than some code not working. If you spot bad code or bad design, speak up, and help improve this example.

'Design patterns' is too strong a term for the designs decisions discussed here, since not all of these design decisions have been applied in multiple applications, and the classic exposition of a design pattern has not been adopted. Nevertheless, this example contains several design insights that are worth understanding. If you understand them, you can choose whether to apply them to the applications that you write.

Use one persistence manager per HTTP request

The first thing to understand when building a Web application using JDO, is to select and stick with the one-pm-per-request design pattern. This pattern was first described in the book Using and Understanding Java Data Objects . It is a simple concept, and it works without subtle errors.

The reason to obtain one persistence manager at the start of each request is the need to release the persistence manager at the end of the request. Persistence managers can consume limited resources which must be shared with other request handling threads. As a rule, they do not serialize, and therefore cannot be stored in the session.

The one-PM-per-request design pattern imposes real constraints on your application's design. We will encounter them at every turn throughout the rest of this discussion. The alternatives, however, can cause worse problems in the form of extra coding, increased complexity, the potential for subtle errors, and a failure to deliver the expected benefits. For further discussion of the alternatives, see Chapter 10, 'Using JDO in a Web Application' in Using and Understanding Java Data Objects.

In the one-pm-per-request design pattern, there is a static factory that produces persistence manager objects. In this example, the PMLocator class (contained within the VLibDataService source file) provides the factory by delegating to JDO's PersistenceManagerFactory class. Each time a request is serviced, a VLibDataService object is constructed, and during its initialization, it uses the locator to get a persistence manager. The important point is that the persistence manager factory is constructed only once for each Web container, but the persistence managers are obtained on each HTTP request. JDO is constructed to put most of the time into the construction of a persistence manager factory. The JDO implementation may use connection pooling, object pooling, and in some cases, data pooling, to reduce the computational burden of constantly getting and releasing persistence managers and the persistent objects they manage.

In the VLib example, the VirtualLibraryEngine class acts as the factory to produce the current request's VLibDataService . The VirtualLibraryEngine closes the VLibDataService (which thereby releases the persistence manager) in its cleanupAfterRequest method, which the Tapestry framework calls after every request. The VLib examples shows a clean and simple implementation of the one-pm-per-request pattern.

Encapsulate JDO transactions within the life-cycle of the service

This is really just a corollary of the pattern above. Since JDO 1.0 transactions do not outlive the persistence manager, if you close the persistence manager you must also end the transaction. In JDO 2.0, extended optimistic transactions will be supported. For further discussion of extended optimistic transactions, see the section 'Add extended optimistic transactional semantics' towards the end of this tutorial.

Divide the model into service and data classes

One the one hand, data objects are plain-old-Java-objects (POJOs), and their classes can be simple JavaBeans. Data classes are identified in the JDO metadata. JDO fetches and stores the state of the managed objects of data classes without the need for your code to implement the magic. On the other hand, your code will need to invoke explicit JDO services to control transactions, to define and run queries, to iterate query results, to create new persistent objects, todelete existing persistent objects, and to create value objects.

Therefore, think of your model as a set of data classes wrapped by a service. What goes into and comes out of the service are value objects (more on this point in the next section). The service itself uses managed data objects. The following diagram illustrates the point.



In the VLib example, the data classes are Book , Person , and Publisher , and the application's service is the VLibDataService .

Use value objects in the controller and presentation code

JDO's management of persistent objects has its good points and its bad points. On the good side, a data object looks like a POJO. They are simple to design, and easy to code and understand. JDO handles all of the code required to shuffle state between the object and the database. On the bad side, JDO's management of the object can be a hindrance in some cases.

For example, JDO's management is a hindrance when changes to persistent objects need to be recorded in memory without having an active JDO transaction. JDO will not allow a managed object to be changed unless a transaction is active. (Surprisingly, JDO's nontransactional-write feature is not the solution it appears to be).

JDO's management is also a hindrance when the persistence manager is to be released, but the data object is still needed for later use. The persistent objects are no longer valid once the persistent manager has been released (in other words, its close method has been called). At the same time, it is natural in most Web applications to store some data objects in the session between requests. The Web container may serialize these objects between requests, and the persistence manager is not around to manage them.

So it would appear that Web applications need data objects when they are not available. How is this problem solved? The answer is to create value objects which contain all of the information required for non-managed operations and for later updating of the corresponding managed object. Some people believe that separate classes should define the value objects, but in the classic words of Abe White, the architect of Kodo, 'data objects can be their own value objects.' By using the data classes to define both managed data objects and unmanaged value objects, both the code and its maintenance are simplified.

The entire process of creating a value object, modifying it, and later updating a managed data object with it is called 'detachment-attachment.' By creating value objects, the controller code can store and modify unmanaged data objects and sends them to the service to update managed data objects. JDO 2.0 will provide the detachment-attachment mechanism. In JDO 1.0, application code must do it. For the simple cases, it is not hard.

In the VLib example, the DataObjectService and the VLibDataService collaborate to insure that only value objects enter and leave the public methods of the VLibDataService .

For a simple example like VLib, the case can be made for using value objects only where needed and using managed data objects everywhere else. This would reduce the amount of garbage generated by servicing a request. Most real world Web applications are not simple. When confronting the inevitable complexity of real-world requirements, grand organizing principles that keep the architecture, design, coding, and debugging simple are generally worth more than a minor improvement in performance.

Encapsulate mutations of relationships in the service

Data classes like all Java classes define state and behavior. All of the state is defined in the member fields of the Java class. Some of this state is inherent to the data class. State stored in primitive fields and state stored in some system classes, such as Integer , Date and String , is the inherent state. It is inherent because the state applies directly to the object. Other state defines the relationships between data objects. A Person may relate to a Book which the person has borrowed. An Invoice may contain LineItems .

The attachment code to handled modified relationships is more complicated than the attachment code to handle modified inherent attributes. Since in JDO 1.0 the application's code must perform the attachment, it is convenient to delegate to the data service all operations that modify the relationships between data objects. The support for attachment in JDO 2.0 will give us a reason to reevaluate this argument.

When writing a Web application, there is another reason to encapsulate mutations of relationships in the service. Often the relationships between data objects are managed by the user through the user interface. In HTML, the objects being referred to by various controls can be uniquely identified by the identity string of the persistent object. When the user borrows a book, the natural thing to expect from the HTML control is the identity string of the new owner. Since the controller code does not know which person goes with the identity string, it is natural to delegate the task to the data service. This reason will remain even with the advent of JDO 2.0.

In the VLibDataService , there are several methods that modify the relationships between data objects, such as borrowBook , returnBook , and transferBooks . Each of these service methods look up persistent Book objects and persistent Person objects and call the book's setHolder and setOwner methods.

Define an interface for your value object's operations.

It is often the case that a value object should not change some state that a managed object can and must change. In the discussion above, the setHolder and setOwner methods of the Book class are not operations for value objects. Combine this with the question of what is the best value object implementation, and you have the perfect prescription for an interface. By making all of your controller and presentation code use an interface to the value objects, you create the means to control what operations the controller and presentation code perform and what implementation will perform it.

In the case of VLib example, the Book , Person , and Publisher data classes each have one of the value interfaces, IBook , IPerson , and IPublisher .

Encapsulate transactions within the service when possible

JDO does not, nor should it, encapsulate transactions. Where transactions are bounded and which code controls the boundaries is a design decision that varies from application to application. In the case of Web applications, since the life-time of the service and the persistence manager it uses is congruent with the processing of the HTTP request (always assuming that you have adopted the one-pm-per-request design pattern), transactions that JDO 1.0 can enforce must be short lived.

Since the transaction is already confined to the duration of the HTTP request processing, it makes sense to go further and confine it to the duration of a data service method call. While not necessary, this confinement simplifies the controller's code. If you find as I do that the most precious development resource is developer time, it pays to adopt the KISS solution as the first solution.

In the case of the VLib application, all JDO transactions live within the confines of a VLibDataService method call.

Add extended optimistic transactional semantics

Concurrent access to the same data is a real problem in database Web applications. In the VLib example, two people can be viewing the same book, and independently decide to borrow it. Without concurrency control, the last borrower wins. Since the view and the request to borrow it are two separate HTTP requests, JDO transactions do not apply (always assuming that you are using the one-pm-per-request design pattern.).

What is required can be called 'extended optimistic transactions' (which I suppose we could shorten to EOTS or 'e-oughts'). With EOTS, the code executing user A's request to borrow the book detects that the book has been changed since the book has been viewed. It is possible for the application to implement EOTS using JDO 1.0. There is an example of an EOTS implementation in the book Using and Understanding Java Data Objects.

The VLib example does not implement extended optimistic transactional semantics for two reasons. One, this was and remains a problem in the original Virtual Library example. Two, JDO 2.0 will provide EOTS for detached objects and it will also provide the tools to write a trivial implementation of EOTS even when the detached objects are discarded.

So this defect remains in the VLib example and is expected to be corrected with the evolution of the example as JDO 2.0 becomes available.