Alexis's Blog on TReCX

Yesterday we (Adam and I) attended the 'Show & Tell' meeting in Manchester. The meeting was held at the University of Manchester Conference Centre, which is very close to Manchester Piccaddilly train station. We stayed there the previous night and went out to a local Thai restaurant with Alex and Juliette from the JoinIn project (in fact it was the former's birthday that day!).

The day was intended for the various project groups to demonstrate actual working pieces of software. I almost had a last minute 'mare with our stuff not working. I had 4 tracking stores all running on the same laptop OK, but as I had not bothered connecting to the wireless network it was baulking on startup as it could not import the XML file which contains the xml:lang attribute required by our schema. As it was, this is unnecessary anyway, because by default we have XML validation turned off. Anyway, once I'd finally tracked it down to this, it was just a case of commenting out a few lines and re-deploying the newly modified war file. I've now modified the code so that the validation mechanics are only instantiated if validation is turned on (sensible enough!). However, clearly a more comprehensive solution would be to use catalogs some how.

Anyway, I think our own presentation didn't go too badly. I thought it was a shame that our fellow members of the Bodington brethen at Leeds, the SOCKET project were unable to demonstrate their stuff properly as their main development server was down. Overall, I would have to say that I think that the R2Q2 project impressed me the most (and not just because of the nifty flash animation of an R2D2 robot they had in their presentation!).

In some ways it would be nice to think that what ever problem domain you're working in, you could start with some kind of XML schema and everything else would just drop out! To elaborate, what I mean is that if you're in the business of persisting XML, you could start with your schema and then have the code that creates and (optionally) validates the schema and then persists this all created for you automatically. Of course, if you're using a relational database backend, it will create the schema and write all the SQL for you too! Well, almost ;-). It's not that far off, but this post just captures the interventions we had to make in order for this to be (an otherwise) seamless end-to-end process!

JAXB Binding Stage

1. In the XML schema an eventList consists of a sequence of event elements. The default JAXB binding is to model this as List<Event> getEvent(); for the sake of correctness, we have a directive in the binding.xjb file to say that the corresponding property should be called events, therefore we get the correct plural form List<Event> getEvents().
2. The schema for XML schema itself, has a plethora of types related to capturing dates and / or times (check out the full list here). We went for the dateTime type in our schema (which basically captures a date and time down to the millisecond with timezone offset). JAXB created a new class to model this in Java called javax.xml.datatype.XMLGregorianCalendar. In Java it's more typical to handle a date time as a java.util.Date (or a java.sql.Date). For us it made sense for the JAXB object to model it as a java.util.Date as this is a type that all JDO implementations must be able to handle and model by default. However, this meant that we had to create an adapter class that can convert from a string in xs:dateTime format to a Date and another to perform the reverse. (Thankfully there are utility methods to help in this regard!). Then we add another directive in the binding.xjb file referring to this adapter class and an instruction to model this "property" as a Date in the corresponding JAXB class.
   

JDO mapping stage

1. It has to be said that JAXB 2.0 models XML schema enumerations beautifully now; it's as natural and as obvious a mapping was one could imagine! However, not so JDO 2.0. Certainly, for this version of the specification they could not quite agree on how to do this formally! With JPOX what you have to do is first of all download a plugin jar jpox-java5.jar and make it available to the runtime. Secondly you can choose to model the corresponding Java enum as either a JDBC VARCHAR or as an INT. For clarity when viewing the database tables during development we chose VARCHAR. This has to be added as a directive in the package.jdo mapping file.
   

apt (annotation processing tool) is a tool that comes with JDKs versions 1.5 onwards. It performs the desired transformations on annotated source code. As we need it for our builds (to run over code containing JAXB annotations) I decided to search for an Ant task to do this (not really wanting to write one myself!). The latest stable version of Ant is 1.6.5. This does not yet have an apt task. Out of interest I had a look at a checkout of the HEAD of the Ant CVS repository. This does, but the version of Ant overall is only described as 1.7-alpha. After some googling, I uncovered from a tutorial that there is one in Sun's JWSDP, or more specifically JAX-WS. You don't need all of the JWSDP, just JAX-WS 2.0 standalone is fine. The main drawback I found is that it doesn't do dependancy checking, therefore I have to add my own "up-to-date" check in the build.xml to compare the timestamps of source code and apt-generated class files, but otherwise, everything works OK.

Whilst JDO is an API that allows for non-relational databases to be the backend, such as object-oriented ones, JPOX is geared solely towards using a relational datasource as the actual back-end implementation (at the moment anyway, it is their intention to support OO ones at some point). The "standard" way to make a relational (i.e. JDBC) datasource available to an application in Java EE, is to add it to the servlet container / application server and make it available through a JNDI lookup. One of the key benefits of doing this is that it gives you database pooling without any further ado. The JDO API supports the notion of adding, then looking up a PersistenceManagerFactory in JNDI, but JPOX doesn't include the supporting classes to do this! Anyway, for this project for greater harmonization with Java EE projects in general we've chosen to make the underlying datasource available as a widely understood JDBC javax.sql.Datasource. Alternatively it is possible just to include the JDBC driver jar in the war itself and obtain a JDBC connection directly.

Whatever underlying datasource we're using, we initialize our PersistenceManagerFactory via a jpox.properties file. The key differences are that for a datasource configured without JNDI, specific properties look like this:

No JNDI

...
javax.jdo.option.ConnectionDriverName=org.apache.derby.jdbc.ClientDriver
javax.jdo.option.ConnectionURL=jdbc:derby://localhost/db/trecx-store
...

JNDI

...
javax.jdo.option.ConnectionDriverName=org.jpox.driver.JPOXDriver
javax.jdo.option.ConnectionURL=jpox:java:comp/env/jdbc/TrecxStore
...

So, where as in the no-JNDI configuration we specify the JDBC user, password, driver and connection URL in jpox.properties, with the JNDI configuration we specify it in the file
/META-INF/context.xml which ends up in the war file. This file also contains instructions including where it should go in the default JNDI context (this is how you add a datasource to JNDI with Tomcat; other servlet containers do something similar). Although some what opaque(!) the specification of a driver of the type org.jpox.driver.JPOXDriver and a connection URL of jpox:java:comp/env/jdbc/TrecxStore informs the JDO helper class that there is a javax.sql.Datasource instance available in the default JNDI context at jdbc/TrecxStore.

(These are just some musings on how one should go about modelling data. I'm sure many other projects have had similar considerations to make!)

In our project we have had the "luxury" of being able to design our data schema from scratch. In other words, we have not been given a data schema that we have to match. However, I think it's interesting to see the forces at work and know where to focus one's energies. For instance, as this is a 'web service' we've always known that there was going to be an associated schema, in this case, XML schema (although equally we could have used DTD or Relax NG). However, we've also known that the data has to be persisted some how. Therefore, we could have concentrated on the (relational) schema of the database. However, as it is, because I was particularly keen to abstract persistence away, it made sense to concentrate on the XML schema itself. By doing this we could have even used an XML database such as eXist. By opting to use JDO with JPOX, this actually generates the relational schema for us from the persistence-capable classes that we specify (which in turn, were JAXB classes generated from an XML schema!). However, you can still work in reverse; if we'd had an existing relational schema, we could add directives in the package.jdo file to specify how we wanted the properties of classes to map to columns in tables. (We do have some directives in fact; we have to specify how Java enums get mapped to columns as the JDO JCP have not agreed on how to standardize this yet!). So, intriguing things to ponder. I guess what it boils down to is you use whatever "schema" language (relational database, XML schema, etc) that you feel most comfortable with.

The fact that hyperjaxb2 is linked to from the main JAXB page is probably what led us into thinking that it was a good project to use initially. It may well turn out to be brilliant one day, but it was proving to be not exactly straightforward to use. In the mean time we had outcomes to produce and using JDO just seemed like a better option to get stuff up and running. This post outlines some of the pros and cons of the two approaches and hopefully gives some idea as to why we're now using JDO.

hyperjaxb2 (cons)

• one-man band effort.
• partially documented.
• "straddles" 3 technologies (and thus documentation!).
• uses older JAXB (JAXB 1.0).
  

JDO (pros)

• well maintained.
• well documented.
• standard API (rather than library). Therefore, leaves open the possible option of using a commerical (superior?) implementation instead.
• tools support; Eclipse plugins and Ant tasks.
  

In practice I was finding that when working with hyperjaxb2 I was having to have 3 pieces of documentation open at the same time; hyperjaxb2, Hibernate and JAXB 1.0! This is because it is a composite process, specifically, it consists of enhancements to the JAXB schema generator that uses Hibernate to achieve persistence. As I've posted previously, in terms of clarity there really is no comparison between JAXB 1.0 and JAXB 2.0. There are plans for hyperjaxb to use the latter (the confusingly named hyperjaxb3!), but this is even less mature than hyperjaxb2, (which itself is only version 0.4!) and does not have any binary releases yet.

What I like about JDO (we're using the reference implementation JPOX incidentally) is that XML and persistence stages are quite separate. What we do is use JAXB to generate classes from an XML schema. We then give those classes to JDO which makes a sub-set of these classes persistence-capable, as specified via a mappings file (package.jdo) through a process of byte-code enhancement. This persistence-capability is effectively transparent in the sense that no (public) methods are added to these classes (unlike hyperjaxb2 which for example adds a primary key method to each persistable class). We can then use these same artefacts to drive the automatic creation of the relational database schema (from a choice of vendors, naturally!). Just like Hibernate it has it's own database agnostic query language, in this case JDOQL. This makes it easier to switch database vendors without needing to change the application itself. The main motiviation we had for doing it this way, is that it should be easier to alter the event XML schema (which is far from cast in stone!) and then automatically generate all the downstream artefacts without too much thought.

In Java there are many ways of handling XML: SAX, DOM, StAX, JAXB, etc. This post attempts to outline a rationale for orienting the APIs within TReCX around JAXB in particular.

• standard mature API (due to become "core" in JDK 1.6 "Mustang").
  
• allows for symmetry i.e. reuse, in the implementation of the publish and store modules. Put another way, the inputs and outputs should be the same, as we're using the same classes on either end!
  
• allows for separation of XML and persistence layers. This makes unit testing easier.
  
• the JAXB generated classes form the POJOs that are then enhanced via JPOX to form the persistence-capable classes used in the persistence layer. (At runtime, the classes used by JAXB to perform unmarshalling / marshalling and the ones used by JPOX to handle persistence are one and the same. The persistence-capable nature of the classes is effectively "invisible" to the JAXB runtime).
  

Put another way, JAXB is more object-oriented and more closely integrated with Java.

Equally, however, it should be noted that if at some point in the future it would make more sense to implement the store module based on something like StAX, for performance or other reasons, then that should be perfectly sensible and possible. It just wouldn't be "symmetrical" with the publish module any more. However, by that stage, the JAXB implementation should have proved it's worth and done it's job!