Vaadin 7 + Spring 3 + JPA project summary and lessons learned

This blogpost is a short summary of the Vaadin 7.0 project I did last year, including tools used, lessons learned and a bunch of screenshots to show the results.

Tools used

• Spring 3.2 including annotation based configuration, Spring Task*Executor framework
• JPA2 with Hibernate 4.2 + JPA modelgen, including Envers for auditing
• JMS 2
• Vaadin 7.0.1
• JAXB 2.1
• Spring WS 2.1
• Shiro 1.2
• JUnit 4.8
• Mockito
• Eclipse 3.7
• SOAP + SoapUI
• MySql 5.5 + H2
• Tomcat 7
• Jenkins
• Fisheye/Crucible
• Sonar
• Subversion
• Maven 3
• Java 6

Lessons learned

• Don't only trust SoapUI of being able to validate your WSDL. Also try to generate stubs with Axis2 and JAXWS generators. *AND* actually make sure you can make a call to the webservice, because on our project it did work perfectly fine in SoapUI, but not in Axis nor JAXWS. Example: soap fault in operation but not in porttype defined. It required the project to defining one or more xyz.jaxb files for JAX-WS  (using .episodes dit not work for us). Then use these in the wsimport etc. Related links: Compiling multiple WSDLs that share a common schema and Customizing Java Packages.
• If sources not found of e.g junit-4.11 (junit.org is also giving a lot of 404s), then run mvn clean dependency:sources, that will download the sources too into the .m2 repo.
• Try to keep the WSDL and XSD as semantic as possible, so use Strings instead of IDs for for example lookup /reference values like: started, finished. Makes the interface much more readable and understandable by just looking at it.
• count(case ...) is NOT supported by Hibernate criteria API, when using it it gives: java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: case near line 1, column 92 [...  Use sum(case) since that is supported.
  
  So instead of:
  
  cb.count(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), workOrder.get(WorkOrder_.priority)).otherwise(0)),
  Use:
  
  cb.sum(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), 1).otherwise(0)) 
  
  SQL equivalent:
  
  Instead of:
  
  COUNT(CASE WHEN priority = 1 THEN priority ELSE NULL END) AS prio1,
  
  Use:
  
  SUM(CASE WHEN priority = 1 THEN 1 ELSE 0 END) AS prio1,
  
  Links about this:
  
  https://forum.hibernate.org/viewtopic.php?f=1&t=992497
  https://forum.hibernate.org/viewtopic.php?p=2393060
  http://stackoverflow.com/questions/11011151/jpa-criteria-query-order-by-enum-values
  Sums replacement for count
  JPA 2.0 spec including 'case expressions' and this one and this one.
  http://stackoverflow.com/questions/775787/plsql-get-sum-for-each-day-of-week-and-total-sum-for-week-in-a-single-query
• When trying to upgrade from Vaadin from version 7.0.1 to 7.0.4 by updating the version number in the pom.xml, a serial UID problem occurred when generating the widgetset for this new version:
  
  [INFO]  at com.google.gwt.dev.Compiler.main(Compiler.java:177)
  [INFO] Caused by: java.io.InvalidClassException: com.google.gwt.dev.jjs.ast.JMethod; local class incompatible: stream classdesc serialVersionUID = 5017484276333252513, local class serialVersionUID = 9103713597467037978
  Tried it then with 7.0.2, that one worked fine! 7.0.3 gave the same error. This post talks about maybe the GWT cache being the problem. So I deleted the directory (even though the timestamp on that directory was recent!) and that worked. In my case the directory to delete was:
  
  D:\workspace\xyz\frontend\app-web\src\main\webapp\VAADIN\gwt-unitCache 

The application

Below you can find a bunch of screenshots of the resulting application Planning Optimizer or ePOp.

This is its official website where you can get it, including a promotion video on what its functionality entails and a video explaining its features.

The login screen, authentication via Shiro with a custom Realm that does a SOAP call to the backend:

The dashboard so the planner can immediately see the status + what needs attention first:

The Vaadin Charts 90 days look ahead graph shows the status for the coming 90 days. Clicking on a bar shows the orders at the bottom table for that day. A similar 12 weeks graph is available on another tab:

Different ways of looking at the status exist, for example these week-based graphs, the first one based on craft, the second one on priority:

Notice in the above graph the multiple y-axis: the bars are for the left y-axis, the lines for the right y-axis.

And other filters:

When clicking on a row in the bottom table, details for that order can be filled in:

Links (from a whitelist) and documents can be attached too (including drag 'n drop), and in the end a PDF can be generated with all the info (including the attachments + where the links are pointing to!) included:

JPA @Version annotation: when is the version updated?

In JPA you can annotate a field of an entity with @Version to specify that property as its optimistic lock value.
But the above documentation does not say when the version number is validated and updated (increased).
The short answer is: at commit time!

I came across this because I was having trouble passing on the updated version number back to the caller, after a merge() operation.

The setup

• Spring 3.1.2
• JPA 2.0
• Hibernate 4.2

Example and the solution

A client class (not in a transaction) invokes the method 

   /** 

    * Updates the order and returns the new version number for the order

    *

    * @param order the entity to update. Must be provided with the current version of the order.

    *             If not provided, StaleObjectStateException will be thrown

    */

   Date updateMe(Order orderWithNewValues);

on a service class, which is annotated with Spring @Service and Spring @Transactional. 

That method invokes 

   /** 

    * Similar java doc

    */

   Date updateMe(SomeEntity orderWithNewValues)

in a DAO which is annotated with Spring @Transactional(propagation = Propagation.MANDATORY) and @Repository.  The DAO searches for the entity to update using its name, sets the version number to the currently known version number, updates it and should return the new version number.

The class Order is annotated (amongst other things) with @Version as follows:

   @Version

   @Column(nullable = false)

   @Temporal(TemporalType.TIMESTAMP)

   private Date version;

   ...

   DateTime getVersion() {

       return version;

   }

The implementation of the DAO method implementing the update originally was:

   Order order = orderDao.findByName(name);

   if (order == null) {

       throw new IllegalArgumentException("No order with name '" + name + "' exists");

   }

   Order updatedOrderNotManaged = new Order(order);

   updatedOrderNotManaged.setVersion(orderWithNewValues.getVersion());

   updatedOrderNotManaged.setTotal(orderWithNewValues.getTotal());

   Order updatedOrderManaged = orderDao.merge(updatedOrderNotManaged);

   return updatedOrderManaged.getVersion();

Expecting that the last line (return updatedOrderManaged.getVersion()) would return the new version number, since the non-managed entity was now managed.

But it isn't! What seems to be the reason is that:

• The JPA implementation (in this case Hibernate) can do some caching, deferring the actual database access to a later moment, causing the changes to not hit the database yet.
• Not hitting the database also doesn't cause the @Version to be "triggered" (it seems).
• So the version is still at its old value!

But after the service method had returned the (old) version to the client calling it (so the transaction was committed), in the database you can see the new increased timestamp in the version column.

The solution is to force the JPA implementation to flush() to the database, such that the @Version annotation gets triggered and thus the version gets updated.

Therefor the following was added, just before the "return"  statement: 

   orderDao.flush();

That did it, now return updatedOrderManaged.getVersion()) is returning the updated version number. Should the transaction commit fail later, then no harm is done since the whole transaction is rolled back and the client should get an exception or similar so it knows it should ignore the returned version number.

Changing format of blog

This blogpost is the last of the 'Best of this Week Summary' series.
Looking at the number of subscribers, I decided it's too low to be worth the effort.

Therefore, from now on I'll only post "real" blogposts written by myself, no more weekly summaries.
Those posts will contain lessons learned, tutorials etc.
This change of format also implies that the posts will be less frequent but hopefully much more interesting and relevant to a greater audience.

For the fans, here's the last set of items for "Best of this Week Summary", 27 September - 3 October 2010:

• Definitely interesting interview with James Gosling
  


• Optimizing JPA Performance: An EclipseLink, Hibernate, and OpenJPA Comparison<
  

PS: I'll change my mind if I get at least 50 (valid, unique) requests in the comments to keep posting my weekly summaries :)

Best of this Week Summary 26 April - 02 May 2010

• Google's Chief Java Architect Josh Bloch "discusses many of the problems facing the Java community, including the ineffectiveness of J2ME, licensing problems, Java 7's late ship date, and even the JCP issues that James Gosling often opined about. He points out that these problems predate the Oracle acquisition of Sun, so everyone should just stop pointing their fingers at Larry Ellison."
  


• Another Getting Started blogpost for a Mahout-Taste based movie recommendation engine, which uses Wicket as presentation layer.
  


• Martin Fowler explaining Inversion of Control and suggesting to call it the Dependency Injection pattern. Notes that the Service Locator pattern can also remove the dependency, just as the DI pattern tries to. Describes which of these options to use. Finally he also gives pros/cons for constructor vs setter injection. Also pros/cons for configuration in the code or via configuration files.
  


• Several Javascript performance-optimization tips taken at JSConf 2010.
  


• An article "which looks at various storage mechanisms - JDBC, JPA, JavaSpaces, Java Content Repository, MongoDB, and DB4O, primarily - from the perspective of how good they are at CRUD operations and queries". The conclusion is that there is no one "best" mechanism, each has its own best application.
  

Best of this Week Summary 19 April - 25 April 2010

• An introduction to Gizzard, an open sourced sharding framework (store data across multiple computers instead of on just one) which is used by Twitter.
  


• A hands-on tutorial of creating a Spring application that uses Hibernate as JPA provider and JTA for transaction demarcation. A simple Order Processing Message Driven Bean is implemented that showcases this integration. It is deployed on a WebLogic 10.3 server.
  


• Last week Jira from the Apache Foundation was compromised. Here's a description of how the hackers gained access via XSS.
  

Best of this Week Summary 14 September - 20 September 2009

• Scala is slowly getting some more and more traction lately. Is it a potential long term replacement for Java?
  Interview with Scala creator Martin Odersky where he tells its history, future and why it's so interesting (like combining OO- and functional techniques).
  


• Six valid "takeaways on what most REST adopters can and should do to get the most from their use of this increasingly popular architectural style" summarised by Dion Hinchcliffe.
  


• Article that describes using Benerator, which is a data generator tool that can be used to feed database with pseudo-random test data.
  


• Tip that shows you how to implement composite keys with JPA and Hibernate. Check also the comments here.
  

Best of this Week Summary 13 July - 19 July 2009

• Pretty basic SOAP monitor (proxy) that allows you to edit SOAP messages before sending them on. Does not support a proxy (yet).
  


• IBM recently released Milepost GCC (website has styling from 1998 ;), a "compiler which analyses the software and determines which code optimizations will be most effective during compilation using machine learning techniques. Experiments carried out with the compiler achieved an average 18% performance improvement. The compiler is expected to significantly reduce time-to-market of new software, because lengthy manual optimization can now be carried out by the compiler." Related to that is also the Collective Tuning wiki, dedicated to developing self-tuning computing systems.
  


• ZeroTurnaround "has released the results of their "Java EE Containers - Heaven or Hell" survey. Using responses from 700 respondents, they cover topics such as: containers most often used on large projects, fastest container, redeploy times and annual costs of redeployment in a series of charts and calculations. Both the charts and raw data are made available for your own analysis."
  


• Looking for petabyte and beyond scale storage? The opensource project Ceph might be an option: it is a distributed network file system designed to provide excellent performance, reliability, and scalability.
  


• Several basic and advanced JPA implementation patterns can be found here.
  

Best of this Week Summary 16 February - 22 February 2009

• JavaFX SDK 1.1 has been released last week including mobile support. Some first impressions can be found here. There've been already over 80000 downloads of the SDK and over 100 million downloads of the runtime. The last number is technically correct, but doesn't say much about the real usage: JavaFX is bundled with the 1.6uXX runtime, so you get it whether you use it or not.
  Here's a tutorial on creating a mashup with JavaFX and a Flickr webservice.
  
  (No it will not win the beauty contest in UI design ;-)
  


• A nice collection of Java resources on the net, from websites to tools and trends. Useful for newbies, but also for experienced Java professionals: maybe there's something in there you did't yet know about!
  


• So how are we going to port applications that you built on a cloud like Amazon's EC2 to run on another cloud like Microsoft's Azure? And what about migrating data stored in a cloud? For that the Unified Cloud Interface project has started recently: "The concept is to provide a single interface that can be used to retrieve a unified representation of all multi-cloud resources and to control these resources as needed." Looking for an overview of cloud-providers out there? Check this list.
  


• First article Understanding transaction pitfalls when using JPA (EJB 3.0) or Spring in a series on developing an effective transaction strategy for the Java platform.
  


• More than 30 US and international cyber security organizations jointly released the consensus list of the 25 most dangerous programming errors that lead to security bugs and that enable cyber espionage and cyber crime. Among the experts were also members from OWASP, known for their top 10 web application vulnerabilities. Here´s seven ways to avoid those 25 programming errors.
  

Best of this Week Summary 29 December - 11 January 2009

• Interesting comparison whether and when to use SOAP or REST.
  


• OpenSocial now has Java, PHP, Ruby and Python client libraries available.
  


• A few weeks ago, PKI which uses MD5 as cryptographic hash function, has now officially been broken at the hackers convention Chaos Communication Congress 2008. Six Certificate Authorities still give out MD5-signed certificates as mentioned in the article. Since 2004 it is already known that MD5-collisions can be created for different data. Therefore, certificates should be issued with at least SHA-1 encryption. Here's a summary of what an MD5 collision is and what Mozilla and Microsoft issued as advisories. Additionally some (other) SSL issues are described.
  


• Nice summary of lessons learned during a project using GWT, Axis and JPA. Some more comments here.
  


• Paper by Kate McKinley (a researcher at iSec Partners, a San Francisco security firm) on the privacy protection mechanisms of FireFox, Chrome, IE and Safari. Conclusion: "We find current browsers are unable to extend tracking protection to third party plug-ins such as Google Gears and Adobe Flash. Some of these require no user prompting under common configurations and even expose tracking data saved with one browser sites visited by a different browser. [...] Safari on Windows fared the worst of all in these tests with respect to private browsing, and did not clear any data at all, either before entering or after exiting the private mode. On OS X, Safari’s behavior was quirky; in no case was the HTML 5 database storage cleared before or after private browsing.".
  

Best of this Week Summary 15 September - 21 September 2008

• It seems the SOA world is finally starting to take the Web-Oriented Architecture seriously.
  


• Summary of a recent whitepaper from IBM Global Services describing five high level best practices for successful deployment of an SOA. 120 IT Architects, developers, IT Specialists and project managers evaluated nearly 100 case studies, with 750 lessons learned and 650 best practices.
  


• SpringSource announced a maintenance policy for SpringSource Enterprise. Definitely check Rod Johnson's clarification comment in this thread on The ServerSide. Important to verify when you're using Spring in your company.
  


• Elaborate introduction to persistence, ORM and JPA, and a comprison of two open source persistence frameworks: iBATIS and Hibernate. Notable points: iBATIS encourages the direct use of SQL queries; it enables the object model and the data model to be independent of eachother via its data mapper (compared to a metadata mapper framework like Hibernate). Note that fully ORM tools generate SQL, where iBATIS uses SQL directly. A nice quick overview of the Hibernate architecture is described; the diagram is shown below:
  
  
  Recommendations from the article:
  
  1. Use iBATIS when you need full control of the SQL
     
  2. Don't use iBATIS when you are in full control of both the application (with its domain model) and the data model.
     
  3. Don't use iBATIS when the database is non-relational
     
  4. Use Hibernate to leverage end-to-end OR mapping
     
  5. A potential reason to use it could be that it is more easy to use for object-oriented programmers who are less familiar with SQL
     
  6. Use JPA when you need a standards-based persistence solution
     
  
  
  My own recommendations:
  
  1. With Hibernate it might be harder to tune the queries because they are generated by Hibernate. You can't really "just handover" the queries to the DBA. The DBA will have to monitor the database for bad queries.
     
  2. Hibernate gets quite complex when you have to map more complex associations (like a unidirectional many-to-many association with custom columns added to the link-table). I've never used iBatis so can't tell how hard it is for that framework.
     
  3. When trying to only use JPA defined elements, you'll find that JPA's possibilities are quite limited (e.g. caching is not well defined in JPA), and that you quickly will start to use (ORM-tool) implementation specific features. A good way to implement this, is to specify the full package path when using implementation specific annotations. That way you can see where you deviate from the JPA standard.
     

Best of this Week Summary 24 February - 02 March 2008

• Great set of JSF pitfalls and anti-patterns.
  


• Nice article explaining Web Oriented Architecture and its basics. Might it be the future of SOA? The differences between SOA and WOA are also described.
  


• A quite old article on using XSDs effectively in WSDL design, but still seems valid.
  


• Nice simple Java Persistence API (JPA) implementation for Amazon's SimpleDB.