Vaadin 7 + Spring 3 + JPA project summary and lessons learned

This blogpost is a short summary of the Vaadin 7.0 project I did last year, including tools used, lessons learned and a bunch of screenshots to show the results.

Tools used

• Spring 3.2 including annotation based configuration, Spring Task*Executor framework
• JPA2 with Hibernate 4.2 + JPA modelgen, including Envers for auditing
• JMS 2
• Vaadin 7.0.1
• JAXB 2.1
• Spring WS 2.1
• Shiro 1.2
• JUnit 4.8
• Mockito
• Eclipse 3.7
• SOAP + SoapUI
• MySql 5.5 + H2
• Tomcat 7
• Jenkins
• Fisheye/Crucible
• Sonar
• Subversion
• Maven 3
• Java 6

Lessons learned

• Don't only trust SoapUI of being able to validate your WSDL. Also try to generate stubs with Axis2 and JAXWS generators. *AND* actually make sure you can make a call to the webservice, because on our project it did work perfectly fine in SoapUI, but not in Axis nor JAXWS. Example: soap fault in operation but not in porttype defined. It required the project to defining one or more xyz.jaxb files for JAX-WS  (using .episodes dit not work for us). Then use these in the wsimport etc. Related links: Compiling multiple WSDLs that share a common schema and Customizing Java Packages.
• If sources not found of e.g junit-4.11 (junit.org is also giving a lot of 404s), then run mvn clean dependency:sources, that will download the sources too into the .m2 repo.
• Try to keep the WSDL and XSD as semantic as possible, so use Strings instead of IDs for for example lookup /reference values like: started, finished. Makes the interface much more readable and understandable by just looking at it.
• count(case ...) is NOT supported by Hibernate criteria API, when using it it gives: java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: case near line 1, column 92 [...  Use sum(case) since that is supported.
  
  So instead of:
  
  cb.count(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), workOrder.get(WorkOrder_.priority)).otherwise(0)),
  Use:
  
  cb.sum(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), 1).otherwise(0)) 
  
  SQL equivalent:
  
  Instead of:
  
  COUNT(CASE WHEN priority = 1 THEN priority ELSE NULL END) AS prio1,
  
  Use:
  
  SUM(CASE WHEN priority = 1 THEN 1 ELSE 0 END) AS prio1,
  
  Links about this:
  
  https://forum.hibernate.org/viewtopic.php?f=1&t=992497
  https://forum.hibernate.org/viewtopic.php?p=2393060
  http://stackoverflow.com/questions/11011151/jpa-criteria-query-order-by-enum-values
  Sums replacement for count
  JPA 2.0 spec including 'case expressions' and this one and this one.
  http://stackoverflow.com/questions/775787/plsql-get-sum-for-each-day-of-week-and-total-sum-for-week-in-a-single-query
• When trying to upgrade from Vaadin from version 7.0.1 to 7.0.4 by updating the version number in the pom.xml, a serial UID problem occurred when generating the widgetset for this new version:
  
  [INFO]  at com.google.gwt.dev.Compiler.main(Compiler.java:177)
  [INFO] Caused by: java.io.InvalidClassException: com.google.gwt.dev.jjs.ast.JMethod; local class incompatible: stream classdesc serialVersionUID = 5017484276333252513, local class serialVersionUID = 9103713597467037978
  Tried it then with 7.0.2, that one worked fine! 7.0.3 gave the same error. This post talks about maybe the GWT cache being the problem. So I deleted the directory (even though the timestamp on that directory was recent!) and that worked. In my case the directory to delete was:
  
  D:\workspace\xyz\frontend\app-web\src\main\webapp\VAADIN\gwt-unitCache 

The application

Below you can find a bunch of screenshots of the resulting application Planning Optimizer or ePOp.

This is its official website where you can get it, including a promotion video on what its functionality entails and a video explaining its features.

The login screen, authentication via Shiro with a custom Realm that does a SOAP call to the backend:

The dashboard so the planner can immediately see the status + what needs attention first:

The Vaadin Charts 90 days look ahead graph shows the status for the coming 90 days. Clicking on a bar shows the orders at the bottom table for that day. A similar 12 weeks graph is available on another tab:

Different ways of looking at the status exist, for example these week-based graphs, the first one based on craft, the second one on priority:

Notice in the above graph the multiple y-axis: the bars are for the left y-axis, the lines for the right y-axis.

And other filters:

When clicking on a row in the bottom table, details for that order can be filled in:

Links (from a whitelist) and documents can be attached too (including drag 'n drop), and in the end a PDF can be generated with all the info (including the attachments + where the links are pointing to!) included:

Lessons learned webservices project

At a recent project we used the following tools & frameworks:

• Java 6


• Spring 3.0


• Hibernate 3.2.7


• JMS


• JBoss ESB 4.5/5.1


• SOAP 1.1


• JAXB


• JAX-WS


• Apache CXF


• Maven2


• Jetty 6.1


• Oracle 10g


• Eclipse Mylyn


• SVN


• JUnit


• Bamboo


• Jira


• Crucible


• Nexus

Below are a couple of lessons learned which I remembered to write down:

JUnit

• To skip a testclass (e.g Util.java) during a test in a maven project: put @Ignore above it.

Java

• Throw checked exceptions in case of functionality errors. To not have to return those errors as return-values of the methods and making them checked, makes the interface clearer.

SOAP

• Should you use the service response for functional errors, and SOAP fault only for technical errors?
  Nothing in the SOAP 1.1 spec says you should do that or are not allowed to put functional errors in the <detail>.
  Makes sense to me to do that: otherwise you can get errors in the response AND in the soap fault. More complex to handle in the client...
  Less of an issue if you say when there are SOAP faults we just give up, only if there's a response we try to see what's going on.
  


• Apparent best practice: no underscores in xml/xsd/wsdl for element nor attributes, for better readability

ESB

• JBoss ESB 4.5 JMS queue had loads of problems with high load and would sometimes just completely drop the queue. We moved to JBoss ESB 5.1 with ActiveMQ, that proved much more stable.

Transactional atomic file manipulation and file locking
Bunch of interesting articles related to this:

• http://stackoverflow.com/questions/595631/how-to-atomically-rename-a-file-in-java-even-if-the-dest-file-already-exists


• http://commons.apache.org/transaction/


• http://www.infoq.com/news/2008/01/file-systems-transactions


• http://www.coderanch.com/t/495417/java/java/Distributed-Tx-Transactional-File-Access


• https://xadisk.dev.java.net/


• Lower-level stuff: http://java.sys-con.com/node/37798


• EJB/XA example: http://matthewneale.net/category/software-development/

Related to file locking:

• See reply 3: http://forums.sun.com/thread.jspa?threadID=5284898


• Another example: http://www.java2s.com/Code/Java/File-Input-Output/DemonstratesfilelockingandsimplefilereadandwriteoperationsusingjavaniochannelsFileChannel.htm


• "But if you can change/write the writing program: You have to change the design. App1 should write to "temporary" files, and when it's done, close and rename them to a naming convention that App2 will recognize as being "finished files, ready for processing"."


• Lock service, platform independent: http://www.devx.com/Java/Article/7870/1954. But requires all access via this service, so external FTP programs for example don't see the lock & the java program won't see a lock so read it.

Maven

• With Maven you can also have it put the sources + javadocs in the repository.
  Running 'mvn eclipse:eclipse' after that, and refresh the project in Eclipse, within Eclipse you can then see the javadocs and debug the sources.
  Just make sure you set <downloadSources> and <downloadJavadocs> to true.
  

Misc

• If you have an action that can't be made transactional (e.g a file move), do that as last thing in your steps. More precisely: start a (e.g database) transaction, move the file. If that move succeeds, update status that move was successful. If that move fails, rollback the transaction, and thus is the status of that file still in "to move".

Best of this Week Summary 13 July - 19 July 2009

• Pretty basic SOAP monitor (proxy) that allows you to edit SOAP messages before sending them on. Does not support a proxy (yet).
  


• IBM recently released Milepost GCC (website has styling from 1998 ;), a "compiler which analyses the software and determines which code optimizations will be most effective during compilation using machine learning techniques. Experiments carried out with the compiler achieved an average 18% performance improvement. The compiler is expected to significantly reduce time-to-market of new software, because lengthy manual optimization can now be carried out by the compiler." Related to that is also the Collective Tuning wiki, dedicated to developing self-tuning computing systems.
  


• ZeroTurnaround "has released the results of their "Java EE Containers - Heaven or Hell" survey. Using responses from 700 respondents, they cover topics such as: containers most often used on large projects, fastest container, redeploy times and annual costs of redeployment in a series of charts and calculations. Both the charts and raw data are made available for your own analysis."
  


• Looking for petabyte and beyond scale storage? The opensource project Ceph might be an option: it is a distributed network file system designed to provide excellent performance, reliability, and scalability.
  


• Several basic and advanced JPA implementation patterns can be found here.
  

Best of this Week Summary 29 December - 11 January 2009

• Interesting comparison whether and when to use SOAP or REST.
  


• OpenSocial now has Java, PHP, Ruby and Python client libraries available.
  


• A few weeks ago, PKI which uses MD5 as cryptographic hash function, has now officially been broken at the hackers convention Chaos Communication Congress 2008. Six Certificate Authorities still give out MD5-signed certificates as mentioned in the article. Since 2004 it is already known that MD5-collisions can be created for different data. Therefore, certificates should be issued with at least SHA-1 encryption. Here's a summary of what an MD5 collision is and what Mozilla and Microsoft issued as advisories. Additionally some (other) SSL issues are described.
  


• Nice summary of lessons learned during a project using GWT, Axis and JPA. Some more comments here.
  


• Paper by Kate McKinley (a researcher at iSec Partners, a San Francisco security firm) on the privacy protection mechanisms of FireFox, Chrome, IE and Safari. Conclusion: "We find current browsers are unable to extend tracking protection to third party plug-ins such as Google Gears and Adobe Flash. Some of these require no user prompting under common configurations and even expose tracking data saved with one browser sites visited by a different browser. [...] Safari on Windows fared the worst of all in these tests with respect to private browsing, and did not clear any data at all, either before entering or after exiting the private mode. On OS X, Safari’s behavior was quirky; in no case was the HTML 5 database storage cleared before or after private browsing.".
  

Best of this Week Summary 04 November - 16 November 2008

• Part 2 in a series on server load architectures: application-level load balancing, including client-stickiness.
  


• How to write maintainable CSS; a presentation by Natalie Downe.
  


• ProgrammableWeb is tracking APIs and has now reached the 1000 milestone. In the pie charts you can see the top 15 categories and which protocols they use: 63% of the APIs uses REST! 22% SOAP. The protocols diagram is shown below:
  
  


• Here's some more details on why Google forked OpenID. See my related post of two weeks ago.
  


• A presentation with the current state (it's growing big time!) of OpenSocial shown at OpenSocial's first birthday at MySpace's headquarters.
  


• I'm currently trying out Wicket with Spring and Hibernate, using AppFuse Light as a starting point. In case you're looking for an introduction to Wicket, check out this one that has been around for a while, and this new one that's been posted this week.
  

Best of this Week Summary 05 May - 11 May 2008

• Free IBM Redbook with SOA and webservices patterns.
  


• Eclipse plugin Monkey to automate tasks like Java projects that don't have a Checkstyle configuration defined or find commented out code and defines tasks for them (TODO/FIXME),
  


• The potential doubts regarding REST in practice, mainly for designers/developers with a SOAP/WSDL background.
  


• An overview of Agile anti-patterns, like "keep refactoring for later" and "let's fix the testcase later, we'll uncomment it for now".
  

Best of this Week Summary 28 April - 04 May 2008

• Nice overview on how Dutch social network Hyves (> 5M users) uses Gentoo.
  


• An explanation how to setup soapUI with Maven for automated SOAP testing.
  


• Interesting rumor is that Twitter might be moving away from using Ruby on Rails; it might move to PHP or Java. The potential reason for switching is most likely scalability. Their system has to support about 3 million messages a day and has over a million users (march 2008 numbers).
  


• Springsource (the guys from Spring) introduced their beta Application Platform (aka S2AP). Summarizing: "an application platform, built on OSGi, that can resolve Spring bean references by looking up services through OSGi - all of the capabilities OSGi provides, without the headache of deployment. And it would be able to deploy .WAR files through the use of a Tomcat (or servlet container) service."
  

Best of this Week Summary 7 April - 12 April 2008

• 12 great points you should know about WOA and REST. Note that these points are not all about technology only...
  


• Difference between WOA and SOA shortly discussed.
  


• New from Google: run your Python webapplication on their infrastructure with Google App Engine. Quite a contrast to Amazon's webservices approach S3, EC2 and SimpleDB.
  
• A status check on OpenID, including where you can get an OpenID (providers), resources and where you can use it (couple of major sites only).
  

Best of this Week Summary 25 March - 30 March 2008

• What is required to start building an RIA + SOA application, and how you could enable these activities. A potential answer could the Appcelerator platform mentioned in the article.
  


• Nice collection of open source software testing tools (and news and discussion :-). These are the listed Javascript unittesting tools.
  


• For the coders: the number of types in the .NET framework visualized (I know, I know, .NET but it's a nice insight anyway ;-)
  


• Comparison of performance on many browsers of the Backbase Javascript engine and some widgets. In general interesting to see how these browsers compare. Includes IE8, Firefox 3 Nightly Build and Safari Nightly Build.
  


• Three open source SOAP testing tools compared: Eviware SoapUI 1.6, PushToTest TestMaker, and WebInject's WebInject. Conclusion: The writer prefers "[...] the middle balance struck by soapUI. The skeletal tests created by soapUI's wizard were easier to flesh out than those built by TestMaker. And, if I needed to do something elaborate and off the wall, I could always call upon soapUI's Groovy capabilities – funny name aside, they do their job well." "In terms of how these products compare to commercial Web service testing tools, I'd say it's a mixed bag. They are, of course, inexpensive (free), and work well for easy to moderately-difficult jobs; on the other hand, they're somewhat less user-friendly than commercial tools and if you need to do something complex, you have to build it yourself."
  


• Five commercial SOAP testing tools compared: AdventNet's QEngine, Crosscheck Networks SOAPSonar, iTKO’s LISA, Mindreef's SOAPscope Server, and Parasoft's SOAtest. Conclusion: "If your testing involves more than just Web services, and your development is primarily Java, then tools such as LISA or SOAtest are worth considering [...]. If, however, you are only interested in SOAP-based Web service testing, and your QA staff is relatively new to the technology, SOAPscope is the obvious choice." But maybe you want to let it depend on how you want to build the tests: by coding or visually with a GUI? This is what SOAPSonar, LISA, and SOAPscope have done. The writer favors coding tests and SOATest came out as winner.
  

Best of this Week Summary 20 - 26 May 2007

• This is a very interesting story about the the converging of the concepts closely related concepts SOA and Web 2.0.
  

  "The core principle of SOA is the decomposition of software into sets of
  services which can be used and composed into new applications that have a very high level of integration and reuse."

  "Web 2.0 is more of a pragmatic extraction of what actually works best in online product design than a rigorous a priori engineering exercise."

  A difference that is still there is that Web 2.0 sees data as the most important, in SOA services are the most important. The diagram in the post nicely shows the relationship between the two. It mentions also that it is interesting to see that the market chose REST as being web-oriented, not SOAP.
  Related to this, note this interesting new tool from Microsoft: Astoria (download). It allows any ADO compliant database (e.g MS SQL Server) to be accessible via REST, thus web-enabling it.
  
  
• Here is an interesting extract from Mark Hansen's book "SOA Using Java Web Services". It explains REST and SOAP and their differences. Then it describes the tools and techniques for implementing SOA Java components using the REST paradigm. As an example the integration of an Order Management System (e.g. SAP) with a Customer Service System (Siebel CRMA) is used. Both the client and server side are explained. The example is shown w/o JWS and then with JWS. For many decisions an explanation is given, some are very basic though (e.g why not to store redundant data). Data transformation via XSLT using the JAXP API is also described.
  
  One major tip: even for REST, use XML Schema (XSD) to define the message structure interface, such that client applications can comply to the valid message structure.
  
  
• A quick look at Guice , the dependency injection framework by Google. It discusses a google tech talk about Guice and then goes through some examples on how you would configure in Guice vs Spring. In Guice you configure everything with annotations instead of XML. The writer isn't convinced this is better, neither am I.
  
  
• A short description and list of 15 free SQL injection scanners. Amongst others for MySQL, MS SQLServer, Oracle.