Vaadin 7 + Spring 3 + JPA project summary and lessons learned

This blogpost is a short summary of the Vaadin 7.0 project I did last year, including tools used, lessons learned and a bunch of screenshots to show the results.

Tools used

• Spring 3.2 including annotation based configuration, Spring Task*Executor framework
• JPA2 with Hibernate 4.2 + JPA modelgen, including Envers for auditing
• JMS 2
• Vaadin 7.0.1
• JAXB 2.1
• Spring WS 2.1
• Shiro 1.2
• JUnit 4.8
• Mockito
• Eclipse 3.7
• SOAP + SoapUI
• MySql 5.5 + H2
• Tomcat 7
• Jenkins
• Fisheye/Crucible
• Sonar
• Subversion
• Maven 3
• Java 6

Lessons learned

• Don't only trust SoapUI of being able to validate your WSDL. Also try to generate stubs with Axis2 and JAXWS generators. *AND* actually make sure you can make a call to the webservice, because on our project it did work perfectly fine in SoapUI, but not in Axis nor JAXWS. Example: soap fault in operation but not in porttype defined. It required the project to defining one or more xyz.jaxb files for JAX-WS  (using .episodes dit not work for us). Then use these in the wsimport etc. Related links: Compiling multiple WSDLs that share a common schema and Customizing Java Packages.
• If sources not found of e.g junit-4.11 (junit.org is also giving a lot of 404s), then run mvn clean dependency:sources, that will download the sources too into the .m2 repo.
• Try to keep the WSDL and XSD as semantic as possible, so use Strings instead of IDs for for example lookup /reference values like: started, finished. Makes the interface much more readable and understandable by just looking at it.
• count(case ...) is NOT supported by Hibernate criteria API, when using it it gives: java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: case near line 1, column 92 [...  Use sum(case) since that is supported.
  
  So instead of:
  
  cb.count(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), workOrder.get(WorkOrder_.priority)).otherwise(0)),
  Use:
  
  cb.sum(cb.selectCase().when(cb.equal(workOrder.get(WorkOrder_.priority), 1), 1).otherwise(0)) 
  
  SQL equivalent:
  
  Instead of:
  
  COUNT(CASE WHEN priority = 1 THEN priority ELSE NULL END) AS prio1,
  
  Use:
  
  SUM(CASE WHEN priority = 1 THEN 1 ELSE 0 END) AS prio1,
  
  Links about this:
  
  https://forum.hibernate.org/viewtopic.php?f=1&t=992497
  https://forum.hibernate.org/viewtopic.php?p=2393060
  http://stackoverflow.com/questions/11011151/jpa-criteria-query-order-by-enum-values
  Sums replacement for count
  JPA 2.0 spec including 'case expressions' and this one and this one.
  http://stackoverflow.com/questions/775787/plsql-get-sum-for-each-day-of-week-and-total-sum-for-week-in-a-single-query
• When trying to upgrade from Vaadin from version 7.0.1 to 7.0.4 by updating the version number in the pom.xml, a serial UID problem occurred when generating the widgetset for this new version:
  
  [INFO]  at com.google.gwt.dev.Compiler.main(Compiler.java:177)
  [INFO] Caused by: java.io.InvalidClassException: com.google.gwt.dev.jjs.ast.JMethod; local class incompatible: stream classdesc serialVersionUID = 5017484276333252513, local class serialVersionUID = 9103713597467037978
  Tried it then with 7.0.2, that one worked fine! 7.0.3 gave the same error. This post talks about maybe the GWT cache being the problem. So I deleted the directory (even though the timestamp on that directory was recent!) and that worked. In my case the directory to delete was:
  
  D:\workspace\xyz\frontend\app-web\src\main\webapp\VAADIN\gwt-unitCache 

The application

Below you can find a bunch of screenshots of the resulting application Planning Optimizer or ePOp.

This is its official website where you can get it, including a promotion video on what its functionality entails and a video explaining its features.

The login screen, authentication via Shiro with a custom Realm that does a SOAP call to the backend:

The dashboard so the planner can immediately see the status + what needs attention first:

The Vaadin Charts 90 days look ahead graph shows the status for the coming 90 days. Clicking on a bar shows the orders at the bottom table for that day. A similar 12 weeks graph is available on another tab:

Different ways of looking at the status exist, for example these week-based graphs, the first one based on craft, the second one on priority:

Notice in the above graph the multiple y-axis: the bars are for the left y-axis, the lines for the right y-axis.

And other filters:

When clicking on a row in the bottom table, details for that order can be filled in:

Links (from a whitelist) and documents can be attached too (including drag 'n drop), and in the end a PDF can be generated with all the info (including the attachments + where the links are pointing to!) included:

Best of this Week Summary 2 August - 8 August 2010

• How to utilize Externalization (faster, less flexible) for high performance applications instead of Serialization (slower, more flexible).
  


• A bunch of pros and cons of Wicket.
  


• An extensive tutorial on how to work with XML, JSON and Google's protocol buffers within Android applications. It shows how to "develop a Web service that converts CSV data into XML, JSON, and protocol-buffers formats. Then you'll build a sample Android application that can pull the data from the Web service in any of these formats and parse it for display to the user". Assumes Froyo 2.2. Also shows a short comparison in performance between the three protocols.
  


• A five part introduction to HTML5 focusing on the mobile web.
  


• Highlights from Black Hat and Defcon 2010. That means security stuff :).
  
  


• A list of (mostly obvious) performance problems identified at Zappos, Monster etc.
  


• A pretty comprehensive overview of Java code obfuscating including these types: name, strings, code and flow, incremental, watermarking etc.
  

Best of this Week Summary 22 March - 28 March 2010

• Whatever happened to programming? These days it is seems only glueing together a bunch of libraries... Where's the creativity? Well, I think you can now more and more focus on what your app can/should do, instead of re-inventing the wheel all the time. And if you have a real unique idea, you still will have to write those libs too :)
  


• Versioning REST services: "when developers need to create a new endpoint versus just adding data".
  


• A basic comparison between Kindle (JME) & Android Java development.
  

Best of this Week Summary 1 March - 7 March 2010

• Of course is the contract between webservices more important than their internals.
  


• How the static analysis tool Coverity was turned into a commercial product. Includes caveats, lessons learned etc.
  Check also how you can measure code quality with Sonar.
  


• The updated 2010 list of most dangerous programming errors.
  


• Four presentations from John Resig on jQuery, and especially Query 1.4.
  

Best of this Week Summary 8 February - 14 February 2010

• Five new features of Spring 3.0 described:
  
  
  • Spring Expression Language (SpEL)
    
  
  
  • Object/XML Mapping (OXM) module
    
  
  
  • Type-conversion system
    
  
  
  • Formatter SPI
    
  
  
  • RESTful web services support
    
  
  What new Ajax simplifications have been introduced can be found here.
  


• The Java EE 6 web tier introduces with Servlet 3.0 spec the ability for asynchronous processing of requests so that the thread may return to the container and perform other tasks. It also "adds annotation based configuration (@WebServlet, @ServletFilter and @WebServletContextListener) making the web.xml file optional, and introduces a new concept of web fragments."
  


• Introductionary article to Apache Click. One difference between Click and Wicket is that Click is stateless by design, while Wicket is stateful. Both can be configured their "opposite" model, though these have their own pros/cons.
  Another main difference is that that Click uses templates (Velocity by default) where Wicket does not.
  
  Sidestep: here's a bunch of nicely designed Wicket sites that scale (note: not really clear where the scalability statements come from).
  


• A 60 minutes presentation on how SOA is a hit at eBay.
  

Best of this Week Summary 07 December - 13 December 2009

• This week Java EE 6 and GlassFish 3.0 were released.
  
  
• Presentation on the Java architecture of SpotiChart.com, a site that creates a Spotify chart based on Spotifiy tweets on songs.
  Includes:
  
  
  • Spring 3rc1 + Spring MVC 3
    
  
  
  • Quartz
    
  
  
  • Maven 3
    
  
  
  • Hibernate, EHCache
    
  
  
  • Transactional & multi-threading
    
  
  
  • Ubuntu 9.04
    
  
  
  • Tomcat 6 (single war)
    
  
  
  • MySQL 5.1
    
  
  
  • Twitter4j
    
  
  
  • OpenFlashChart
    
  
  
  • JTreeMap
    
  
  
  • Rome (for Atom feeds)
    
  
  
  • Use of URL shortening services
    
  
  
  • Developed in 3 days(!)
    
  
  
  • Statistics of the site
    
  
  
  • Lessons learned
    
  
  


• Five Javascript performance tips.
  


• Read the interview with the writer of Javas SOA cookbook, including how he uses Agile in SOA projects. Includes a free download of the Governance chapter.
  

Best of this Week Summary 16 November - 22 November 2009

• Did you know Java 5 reached its End of Service Life last month? But luckily not if you've got Java SE for business (support).
  


• This week Google introduced the first version of its Chrome OS. Here's a pretty good explanation on how it differs from current operating systems. And here are 5 (other) points describing them:
  
  
  • "The browser is the OS",
    
  
  
  • "It’s designed to fix itself",
    
  
  
  • "All apps are web apps, no installations", (sounds like Citrix to me ;)
    
  
  
  • "Chrome OS doesn’t support drivers and will not run on your laptop",
    
  
  
  • "Super-fast startup speeds"
    
  
  


• A step-by-step tutorial on creating a CRUD Web application and run it on the Amazon Elastic Compute Cloud. The web application leverages Groovy, Spring, and Hibernate (via the Grails framework), MySQL and Tomcat. For deployment the Eclipse AWS-plugin is used.
  


• Great, here you can download chapters 20, 21, 22 for free from the book "Web Service Contract Design and Versioning for SOA" by Thomas Erl, Anish Karmarkar, Priscilla Walmsley, Hugo Haas, L. Umit Yalcinalp, Canyang Kevin Liu, David Orchard, Andre Tost, James Pasley. These chapters address the issues related to service contract versioning. In those chapters described are versioning of WSDL and schema definitions.
  

Best of this Week Summary 09 November - 15 November 2009

• A post on experiences with Git coming from Subversion. Too bad the reasons why the author is having more and more problems with Subversion were not given. Though you can deduct (some(?) of) them from the text.
  


• Presentation by John Resig (from JQuery fame) on testing javascript. Shortly describes QUnit, JSUnit, YUITest for unittesting, Screw.Unit, JSSpec for behavior testing (unittesting broken up by task), Selenium for functional testing and Crosscheck, Env.js, Blueridge for server-side Javascript testing. Also describes browser launching, which I would translate into "build/test management", distributed testing (Selenium, TestSwarm) and browser support (cost vs benefit).
  


• "A Decade of SOA: Where are we, Where are we Going?" The viewpoints from a virtual panel of Enterprise Architects who have lived and implemented SOA for most of this decade.
  


• Why iStockphoto embraced Google's Gears.
  


• And as grand final this week: the hardware and (rails) software architecture of Ravelry, a knit and crochet community.
  

Best of this Week Summary 21 September - 27 September 2009

• Always feeling a bit less up-to-date when using RSS/Atom feeds? Well, convince your RSS/Atom provider to implement the PubSubHubBub protocol: it provides a callback-hook, which when registered to as feed subscriber, causes an immediate update (ping) to the subscriber (instead of waiting for the next RSS/Atom feed request).
  Or will rssCloud be the winner of this realtime RSS extension? Here's the basic (simpler) flow of rssCloud:
  
  
  Differences between the two are described here (including some issues) and here. A real old-fashioned flame-war has even started between the creators of the two protocols.
  And a good comparison which explains quite well why PuSH is the better choice of the two. Note that both protocols seem to be server-to-server only! (e.g think when you're behind a firewall or NAT...)
  See this article for an explanation of the PuSH flow of notifications/information in below's diagram:
  
  
  Update: PubSubHubbub explained by co-creator Brett Slatkin (an engineer at Google).
  
  
• Want to get to know Hadoop? In short "Hadoop allows you to write and run your application in a distributed manner and process large amounts of data with it. It consists out of a MapReduce implementation and a distributed file system." Check the rest of this introduction for more details.
  


• Five Java anti-patterns to prevent (out-of) memory problems.
  


• Three Best and three worst practices in BPM and SOA.
  


• "Tom Killalea, Vice President of technology with responsibility for infrastructure and distributed systems engineering at Amazon.com wrote an article on ACM queue on building scalable web services. He outlines guiding principles to building scalable web services with a lot of real-world examples, the core theme of which is “build only what you need”."
  

Best of this Week Summary 07 September - 13 September 2009

• Wondering whether you should add this new piece of functionality quick & dirty or take a bit longer, but cleaner? Taking the first one sets you up for a Technical Debt, as some more explained by Martin Fowler.
  


• "Test logic can be buried in this unrelated code, which has nothing to do with test logic itself, making test code hard to read and maintain. In this article, the layered architecture of test automation is presented to solve this problem. In this layered architecture, the test automation code is divided into three layers: (1) test cases, focusing on the test logic of the application, (2) the domain layer, modeling the system under test in domain terms, encapsulating http requests, browser control, result parsing logic and providing an interface for the test cases layer, (3) the system under test, which layer 2 will operate directly on."
  


• A good explanation of the Canonical Message Model in the context of an ESB, and why you'd need one within an SOA. Great description of different approaches for implementing a CMM in an ESB and their pros and cons.
  A short introduction to CMM modelling (data model and message model) is described here.
  


• From The Open Group two free PDF books: "The Open Group SOA Integration Maturity Model (OSIMM) provides consultants and IT practitioners with a means to assess an organization’s Service Oriented Architecture (SOA) maturity level."
  "This document describes a framework that provides context and definitions to enable organizations to understand and deploy SOA Governance."
  


• The JavaServer Faces 2.0 specification (included in JEE6) got finalised last May. Here's a reference to it and a set of references related to this specification.
  

Best of this Week Summary 3 August - 9 August 2009

• Securing SOAP/webservices overview and tips including encryption, testing, maintainability and reusability.
  


• Four steps to securing webservices with focus on testing.
  


• Several junit best practices. And related antipatterns.
  


• Overview of StackOverflow.com's architecture, including general tips. The tools are MS, but the mentioned principles apply everywhere. Quite a lot on scaling up and out. Also included is how Saleforce.com solved its perforance problems in the database (Oracle).
  


• Awesome, parts of the original source code of Apollo 11's Command Module (Comanche054) and Lunar Module (Luminary099)! Pretty funny are some of the comments.
  


• A couple of more or less advanced Firebug usage tips: advanced debugging with the console, replacing a Javascript funtion in the console and profiling.
  

Best of this Week Summary 23 March - 29 March 2009

• Need to know whether a collection is modified concurrently, which causes the ConcurrentModificationException? Check this handy wrapper solution here.
  


• The second article in the Transaction strategies series. "Using examples from the Spring Framework and the Enterprise JavaBeans (EJB) 3.0 specification, explained is how the transaction models work and how they can form the basis for developing transaction strategies ranging from basic transaction processing to high-speed transaction-processing systems".
  


• For the Amazon EC2 now a free AWS Eclipse plugin is available. It facilitates remote development, deployment, and debugging Java applications on Amazon Web Services.
  


• Three common mistakes to avoid when implementing XML and webservices: don't make your messages too large, don't create too fine-grained services, and don't forget to create an XML schema.
  


• Javascript is getting more and more important the last couple of years, so here's a bunch of best practices and efficiency tips.
  

Best of this Week Summary 26 January - 01 February 2009

• Part 1 of a series on how to represent XML schema types taking into account the mismatch between XML and the datatypes of a programming language.
  


• Nice non-techie explanation of what OAuth is. Plaxo and Google are now trying out a combination of OpenID for federated authentication and OAuth for the authorization process, and including the Google Contacts API in the workflow. Thus, more and more towards the Open Stack.
  


• A great series of SOA Patterns by Thomas Erl. This is the first one, a Service Façade.
  


• An interesting opensource effort to make consuming REST services more object oriented and Rails ActiveResource like with Rapa. "Rapa uses the tried and tested Jakarta Commons HttpClient and JAXB."
  


• And as the last item this week, get ready to test your websites with Internet Explorer 8, RC1 can be downloaded now and should be nearly identical to the final version.
  

My clients are getting thicker and thicker: MVC implementation shift

Last week I stumbled over this interesting article on the potentially changing Model View Controller architecture implementations. (Sidenote: here's the more subtle distinction between MVC1 and MVC2 described. And here's an article on the ambiguity of the term and pattern MVC.)
An example logical diagram of MVC2 is shown below:


For quite some years, the general practice was to generate HTML as result of the View, with the browser rendering it and being the thin web client. These days though, more and more the web client is getting heavier ("fatter") because of all the Javascript being used for AJAX, SPI and, in general, more logic on the web client.
The article takes a look at the different approaches to MVC implementations:

1. the HTML-only approach;


2. the HTML+Javascript "basic" approach; some of the Model's "business logic" is duplicated on the client;


3. the "AJAX" approach; partial views can be updated, the view consists of multiple HTML fragments, Javascript creates the View, not the application server;


4. and the full-blown Javascript-only ("a la gmail") approach; Javascript in the browser creates the views, near-complete responsibility for the Model business rules in the web client (browser), the web client would be the Controller.

The approaches range from, at the one extreme end, the definitive "thin client", and at the other extreme end, the definitive "thick client"." Example frameworks that can easily facilitate the last option are Google Web Toolkit (Java to Javascript) and Pyjamas (Python to Javascript). Below GWT's architecture diagram is displayed. Note the (relatively) limited application server procedures, part of the Model, preferably only performing basic validation, security, etc.


The new MVC2 diagram for the above mentioned fourth approach would look something like this:

Included in the article is also how you could split up work in a team between people working on the View and Model parts.

I do not completely agree with the statement in the beginning of the article that says "The Web client takes care of keystrokes and mouse movements: the Web browser application itself is the Controller". I'm reading this as: the browser is the controller. Though, when you look at the first picture, that is not what the Controller is supposed to be.
But, in general the whole potential movement of going to thick clients is an interesting one to keep an eye on.
Other pros for thicker Javascript-based clients (not mentioned in the article) are:

• Easier distribution of new versions than in the old days with fat clients like Visual Basic, where you had to make sure all clients got updated;


• Easy opening up of your SOA webservices. The fat client could directly access them;


• Easier mashup possibilities at the View layer via Javascript;


• Potentially a large improvement in scalability: the client processing power is used as much as possible, the application server only performs several basic checks;


• The Javascript generator/builder takes care of browser compatability.

Several cons are though:

• Potentially quite significant longer page loading times when the Javascript all has to be loaded at once;


• If not designed correctly, there could be significant extra network traffic;


• Being dependent on the Javascript generator/builder when a browser is (not anymore) supported (though an opensource framework allows you to build it yourself if really needed).

Best of this Week Summary 17 November - 30 November 2008

• Interesting recent long discussion (with a bit of framework flame-war in between) and tips on when to pick what web client framework. Tapestry, Wicket, Stripes, GWT, Flex seem to jump out.
  


• Interesting Javascript optimizer: Razor Optimizer. It dynamically analyzes the runtime profile information, trying to load each Javascript function only when needed, potentially creating 60-90% savings. It does require a client and server component. The question is: are you prepared to take a (small?) risk that your Javascript is dynamically modified?
  


• Good article that explains why Scrum often fails as implementation of Agile. Lots of times Scrum is only seen as fast Sprints and Scrums. But: "Don't leave out the 'regular' agile good engineering practices". Check also the comments for good tips.
  


• A good introduction to RESTful webservices.
  


• Great, soapUI 2.5 now also suppports testing RESTful webservices. It includes testing JSON output.
  

Best of this Week Summary 20 October - 26 October 2008

• Good tips on how to write clean code.
  


• A high level comparison of JAX-RS (Java API for RESTful Web Services) four implementations: CXF (Apache), Jersey (Sun), RESTEasy (JBoss), and Restlet (open source). Here's how they integrate with Spring.
  


• Four approaches to get to a canonical message model (CMM) in an ESB. Check also the previous two articles in this series.
  

Best of this Week Summary 12 May - 18 May April 2008

• How to setup Spring with Wicket. This is a "challenge" because Wicket is an unmanged framework, this in contrast to Spring. I'm personally now using the Annotation-based Approach.
  


• IBM developerWorks overview of services-based enterprise integration patterns, 3 parts.
  


• Interview with Donald Knuth regarding the success of open source, the problem with multicore architectures, reusable code and unit testing.
  


• Nice bunch of tips that can help you decide when to use ESB or BPEL in your solution.
  

Best of this Week Summary 05 May - 11 May 2008

• Free IBM Redbook with SOA and webservices patterns.
  


• Eclipse plugin Monkey to automate tasks like Java projects that don't have a Checkstyle configuration defined or find commented out code and defines tasks for them (TODO/FIXME),
  


• The potential doubts regarding REST in practice, mainly for designers/developers with a SOAP/WSDL background.
  


• An overview of Agile anti-patterns, like "keep refactoring for later" and "let's fix the testcase later, we'll uncomment it for now".
  

Best of this Week Summary 21 January - 27 January 2008

• Nice routing pattern for webservices that need to provide both synchronous and a-synchronous calls. You can even make the router such that depending on certain criteria (like load, response time), it will either turn the call into an a-synchronous call when needed. Disadavantage of this approach is of course that the caller has to be able to support both an a-synchronous and a synchronous version of each call. Related patterns you can find in these seven Self-Service: select application patterns.
  


• An effort to give answer to the question whether Tomcat is an application server or not. GREAT introduction to JEE btw, so good read for beginners or as a refresher. The answer: Tomcat does not support the entire Java EE stack: no distributed transactions, EJBs, and JMS.
  


• IBM made their Jazz tool available for the public. It can be used to "build a scalable, extensible team collaboration platform for integrating work across the phases of the development lifecycle." Note that it is not fully opensource yet. There's also a Mylyn connector available for Jazz.
  


• Version 3.0.0 of HTML Purifier has been released. It will scan your HTML for malicious XSS code, check for compliance and will try to fix your code.
  


• Summary of a statement made by Bruce Eckel in this article that Java should not change much anymore: it should stop growing and just become stable, and Sun should focus on optimizing/extending the JVM. Interesting considerations.
  


• And finally for this week, for the developers among us, a bunch of shortcut keys in Eclipse you might not know about.
  


• The Open Source hardening Project of the US Homeland Security department together with Coverity has led to the milestone where in 11 opensource projects all main bugs have been fixed. These 11 have been moved up to the new level Rung 2. Here you can see all the results of the scan.
  

Best of this Week Summary 03 Sept - 08 Sept 2007

• Good interview with IBM VP about BPM and SOA and how they are related.
  


• Microsoft released Silverlight 1.0 and is also going to built a version for Linux, together with Novell. That version, based on Mono, will be named Moonlight. Some more reports of this news here and here. See my post from May for a quick overview of Silverlight vs AIR vs JavaFX vs Flash/Flex.
  


• 6 interesting questions you should address when considering SOA. Questions range from security to ROI.