Friday, November 12, 2010

My experience with Android Market Licensing and LVL

This post describes my lessons learned with the Android Market Licensing service and the accompanying License Verification Library (LVL). I used it the first time for a completely new application, never published on the market before.


Getting started
From the description and instructions page you can read:
"Android Market offers a licensing service that lets you enforce licensing policies for paid applications that you publish through Android Market. With Android Market Licensing, your applications can query Android Market at run time to obtain their licensing status for the current user, then allow or disallow further use as appropriate".
"The License Verification Library (LVL) handles all of the licensing-related communication with the Android Market client and the licensing service. With the LVL integrated, your application can determine its licensing status for the current user by simply calling a library checker method and implementing a callback that receives the status".


That page is quite elaborate and definitely a good starting point for integrating the LVL into your project. Note that your app needs to support SDK 1.5 or higher (API level 3 and onwards).

The LVL I did not set up as a Library Project in Eclipse because it messed up my Subversion. It got totally confused, indicating stuff wasn't checked in, but I also could not check anything in even after a cleanup. Or it kept indicating there were waiting changes to check in, even though I had it checked in. Maybe it's Eclipse Ganymede, maybe it's me :). So I just imported the source code in my project. Disadvantage is I'll have to update each project that uses the LVL when an update of the LVL arrives.

Furthermore I used the ServerManagedPolicy and changed the salt for the AESObfuscator (which is used to securely obfuscate data written to and read from storage).
Note that the AESObfuscator has been reported to be quite slow.

Setting up a test response for license checks works fine for brand spanking new apps like mine was. On my emulator I used my developer account, on the real device (a Nexus One) another gmail account as test-account.

Quite essential in the documentation is the line "Signing in using a publisher account offers the advantage of letting your applications receive static test responses even before the applications are uploaded to the publisher site".
It took me (and apparently others) quite a while to figure out that that means that the test accounts only get the Test response when you've uploaded the app to the market and saved it! Otherwise they will get ERROR_NOT_MARKET_MANAGED.


To obfuscate or not to obfuscate
I decided not to obfuscate my application. Main reasons being:

  • It won't prevent hackers from decompiling your code for 100% sure.

  • It gets a real pain in the @ss to figure out the real line of the code when you get an error report or exception.

  • You should do a full regression test after obfuscation. Of course :) you have that all automated but still...

Note also that the post explaining ProGuard obfuscation requires a fix (it obfuscates too much). It might be updated now.
Here's a tip when you're using Proguard obfuscation to easily indicate which classes shouldn't be obfuscated (in short: by letting those implement a specificly named interface and put that as a keep-rule in the proguard config).

Great tip here to even more minimize the number of classes/interfaces you shouldn't obfuscate.

For another nice introduction and more in-depth details on why you could use Proguard for obfuscation, shrinking and optimization, see this post.


Server Response Extras
During testing on the emulator I found out that the Server Response Extras are never set for the test responses, so it's quite hard to test different behaviour here on different values. On the other hand, the default implementation seems to handle it quite ok. If you don't like it, you'll have to dive into the LVL code...


Testing new versions of an app
I also found out that if you increase the versionCode of your app, and try to test that app on a real phone with a test account, you'll get back ERROR_NOT_MARKET_MANAGED. Most likely this is because of the new versionCode, which the market licensing service does not recognize/accept. I have not seen the ability to just upload a new version of an app and only save it, it seems you can only publish it... And this thus means your test accounts can't test the app before release...

Looking at a reply in this post it seems you don't even want to save a newer version because the current version of your app disappears from the market! Another report of this here.

As mentioned in the first post, the only way to test a newer version seems to set the versionCode to the currently published version and use that during testing on a real device with test accounts. Sounds doable, there's not too much risk of forgetting to increase the versionCode when publishing: the market won't let you upload an app with the same versionCode as already published.

Note: Maybe the new "draft upload" feature fixes the above issues (see sixth item in this post), didn't try that out yet.

Some answers from Trevor Johns in the post clarify things re: responses a bit but not all:

"If an app is not published AND not draft, then you'll get
ERROR_NOT_MARKET_MANAGED."

Yes, but if your testing with your developer account on the emulator, you get back whatever is set in the test response. Even for an app with a newer versionCode.

"If an app is in draft (never published), then we send LICENSED for all requests for that app."
Is it? It seems it sends back to test accounts on a real device what's set in the Test response. Not tested yet for increased versionCode.
The reply on 6aug10 from Trevor Johns in this thread says it too. Unless I did something wrong, I noticed different behaviour.

"If an app is published (or has been published then unpublished), then
the response is driven by the dev console settings for the developer/testers, and purely by purchase history for everyone else."

Yes, but if you have increased the versionCode, test accounts on a real phone get back ERROR_NOT_MARKET_MANAGED.

See also other responses in this post.

I also wanted to test what happens when I publish a new version, would people with an older purchased version of the app still get verified successfully, and what does the provided LVL code do when it gets back LICENSED_OLD_KEY?
My conclusion: that combination seems only testable on the emulator for the developer account before publishing the app (note I didn't try the scenario on a real device with the developer account). After really publishing the app you can test it of course, but that's a bit late, you want to try it before publishing, before the world can download it.
For an app never published before, you can test the LICENSED_OLD_KEY as described here.


More securing your implementation
Of course I followed also most steps described in Securing Android LVL Applications. I did not add a CRC check. Tips on how to do that can also be found here and here.


Wishes
What would be great to have available in the developer console is to see how many verification requests failed, and with what error. This way you can get an indication on how many people try to illegally use your app, and get an indication whether something might be competely wrong with your app (e.g nobody gets correctly verified anymore).
You could then also see how many people haven't upgraded yet via distinct the OLD_LICENSE_KEY responses.
As long as this kind of statistics is not provided by the market, but you still want to know about these kind of failures, what you could do is report any error results to your own separate server (though of course a skilled hacker could prevent that from being sent).

Finally: don't forget to add any issues you find to the market licensing issues page.

Saturday, October 16, 2010

Saving precious Android Market description text space

Pretty annoying that you only have 325 characters in the description field when submitting an app right? You really have to use the space wisely.

One of the things you usually want to put in what you have added/changed in the current release, so people who want to update know what to expect.
That takes up precious space of those 325 characters. But from now on you don't need to worry about that anymore because... it turns out you can put any text in the android:versionName attribute in the AndroidManifest.xml!

So what I usually do is put in versionName the digits from the android:versionCode attribute separated by dots, then a space and then a short description of what has changed since the last version. For example, suppose android:versionCode="103" then androidName becomes:

android:versionName="1.0.3 Preferences are now correctly stored"

So what does it look like? Well here's first a screenshot of an app that only puts in a version number in the android:versionName attribute:


And here's an example of an app that fully uses that attribute:


I do not know what the maximum lenght is of the versionName field, but apparently you can put quite some text in there, as can be seen from above screenshot.

Update: well, putting the version information in the versionName is not necessary anymore, since from now on there's a separate field to put the recent changes into. What now quickly comes to mind to put into the versionName could be the minimal SDK version number your app supports. That can be useful for people who want/need to get your app on non-Google markets. This way they get at least an indication whether the app can run on their phone or not.

Update: and recently (definitely since 23 Feb 2011) they have limited the length of the versionName field to 30 characters, because if you exceed that, at apk-upload time you'll see: "The versionName in AndroidManifest.xml exceeds maximum (30)."

Sunday, October 3, 2010

JBoss Application Server 5.1/ESB 4.5 GA and ActiveMQ 5.4 integration

On one of my last projects I had to integrate JBoss ESB 4.5 GA with ActiveMQ 5.4.0 and Spring 3. NOTE: the fixes below also work for integration with JBoss 5.1 Application Server.



My starting point was of course this post: Integrating Apache ActiveMQ with JBoss. (Or is this the original?)
Part of the requirements mentioned in that article (Apache ActiveMQ 4.0.1+ and JBoss 4.0.4+) got me worried that the steps described might not work for my version of JBoss and ActiveMQ. Even though there's a '+' after the version numbers :)

First I had quite a hard time finding the .rar file! Finally found it in \lib\optional in the ActiveMQ zip: activemq-rar-5.4.0.rar

After that I followed the steps including up to 'Configuring JBoss'. In there you also have to start JBoss again.
I didn't see any exceptions fly around so thought all was fine, so I started the consumer ('ant consumer').
But there I got:



[java] javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616. Reason: java.net.ConnectException:
Connection re[Thread-2] Caught: javax.jms.JMSException: Could not
connect to broker URL: tcp://localhost:61616. Reason:
java.net Connection Exception : connection refused


Strrrange, because I thought the server started fine. After quite some searching, I figured out the error message probably indicates that there's just nothing listening at localhost:61616.
After carefully checking the startup log (searching for 'activemq') I found out it hadn't started!



17:17:31,566 INFO [XBeanXmlBeanDefinitionReader] Loading XML bean
definitions from class path resource [broker-config.xml]
17:17:31,768 WARN [ActiveMQResourceAdapter] Could not start up embeded
ActiveMQ Broker 'xbean:broker-config.xml': Line 29 in XML document from
class path resource [broker-config.xml] is invalid; nested exception is
org.xml.sax.SAXParseException: cvc-elt.1: Cannot find the declaration
of element 'beans'.


That's weird, some namespace problem?
Searching on the biggg internet I found this page (via), explaining that the namespaces are different from 5.1 onwards.

Since I'm using AMQ 5.4.0 I tried in broker-config.xml:



<beans xmlns="http://activemq.apache.org/schema/core">


But that still gave:



17:23:38,638 WARN [ActiveMQResourceAdapter] Could not start up embeded
ActiveMQ Broker 'xbean:broker-config.xml': Line 29 in XML document from
class path resource [broker-config.xml] is invalid; nested exception is
org.xml.sax.SAXParseException: cvc-elt.1: Cannot find the declaration
of element 'beans'.


So now I just tried all the namespace definitions:



<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core-5.2.0.xsd
">
</beans>


Darn, still not ok:



17:27:42,265 WARN [ActiveMQResourceAdapter] Could not start up embeded
ActiveMQ Broker 'xbean:broker-config.xml': Line 49 in XML document from
class path resource [broker-config.xml] is invalid; nested exception is
org.xml.sax.SAXParseException: cvc-complex-type.2.4.c: The matching wildcard
is strict, but no declaration can be found for element 'persistenceAdapter'.


But that was easy, missing the namespace prefix. So I modified:



<amq:persistenceAdapter>
<amq:journaledJDBC journalLogFiles="5" dataDirectory="activemq-data"/>
<!-- To use a different datasource, use th following syntax : -->
<!--
<journaledJDBC journalLogFiles="5" dataDirectory="../data" dataSource="#postgres-ds"/>
-->
</amq:persistenceAdapter>


But again an error:



17:29:50,840 WARN [ActiveMQResourceAdapter] Could not start up embeded
ActiveMQ Broker 'xbean:broker-config.xml': Line 50 in XML document from
class path resource [broker-config.xml] is invalid; nested exception is
org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was
found starting with element 'amq:journaledJDBC'. One of
'{"http://activemq.apache.org/schema/core":amqPersistenceAdapter,
"http://activemq.apache.org/schema/core":jdbcPersistenceAdapter,
"http://activemq.apache.org/schema/core":journalPersistenceAdapter,
"http://activemq.apache.org/schema/core":kahaDB,
"http://activemq.apache.org/schema/core":kahaPersistenceAdapter,
"http://activemq.apache.org/schema/core":memoryPersistenceAdapter,
WC[##other:"http://activemq.apache.org/schema/core"]}' is expected.


So journaledJDBC is unknown. There must be something with a wrong namespace version or something. From the XSDs from here it seemed there was an element in the wrong place. So I just removed the whole persistenceAdapter element.

I also noted that there is a persistenceFactory element in the AMQ 450 broker-config.xml, which is not present in the Integrating Apache ActiveMQ with JBoss post.
So I also changed that one's dataDirectory to be the same as in the (just removed) persistenceAdapter:



<amq:persistenceFactory>
<amq:journalPersistenceAdapterFactory journalLogFiles="5" dataDirectory="${jboss.server.data.dir}/activemq"/>
</amq:persistenceFactory>


Yes that did it! See the log part below:



17:45:01,215 INFO [XBeanXmlBeanDefinitionReader] Loading XML bean definitions
from classpath resource [broker-config.xml]
17:45:01,542 INFO [DefaultListableBeanFactory] Pre-instantiating singletons
in org.springframework.beans.factory.support.DefaultListableBeanFactory@157402b:
defining beans [org.apache.activemq.xbean.XBeanBrokerService#0]; root of
factory hierarchy
17:45:01,697 INFO [PListStore] PListStore:activemq-data\bruce.broker1\tmp_storage
started

... stuff deleted ...

17:45:02,896 INFO [BrokerService] ActiveMQ 5.4.0 JMS Message Broker
(bruce.broker1) is starting
17:45:02,911 INFO [BrokerService] For help or more information please
see: http://activemq.apache.org/
17:45:03,083 INFO [SchedulerBroker] Scheduler using directory:
activemq-data\scheduler
17:45:03,145 INFO [JournalPersistenceAdapter] Journal Recovery Started from:
Active Journal: using 5 x 20.0 Megs at: C:\jbossesb-server-4.5.GA\bin\${jboss.server.data.dir}\activemq\journal
17:45:03,176 INFO [JournalPersistenceAdapter] Journal Recovered: 0 message(s)
in transactions recovered.
17:45:03,207 INFO [TransportServerThreadSupport] Listening for connections at:
tcp://localhost:61616
17:45:03,223 INFO [TransportConnector] Connector bruce.broker1 Started
17:45:03,223 INFO [BrokerService] ActiveMQ JMS Message Broker (bruce.broker1,
ID:PC555-4930-1389432149-0:0) started


Also after that the consumer and producer worked fine.

And as a final optimization to not having to change the namespace for each new ActiveMQ version, I removed the version from the activemq-core xsd:



<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
">


To summarize, this is the final broker-config.xml (some comments deleted for space):



<?xml version="1.0" encoding="UTF-8"?>
<!-- START SNIPPET: xbean -->
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
">

<!-- shutdown hook is disabled as RAR classloader may be gone at shutdown -->
<amq:broker useJmx="true" useShutdownHook="false" brokerName="bruce.broker1">

<amq:managementContext>
<!-- use appserver provided context instead of creating one,
for jboss use: -Djboss.platform.mbeanserver -->
<amq:managementContext createConnector="false"/>
</amq:managementContext>

<amq:persistenceFactory>
<amq:journalPersistenceAdapterFactory journalLogFiles="5" dataDirectory="${jboss.server.data.dir}/activemq"/>
</amq:persistenceFactory>

<amq:transportConnectors>
<amq:transportConnector name="bruce.broker1" uri="tcp://localhost:61616" discoveryUri="multicast://default"/>
</amq:transportConnectors>

</amq:broker>
</beans>


JBoss 5.1 AS specific stuff:

I just copied over the above created .rar dir to the jboss-5.1.0.GA/server/default/deploy/activemq-ra.rar/META-INF directory and started it.

Got one error trying:



18:00:48,611 ERROR [AbstractKernelController] Error installing to Parse:
name=vfsfile:/C:/jboss-5.1.0.GA/server/default/deploy/activemq-ra.rar/
state=Not Installed mode=Manual requiredState=Parse
org.jboss.deployers.spi.DeploymentException: Error creating managed object
for vfsfile:/C:/jboss-5.1.0.GA/server/default/deploy/activemq-ra.rar/
at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(
DeploymentException.java:49)
...
Caused by: org.jboss.xb.binding.JBossXBException: Failed to parse source:
cvc-complex-type.2.4.d: Invalid content was found starting with element
'config-property-value'. No child element is expected at this point. @
vfsfile:/C:/jboss-5.1.0.GA/server/default/deploy/
activemq-ra.rar/META-INF/ra.xml[100,36]


Ah for some reason I had an empty in ra.xml. Removing that made the 5.1 server start fine and the producer produce and the consumer consume!

PS: the example consumer & producer now send out 2000 messages instead of the 10 you see in the Integrating Apache ActiveMQ with JBoss post.

Hope this might help somebody some time :)

Changing format of blog

This blogpost is the last of the 'Best of this Week Summary' series.
Looking at the number of subscribers, I decided it's too low to be worth the effort.

Therefore, from now on I'll only post "real" blogposts written by myself, no more weekly summaries.
Those posts will contain lessons learned, tutorials etc.
This change of format also implies that the posts will be less frequent but hopefully much more interesting and relevant to a greater audience.

For the fans, here's the last set of items for "Best of this Week Summary", 27 September - 3 October 2010:


PS: I'll change my mind if I get at least 50 (valid, unique) requests in the comments to keep posting my weekly summaries :)

Sunday, September 26, 2010

Best of this Week Summary 14 September - 26 September 2010

Sunday, September 19, 2010

Best of this Week Summary 13 September - 19 September 2010

  • Interesting (sort of inside) view on how Google controls what's shipped with Android on mobile phones, how smart their usage of open source is, and why Android probably got such a big momentum quickly.
    The controls are: private branches, closed review process, speed of evolution of the platform versions, incomplete public source code, gated development community, anti-fragmentation agreement, private roadmap and Android trademark.

  • Good list of lessons learned when building social sites.

  • Atomiq could be handy tool to check for duplicate code in Java, C#, VB.Net, ASPX, Ruby, Python, C, C++, ActionScript and XAML. A related similar tool is the Copy Paste Detector in PMD.


Sunday, September 12, 2010

Best of this Week Summary 06 September - 12 September 2010

  • Small tuturial for sending emails with Spring Framework, which provides some abstraction from the underlying mailing system.

  • Nice: "HTML5 Boilerplate is the professional badass's base HTML/CSS/JS template for a fast, robust and future-proof site.
    After more than two years in iterative development, you get the best of the best practices baked in: cross-browser normalization, performance optimizations, even optional features like cross-domain ajax and flash. A starter apache .htaccess config file hooks you the eff up with caching rules and preps your site to serve HTML5 video, use @font-face, and get your gzip zipple on.
    Boilerplate is not a framework, nor does it prescribe any philosophy of development, it's just got some tricks to get your project off the ground quickly and right-footed."

  • Different existing logging strategies described and a new one: grammar based event logging.

  • Be cautious to not over-engineer your software solutions...


  • Are Spock, Geb and WebDriver the future of functional web testing?

Sunday, September 5, 2010

Best of this Week Summary 30 August - 05 September 2010

  • A "series of (currently 5) articles introduces Contexts and Dependency Injection for Java EE (CDI), a key part of the Java EE 6 platform. Standardized via JSR 299, CDI is the de-facto API for comprehensive next-generation type-safe dependency injection as well as robust context management for Java EE. Led by Gavin King, JSR 299 aims to synthesize the best-of-breed features from solutions like Seam, Guice and Spring while adding many useful innovations of its own".

  • An extensive tutorial on creating a Spring MVC based mobile web application using MyEclipse. The IDE has already an iPhone skinned version built in, this tutorial shows how to add other mobile handsets like Android and Palm Pre. Basically: providing another style :)


  • How to use Apache ActiveMQ in combination with Spring.

  • Why ACID is hard to scale. and an argument that NoSQL/NoACID is the lazy way around these difficulties. Here's a post considering the performance argument for switching to a NoSQL database.

  • Great interesting series on the progress of converting the native code BBC News iPhone/iPad app to a web app using HTML5.

Sunday, August 29, 2010

Best of this Week Summary 23 August - 29 August 2010

Wednesday, August 25, 2010

Endless flashing TextView animation in Android

A little while ago I needed a flashing text in Android.


Pretty easy you'd think: stick two animations in an AnimationSet, set the repeatMode to Animation.RESTART and repeatCount to Animation.INFITITE like this:

XML


<set xmlns:android="http://schemas.android.com/apk/res/android"
<alpha
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:fromAlpha="0.0"
    android:toAlpha="1.0"
    android:duration="1000"
/>
<alpha
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:fromAlpha="0.9"
    android:toAlpha="0.0"
    android:duration="1000"
    android:startOffset="1000"
/>
</set>



Activity code

AnimationSet c = (AnimationSet) AnimationUtils.loadAnimation(this, R.anim.flash);
c.setRepeatMode(Animation.RESTART);
c.setRepeatCount(Animation.INFINITE);
TextView tv = (TextView) findViewById(R.id.flashingTextView);
tv.clearAnimation();
tv.startAnimation(c);



and you're done.
But no! It only runs one time, or 1.5 times or whatever; definitely not endlessly.

This pretty basic requirement of looping a set of animations indefinitely is apparently a known bug. I haven't been able to find out if it's still an outstanding issue today (August 2010) but unless I implemented it incorrectly, it is definitely present in SDK 1.6.
Based on the answers in that thread and the example code here, I came up with the following solution:

XML
A fadein.xml and a fadeout.xml. See the end of this post for the complete code.

Activity code


    // Get ref to what we're going to animate (fade-in and fade-out indefinitely)
    tv = (TextView) findViewById(R.id.flashingTextView);

    // Setup fadein/out animations
   fadeIn = AnimationUtils.loadAnimation(this, R.anim.fadein);
   fadeIn.setAnimationListener( myFadeInAnimationListener );
   fadeOut = AnimationUtils.loadAnimation(this, R.anim.fadeout);
   fadeOut.setAnimationListener( myFadeOutAnimationListener );
   
   // And start with the fade in
   launchInAnimation();

    public void launchInAnimation() {
       tv.startAnimation(fadeIn);
    }    

    public void launchOutAnimation() {
       // Launch the second animation :
       tv = (TextView) findViewById(R.id.flashingTextView);
       tv.startAnimation(fadeOut);
    }    
    



Two listeners which start the other animation at the end of an animation:

    private class LocalFadeInAnimationListener implements AnimationListener {
       public void onAnimationEnd(Animation animation) {
           tv.post(mLaunchFadeOutAnimation);
        }
       public void onAnimationRepeat(Animation animation) {
       }
       public void onAnimationStart(Animation animation) {
       }
    };
    
    private class LocalFadeOutAnimationListener implements AnimationListener {
       public void onAnimationEnd(Animation animation) {
           tv.post(mLaunchFadeInAnimation);
      }
       public void onAnimationRepeat(Animation animation) {
      }
      public void onAnimationStart(Animation animation) {
      }
    };
    
    private LocalFadeInAnimationListener myFadeInAnimationListener = new LocalFadeInAnimationListener();
    private LocalFadeOutAnimationListener myFadeOutAnimationListener = new LocalFadeOutAnimationListener();




Runnables that do the fading

    private Runnable mLaunchFadeOutAnimation = new Runnable() {
       public void run() {
            launchOutAnimation();
       }
    };    
    
    private Runnable mLaunchFadeInAnimation = new Runnable() {
       public void run() {
        launchInAnimation();
       }
    };    



Note that I improved efficiency of the mentioned example code by only setting the animations one time and getting the TextView to animate only once.

That's it!

Note: in the logcat I do get these messages (2-3 times in about 5 minutes):


08-22 15:33:28.078: WARN/SurfaceComposerClient(252): lock_layer timed out (is the CPU pegged?) layer=0, lcblk=0x41048020, 
state=00000002 (was 00000043)
08-22 15:33:28.078: WARN/SurfaceComposerClient(252): lock_layer() timed out but didn't appear to need to be locked and 
we recovered (layer=0, lcblk=0x41048020, state=00000002)



Pretty strange, the CPU doesn't seem to be pegged... It says it recovers, but I guess the warning is there for a reason. Can the code be modified to avoid these warnings?
It might be a platform bug. Indeed I do see it appear in 1.6 but not in 2.1.

Full code (tested on 1.6) can be downloaded here.

Sunday, August 22, 2010

Best of this Week Summary 16 August - 22 August 2010

  • Great performance analysis (PDF) between Java IO and Java NIO by Google engineer Paul Tyma. Java IO is performing about 25% better.

  • Of course I have to mention Oracle's Java lawsuit against Google. Several viewpoints can be found here (more on the profit/free markets) and here (more elaborate and technical details) and here (Android == Java?).

  • A short comparison between Spring Web Services and Apache CXF.

  • Several short HTML5 tutorials by Bob Leah from IBM collected at one place.

  • Five things you might not know about .jar files.


Sunday, August 15, 2010

Best of this Week Summary 9 August - 15 August 2010

Sunday, August 8, 2010

Best of this Week Summary 2 August - 8 August 2010

Sunday, August 1, 2010

Best of this Week Summary 26 July - 1 August 2010

  • Pretty nasty, some application apparently depend on the the company name in the JDK! Including Eclipse, causing OutOfMemoryErrors since Oracle changed the name from 'Sun Microsystems, Inc' to 'Oracle'. Sounds like a magic constant somewhere... Though as it says here it had to be put in to work around another bug. Oracle rolled it back until JDK 7.

  • Evolutionary architecture and emergent design: Leveraging reusable code, Part 1 and Part 2.

  • A Spring MVC 3 Showcase, which should give you have a good idea of what the technology can do.It includes a sample project, along with a supporting slide presentation and screencast. After digging in, you should have a good understanding of what Spring MVC can do and get a feel for how easy it is to use.

  • Some random Android tips/lessons learned.

  • Java Iterator Quiz time!

Sunday, July 25, 2010

Best of this Week Summary 19 July - 25 July 2010

Sunday, July 18, 2010

Best of this Week Summary 28 June - 18 July 2010

Wednesday, July 7, 2010

Android performance tips

All videos and slides from sessions during Google's I/O conference last May 2010 are available here.

I watched The world of ListView and Writing zippy Android apps. Below are the items that were most interesting to me. They are all Android performance related.

The world of ListView

  • 9:15: A View in Android costs about one or 2K of RAM

  • 16:55: Call notifyDataSetChanged() (from the UI thread) when something in your adapter has changed.

  • 18:27: getItemViewType() is used to make sure convertView is of the correct type in getView().

  • 19:06: Make sure getViewTypeCount never changes. Note that it doesn't hurt performance if you always return say 10 even if you only have 2 different view types returned by getItemViewType().

  • 30:13: Shows how to create a custom list selector in xml.

  • 40:40: android:smoothScrollbar to prevent scrollbar changing size. Especially useful for listviews with items of which height can differ greatly

  • 41:00: Use fill_parent instead of wrap_content in a ListView

  • 53:10: If refreshing the whole screen via notifyDataSetChanged() is too much of a performance hit, you can try to get the visible position and use getChildAt() to figure out what view to update. Example code for that is here.

  • 53:50: Use inflate(item, parent, false) to prevent relative layout attributes being ignored in a listview.

In general: don't try to outsmart the ListView by building your own caching, nor depend in any way on the order of getView(position) getting called.


Writing zippy Android apps

Zippy means non-janky which means sluggish, slow etc.
  • 08:00: example apk of how not to do things/see how stuff performs. Includes small Perl sqllite wrapper script to see how much is being read from sqlite

  • 15:00: read/writes I/O on emulator is a lot faster than on a real device

  • 19:00: An AsyncTask can get killed before it finishes, e.g if the user hits the Home key. If it's important that the task finishes, use IntentService.

  • 24:15: Profiling/tracing apps 'adb shell am profile ...' with the TraceView tool.

  • 47:00: handling rotation when you don't want to reload your heavy objects: usegetLastNonConfigurationInstance() and onRetainNonConfigurationInstance(). See the Android reference for more info.

  • 51:00: Think about using files instead of sqlite (if you have limited structured data for example)

Sunday, July 4, 2010

Lessons learned Wicket + Spring + Hibernate + Mod4J project

At a recent project we used the following tools & frameworks:


Below are a couple of lessons learned which I remembered to write down:

Wicket
  • In a ModalWindow you'd probably want to use a AjaxSubmitLink, not a SubmitLink, if you want to use modalWindow.setWindowClosedCallback(). See here for explanation.

  • ajaxrequesttarget.addComponent: addComponent name might be a bit confusing for beginners. It means "add the component to the list of components be re-rendered/refreshed".

  • An AjaxSubmitLink doesn't update the model when setDefaultFormProcessing() is set to false. Not totally illogical, but you still might run it to it when you don't expect it.

  • Here are tips to validate related fields. (In the original the example code is missing.)

  • AjaxSubmitLink: if you get a "Component-targetted feedback message was left unrendered. This could be because you are missing a FeedbackPanel on the page" warning in your Tomcat server console, it seems you have to tell in the onError() of the AjaxSubmitLink which feedback panel(s) you want to have updated.
    You get the warning even when you have feedback panels higher up in the tree of the (Base)Page. An example to update those panels could be:

    Component infoFeedback = getPage().get("infoPanel");
    target.addComponent(infoFeedback);
    Component warnFeedback = getPage().get("warningPanel");
    target.addComponent(warnFeedback);
    Component errorFeedback = getPage().get("errorPanel");
    target.addComponent(errorFeedback);

    This solution was inspired by these posts: post1, post2, post3, post4.

  • To show/hide any HTML markup block dynamically, just wrap it with a WebMarkupContainer. In the code create that wrapper and make it visible or not depending on your requirements:

    boolean makeVisible = false;
    WebMarkupContainer blockContainer = new WebMarkupContainer("blockWrapper");
    blockContainer.setVisible(makeVisible);
    add(blockContainer);


  • Sometimes you might get a popup when using a ModalWinow that says "Reloading this page will cause modal window to disappear" when you don't expect it. Check the logs/console, you might just got an exception in your app (after which Wicket tries to redirect to the error page, which causes the popup to show; at least that's my reasoning).

JUnit
  • If JUnit can't find the Spring context.xml in the resources directory, then you have to add the resources dir to your Build path

Mod4J
  • Does not really support LazyLoading so not very efficient for large "graphs" of data/dependencies. In those cases you could decide to skip the DTO layer and directly access the domain model.

  • The Maven plugin IAM in Eclipse can be turned off for the Mod4J models project in Eclipse, otherwise it runs twice: once by Mod4J features, once by Maven (plugin).

Sunday, June 27, 2010

Best of this Week Summary 15 June - 27 June 2010

Sunday, June 20, 2010

Best of this Week Summary 14 June - 20 June 2010

Sunday, June 13, 2010

Best of this Week Summary 7 June - 13 June 2010

  • When rewriting an app from scratch doesn't make sense, and when it might make sense.

  • Java deadlocks through Cyclic Dependencies: title says it all.

  • Carnegie Mellon University's CERT (Computer Emergency Response Team) has released the Basic Fuzzing Framework (BFF), which is a simplified version of automated dumb fuzzing and includes a Linux virtual machine that has been optimized for fuzz testing and a set of scripts to implement a software test. It helps identify and eliminate security vulnerabilities from software products.

  • A nice (external) analysis of the FIFA World Cup website against performance best practices.

  • "Nokia's set of learning and teaching materials on Java in Mobile Devices (JME/MIDP). In fact, it is Forum Nokia’s most comprehensive set of information ever. Included are five courses, 23 lectures, 25 lab exercises with source code, and more than 1,200 slides with notes. The material has been designed for university instructors and students, developers, and self-learners. Five main topics are covered:

    • Java ME basics

    • Multimedia

    • Networking and messaging

    • Games and graphics

    • Security

    Of course JME is not so sexy as Android these days, but Nokia still has the largest number of mobile phones out there. And most (even quite low end) support JME, so from a market perspective the potential is much higher than building apps for Android phones...

  • Several CSS reset examples, including a few really short ones.

Sunday, June 6, 2010

Best of this Week Summary 30 May - 6 June 2010

  • Google Web Toolkit vs. Smart GWT: Which should you choose as front-end? Start immediately with Smart GWT, or start with GWT and pick SGWT components when needed? Some insights here.
    Related to that, you might want to check Vaadin: "Vaadin is an open source web application framework for rich Internet applications. In contrast to Javascript libraries and browser-plugin based solutions it features a server-side architecture, which means that the majority of the logic runs on the servers. Ajax technology is used at the browser-side to ensure a rich and interactive user experience. On client-side Vaadin is built on top of and can be extended with Google Web Toolkit. Vaadin utilizes Google Web Toolkit for rendering the resulting web page. While Google Web Toolkit operates only client-side (i.e. a browser's JavaScript engine) – which could lead totrust issues – Vaadin adds server-side validation to all actions. This means that if the client data is tampered with, the server notices this and doesn't allow it.
    Historically, Vaadin has been compared to Echo and ZK frameworks that use similar of server-side programming model. The server-side APIs are quite similar providing both events and GUI components, but the client-side (i.e. web browser) interaction differs in the way that Vaadin uses Java programmed GWT widgets, while ZK is jQuery based, and Echo has its own implementation. Currently, the most frequently compared frameworks include Adobe Flex, Google Web Toolkit, Apache Wicket and ICEfaces."

  • A new open spec collaboration has started: OExchange, which is an open protocol for sharing any URL with any service on the web. Bigger parties involved are LinkedIn, Microsoft, Google.

  • A new way of phishing: TabNabbing - phishing by switching background tab content. Discovered by Firefox's creative lead Aza Raskin. Biggest challenge seems to get the malicious Javascript on a site the user goes to.

  • Google announced a partnership with VMWare (and thus SpringSource and thus Spring) at I/O by adding its (GTW) widgets to Spring and deployment to the VMWare cloud. In marketing speak: "This is VMware and Google's view of the power of using Spring along with Google's presentation widgets to get apps started in hours, delivered in days, and deployed in minutes". Below it's shown in a diagram:



  • The iPad isn't without "errors" in its usability (UI) according to Jakob Nielsen. For example: cross-app UI experience is inconsistent, and for some reason almost no app supports scrolling and shows information only per page.

  • "An overview of how to design websites and optimise them for Maemo, iPhone, Android, and a variety of touch and non-touch devices based on S60 on Symbian OS. After reading the document, you will have the basic knowledge you need to start developing mobile web pages that provide cross-browser-compatible content in a user-friendly manner. Furthermore, with the tips and advice contained in the document, you can avoid making design choices that could eventually lead to a dead end or poor design, thus saving time in implementing and debugging features that will not work".

  • Show Slow: an open source tool that helps monitor various website performance metrics over time. It captures the results of YSlow and Page Speed rankings and graphs them, to help you understand how various changes to your site affect its performance

Sunday, May 23, 2010

Best of this Week Summary 17 May - 23 May 2010

  • Codelab ("tutorial") from Google that "shows how web application vulnerabilities can be exploited and how to defend against these attacks. The best way to learn things is by doing, so you'll get a chance to do some real penetration testing, actually exploiting a real application. The codelab is built around Jarlsberg, a small, cheesy web application that allows its users to publish snippets of text and store assorted files. "Unfortunately," Jarlsberg has multiple security bugs ranging from cross-site scripting and cross-site request forgery, to information disclosure, denial of service, and remote code execution. The goal of this codelab is to guide you through discovering some of these bugs and learning ways to fix them both in Jarlsberg and in general."

  • A new IBM Redbook on WebSphere Messaging and High Availability has just been released.

  • A cool Android with Hudson continuous integration. Led on mobile phone changes color depending on the build status. Remember the Lava Lamps version?

  • Java dynamic proxies explained. They are for example used in Hibernate for lazy loading entities and in Spring for AOP. Also explains what CGLib proxies are for.

Sunday, May 16, 2010

Best of this Week Summary 10 May - 16 May 2010

Sunday, May 9, 2010

Best of this Week Summary 03 May - 09 May 2010

Tuesday, May 4, 2010

Clonezilla backup steps

In the last two years I had to get my harddisk replaced twice by a new one.
To minimize effort of setting up the new disk, I investigated which harddisk-copy program would best fit my needs, without complex setup things or complex instructions to create a bootable/live CD.

After trying several things, Clonezilla 1.2.2-31 did it for me.
I tried DriveImage XML, but that required me to make a streamlined CD for XP first, because I got a d:\i386\layout.inf not found error. I also looked at Paragon Backup & Recovery, but couldn't find how to create a live CD.

Creating a bootable live DVD was easiest with Clonezilla, and it's standalone, so no OS disks like Windows XP needed. The actual steps to make a backup a bit difficult for non-unix/linux people, but not impossible.
Below are the steps I now use to make a complete backup of my harddisk (including Windows) to a networked external disk via Samba.
Whether you can use these steps for your setup completely depends on your setup. Main reason for listing the steps here is so I have an easy reference to them :)

Prerequisites

  • Make sure you have an external harddisk connected to your network.

  • Make sure you have its IP address. If you're normally using DHCP, you might want to give it a fixed IP.

  • Make sure you have enough diskspace left on that external harddisk

  • To minimize network errors during the copying, you might want to use a network cable instead of a wireless connection. I definitely did that.

  • I used it for Windows XP, but any OS should do (it makes a byte-by-byte copy of the source disk


Steps to make copy
  1. Put the bootable CD/DVD in the drive to boot from it. Choose F8 or F12 at startup if you manually need to select your machine to boot from it.

  2. Wait until the boot sequence is done

  3. Choose language, and Don't touch keymap

  4. Choose Start Clonezilla

  5. Now you have to get the network card to get activated. That happens in the next steps

  6. Choose as Mount dir: samba_server

  7. Choose eth0. Choose DHCP

  8. Enter the IP of the networked external disk. E.g: 192.168.1.70

  9. I canceled the Domain configuration option

  10. Enter the account to use to connect to the networked external disk

  11. Pay attention when you enter the directory where the image must be stored. It is the directory in which the directory with the image information will be put. If you enter a non-existing directory, the mount (happens when you hit OK) will fail. Not a problem, just that you know. If the mount fails, just restart the process, this time with an existing path. Example path:


    /public/home/partimag


  12. Hit OK. This will start the mount command. It will ask you for a password if applicable. When the command is successful, you'll see the filesystem mount points listed.

  13. Choose Beginners Mode. Choose savedisk.

  14. Enter a name for the image. I usually append some more identifying information to the prefilled filename, which is something like: 2010-05-02-18-img

  15. Then choose the disk from the machine you want to make a backup for. For me it's only one: sda. Hit OK.

  16. Now you should see a summary of the command, which looks like:


    /opt/drbl/sbin/ocs-sr -q2 -c -j2 -z1 -i 2000 -p true savedisk "2010-05-02-18-img" "sda"


  17. Hit ENTER. That should start the backup. No errors should occur. My 500G harddisk with about 100G of data took about 4,5 hours to get backed up. One time I had an error in the output, saying something like the eth0 NIC was "gone". But the backup continued, and no other official errors were shown. It was not clear whether Clonezilla recovered successfully from the connection being away for a little while. Still, to be on the safe side, I made another backup, that had no network errors at all.



Steps to restore copy
These steps are about the same as making the copy, except that you select restoredisk instead of savedisk. But for completeness, here are the steps:

  1. Choose language

  2. Select Don't touch keymap

  3. Select Start Clonezilla

  4. Choose first option device_image

  5. Choose samba_server

  6. Choose eth0

  7. Choose DHCP

  8. Enter IP address of server you've stored the image on (e.g. 192.168.1.70)

  9. Cancel domain

  10. Enter account name to login on server

  11. Enter as directory: /public/home/partimag (note it's the same as when making the copy)

  12. Enter the password for the server. If correct, you'll see again the mount result (filesystem diskspace usage)

  13. Choose Beginner

  14. Choose Restore disk

  15. Choose the image you want to restore (e.g 2010-05-02-18-img)

  16. Choose target disk: sda

  17. Now you should see a summary of the command, which looks like:


    /opt/drbl/sbin/ocs-sr -g auto -e1 auto -e2 -c -r -j2 -p true restoredisk "2010-05-02-18-img" "sda"


  18. Hit ENTER. That should start the backup. No errors should occur. My 500G harddisk with about 100G of data took about 2,5 hours to restore.

Sunday, May 2, 2010

Best of this Week Summary 26 April - 02 May 2010

  • Google's Chief Java Architect Josh Bloch "discusses many of the problems facing the Java community, including the ineffectiveness of J2ME, licensing problems, Java 7's late ship date, and even the JCP issues that James Gosling often opined about. He points out that these problems predate the Oracle acquisition of Sun, so everyone should just stop pointing their fingers at Larry Ellison."

  • Another Getting Started blogpost for a Mahout-Taste based movie recommendation engine, which uses Wicket as presentation layer.

  • Martin Fowler explaining Inversion of Control and suggesting to call it the Dependency Injection pattern. Notes that the Service Locator pattern can also remove the dependency, just as the DI pattern tries to. Describes which of these options to use. Finally he also gives pros/cons for constructor vs setter injection. Also pros/cons for configuration in the code or via configuration files.

  • Several Javascript performance-optimization tips taken at JSConf 2010.

  • An article "which looks at various storage mechanisms - JDBC, JPA, JavaSpaces, Java Content Repository, MongoDB, and DB4O, primarily - from the perspective of how good they are at CRUD operations and queries". The conclusion is that there is no one "best" mechanism, each has its own best application.

Sunday, April 25, 2010

Best of this Week Summary 19 April - 25 April 2010

  • An introduction to Gizzard, an open sourced sharding framework (store data across multiple computers instead of on just one) which is used by Twitter.

  • A hands-on tutorial of creating a Spring application that uses Hibernate as JPA provider and JTA for transaction demarcation. A simple Order Processing Message Driven Bean is implemented that showcases this integration. It is deployed on a WebLogic 10.3 server.

  • Last week Jira from the Apache Foundation was compromised. Here's a description of how the hackers gained access via XSS.

Sunday, April 18, 2010

Best of this Week Summary 12 April - 18 April 2010

Sunday, April 11, 2010

Best of this Week Summary 5 April - 11 April 2010