My most recent problem/breakthrough was around enums. The particular app in question uses GXT in the browser, a set of native client apps on the users PC and an XML-RPC layer that glues them all together (yes, I still love XML-RPC, give me type safety over syntactical sugar any day of the week. Plus, we have libraries that make syntax a non-issue. But I diverge..). The client is written in C++/C# and does a lot of state tracking using, of course, enumerations, with a mirrored set of enums on the server-side.

These enums are natively persisted into the database using Cayenne’s Extended Enumeration feature making the entire transaction fairly seamless, until I got to the browser level and realized that GXT likes to treat everything as a Store and, beyond that, sending a native Cayenne class to the browser as a module was not going to work well since the Javascript Compiler would probably flip over and croak. After searching for awhile I finally came across a mix of recipes that works for me and thought I would commit my knowledge to Google and hope that I help others in the future.

Step 1: Normalize your Enum into your GWT *.client.* package space.

Yes, you must duplicate your enum OR move it to this space and change all accessors. Its a bit more complicated if your using a non-native enum (like the above mentioned Extended Enumeration for Cayenne). I ended up just making a copy of my Extended Enumerations as POE (Plain Ole Enumerations). Duplicative code AND prone to error when we change states but it worked.

Mine looks like this:

public enum BudgetPeriod {
     DAILY("Daily"),WEEKLY("Weekly"),MONTHLY("Monthly"),QUARTERLY("Quarterly"),YEARLY("Yearly");
     private String value;
     BudgetPeriod(String value) {
		this.value = value;
     }
     public String toString() {
		return value;
     }
}

Step 2: Use the SimpleComboBox

The combobox insists on a Store. At this point you can either convert your enum to ModelData and call it a day (which is what most people do) or realize that, since your using Dozer (well, I am), that this blows up fantastically and requires all sorts of conversion routines to move back and forth between ModelData and enum if Dozer is going to do its job right.

This is where the SimpleComboBox comes in, it allows you to bind any class to the combobox and thus gives you seamless abilities that glue your Enum directly to the form (and back again). It looks like this in my case:

SimpleComboBox<MyEnumClass> theComboBox = new SimpleComboBox<MyEnumClass>();

Now my box can move my enums around nicely in this combox format.

Step 3: Populate the list

Even a SimpleComboBox needs standards and its standard is either a Store or a List. We choose list and convert our Enum to that list for adding it to the list.

java.util.List enumList = Arrays.asList(MyEnumClass.values());
theComboBox.add(enumList);

Step 4: Pre-select whats in the database

theComboBox.setSimpleValue(entity.getEnum());

theComboBox.setTriggerAction(TriggerAction.ALL);

myModel.setBudgetperiod(theComboBox.getSimpleValue());

public enum BudgetPeriod {
     DAILY("Daily"),WEEKLY("Weekly"),MONTHLY("Monthly"),QUARTERLY("Quarterly"),YEARLY("Yearly");
     private String value;
     BudgetPeriod(String value) {
		this.value = value;
     }
     public String toString() {
		return value;
     }
}

theComboBox = new SimpleComboBox<BudgetPeriod>();
java.util.List periodList = Arrays.asList(MyEnumClass.values()); 
theComboBox.add(periodList);
theComboBox.setLazyRender(false);
theComboBox.setForceSelection(true);
theComboBox.setTriggerAction(TriggerAction.ALL);
theComboBox.setSimpleValue(project.getBudgetperiod());

createButton.addListener(Events.Select, new Listener() {
     public void handleEvent(ButtonEvent e) {
         grid.mask("Saving..");
         myModel.setBudgetperiod(projectBudgetTimeSpan.getSimpleValue()); // THIS IS ALL YOU CARE ABOUT IN THIS CODE CHUNK
         MyRPC.Util.getInstance().createProject(project, ClientSession.getInstance().getSessionID(),
                                                new ProjectCreatedCallback(grid, index, myModelBean));
         }
});

Thankfully there is, as with most things software related, a pattern! The pattern in this case is to use a Data Transfer Object (DTO). A DTO is basically a serializable bean that mirrors the persistent object. This is an annoying duplication of work and obviously violates the DRY (Don’t Repeat Yourself) principle since you are essentially cloning the Persistent object as a basic java bean. The upside of it is that you can control exactly what gets into your DTO (and thus what is transferred to the client). Originally I was constructing these beans as required, doing a pretty ad hoc job of it and generally causing myself headaches.

My first attempt at streamlining this was to use the BeanUtils apache commons project. This library has some simple methods for copying properties between beans which, in the end, is what the DTO is all about. This worked well for common and primitive types that were shared (name and type) between the two objects but it fell over pretty quickly if the types mismatched. Additionally, there is no easy way to exclude or customize an individual field level map and no way to do one-way mappings (there is never a reason to map the ID from the Bean to the Persistent Object!)

What are my requirements and why didn’t BeanUtils not working?

• BeanUtils is exceptions based: If it has a mismatched attribute it blows up spectacularly and you then have to treat those objects differently.
• BeanUtils doesn’t have an exceptions mechanism: In my situation where I have enumerations everywhere this meant that many objects of mine are not eligible for BeanUtils. (GWT doesn’t easily support enumerations so I have to convert them to int before sticking the int in the DTO with a map of values)
• BeanUtils doesn’t have an extensions mechanism: Once your object is out of the path of automation there is no way back, its a manual coding exercise.

There was a better way: Dozer

I had looked at Dozer a few times and it just seemed Scary and Large but in my desperation I started to dig around and discovered how addictingly easy it is to start with Dozer:

public static void setUser(UserBean bean, Pguser user){
     Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance();
     mapper.map(bean,user);
}

This is your base ‘BeanUtils replacement’ usage of Dozer. In my case this is part of a larger static class that is a big BeanFactory, eg: A bit file that has a bunch of static methods for converting beans to objects and objects to beans.

Fantastic! Now my conversions did not blow up in the general case where one object had a field that the other one lacked. However, it still exploded when types mismatched. Thats where the Dozer mapping comes in. Dozer allows you to create an XML file of exceptions that run as a part of the map(a,b) process. You don’t have to specific ALL fields in this mapping, just the differentials! Once I understood that it was happy-pants-time!

All you do is toss a file called ‘dozerBeanMapping.xml’ into your class path and Dozer automatically picks it up. No muss no fuss. Lets look at one of my beans in this file. In the below example I am basically excluded my enumerations from the mapping process. Note: you can do a LOT more then this, my goal here is to show how easy it is to use Dozer to manage your bean<->object exceptions.

<mapping>
   <class-a>com.pgi.common.Clientsettings</class-a>
   <class-b>com.pgi.webapp.client.dto.ClientsettingsBean</class-b>
   <field-exclude>
      <a>jobmode</a>
     <b>jobmode</b>
   </field-exclude>
   <field-exclude>
      <a>removemethod</a>
      <b>removemethod</b>
   </field-exclude>
   <field-exclude>
      <a>reportingmode</a>
      <b>reportingmode</b>
   </field-exclude>
</mapping>

But now how do I put my enumerations into my DTO? Two options really

For now I am putting this mapping in my factory method post auto-map

public static ClientsettingsBean getClientsettingsBean(Clientsettings settings){
   Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance();
   ClientsettingsBean theBean = mapper.map(settings,ClientsettingsBean.class);
   theBean.setJobmode(settings.getJobmode().ordinal());
   theBean.setReportingmode(settings.getReportingmode().ordinal());
   theBean.setRemovemethod(settings.getRemovemethod().ordinal());
   theBean.setId(DataObjectUtils.intPKForObject(settings));
   return theBean;
}

That said, since this is the Internet, I am going to put my $.02 in. So without further ado here is my Wave experience.

Scenario

Once a quarter or so my team rolls a release to our production servers. This is a very scripted event with ‘pilot checklist’ steps that we go through, one by one, confirming each along the way (Step 12: database dumped, confirmed. etc) . When we encounter any unknown problem we stop, analyze and decide whether to continue. Our ‘points of no return’ (or problematic return) are clearly noted and overall this approach lends itself well to our release process

Since we do our releases in the late evening and would rather not sit at the office all night the entire event is conducted via chat a group session. Traditionally this works well with some caveats.

• Gtalk randomly likes to bump people and people tend to hit the wrong button or close the window or have browser crashes. This leads to some tense moments in mid-roll while we invite them back etc.
• When things get fast and furious the interleaved conversations get really difficult to track
• The inevitable off-topic chatter gets in the way while we wait for things to complete
• Reassembling a record of events, even with chat logging, can be difficult since its not threaded making the post-roll analysis we do a bit more tiresome (‘was he saying ’shit’ because the database crashed or was this when his cat knocked over his beer?’)

So, in the interests of all of the above we decided to do a release on the Wave.

I fully acknowledge that I am a Wave Newbie and possibly every single point I raise is invalid. However, as the Wave Newbie who is accustomed to picking up Google tools and just Using Them I feel that my perspective is valuable.

Riding the Wave

First Impressions

Initial tests were confusing. Coming at it from a group chat paradigm, we did not really understand what we were supposed to do in this new world. Our confusion was quickly dispelled when someone uploaded a funny picture into a new thread. The game was on.

We then all very quickly realized how boring it is to watch people type. I mean seriously boring. In fact, I miss the suspense of ‘xyz is typing’. What is xyz going to say? How long will it take him to say it and when he is done will I care? Oh xyz, hurry hurry and type to me! Instead of ‘xyz is typing‘ we now were treated to a stream of consciousness. Random threads popping up empty and disappearing as they were deleted. Thoughts half baked and other thoughts that normally would not be expressed were exposed as the poster forgot that he has, essentially, lost the ‘draft’ mode that non-realtime chat provides (Note: This can be very embarrassing!). Finally, if I have to watch someone mistype and studiously correct the word ‘the’  again (‘teh‘ for anyone curious) I may go insane.

Once we got over our annoyance at our fellow developers typing skills we proceeded to treat it like chat. Every post was a new thread, nothing was connected and it just seemed like really annoying, standard issue chat. Finally, with some discipline imposed by one of my leads, we got herded into a threaded mode and started into our process.

Second Impressions

Once we got into the mode of thinking in threads the inevitable dispute broke out, what sort of threads should we use? Topical? Checklist related? Eventually it just evolved into a semi-chaotic system of topical threads that ran at different paces (See ‘The Good’ below). It was interesting to watch the team self-organize into little pockets of discussion and to be able to jump in and participate as required. In a strict group chat setting many of these probably happen in one-on-one chats outside the group and are both lost to the record and less collaborative then they could be. We are big on ‘roll night records’ so pulling these chats in was very valuable for us and avoided a few confusing situations.

This multitude of threads was taxing though. Thoughts seemed disconnected and hard to follow. The amount of information quickly became overwhelming and everyone was resorting to constantly scrolling up and down, hunting for updates and looking for information. This lack of ‘indexability’ really made keeping track of things difficult and time consuming. In more then one instance we had to revert to chat to let someone in the wave know that a thread had been updated. Compounding this problem was the fact that there is no Quick Search feature for just the wave in question. Google without search? Totally odd. Yes, you can create big mega-searches etc but in my case I just wanted a quick intra-wave search that let me navigate the content.

Eventually we hit a nice rhythm. Less threads created, more centralized discussion and the pace picked up. People were able to step in and out of the release process as required, picking up and getting immediate context to the task at hand. It was pretty fun in an odd sort of nerd way. Definitely a different experience for us and a different sort of chaos then we are used to. Not less or more, just different. It was less boring as well since you could have multiple discussions on screen, in real-time, at the same time. Much more ‘CNN news ticker while watching a debate’

The Good

Comment Editing: The ability to edit, with changes tracked, threads is fantastic. It allowed us to keep a ‘notes’ thread and a ’status’ thread that were continually updated. Not only that but you can keep the root node ‘current’ and then do threaded comments beneath. This was very handy in tracking our status and noting deviance from the script.

Addictive: You pretty quickly find yourself addicted to the amount of information you can consume at once. It was a bit thrilling to be watching one person troubleshooting a code problem, another guy running automated test scripts while yet another guy ran update tools to bring the database up to date. It was as close to a birds eye view as I have ever seen for our process.

Totality: We were able to capture everything that happened in a clear way. When I had to call my QA engineer to come online 30 minutes early it was a side-thread and did not bother anyone that did not care about that thread. More then just content and context is the temporal nature of it. Being able to replay and easily see the context of a comment in both time and space is invaluable. This 3 dimensional view of our software releases is fascinating.

Pacing: The threaded view allowed us to run different threads at different paces. Some were fast and furious as live problems were debugged while others were very slow as they tracked overall release status. In a group chat everything is Now and Urgent and (for us) the slow lead items are paper based to ensure completeness. Wave seems to have done away with that by allowing us to pace different conversations differently.

The Bad

Comment Editing: Confusing as all can be sometimes. The change tracking in that context can make your head spin and you never know if its authoritative, what has changed etc.

No Navigation: The lack of a table of contents, index or any other sort of thread navigation tool outside of the scrollbar was the most painful (‘tedious‘ as one engineer commented) part of the evening. Having to scroll up and down constantly, trolling for updates, sucked and made the tool very difficult to use. It almost forced us to use the tool much less collaboratively then we would have, reverting us back to group chat mode with minor side threads for the most part.

No Quick Search: Without any sort of index to the wave some sort of quick search is imperative. I fully expected to be able to type anything into a box and have a list of thread matches pop-up with anchor links to them. Nope. Really? No Search in a google App? Sigh. Yes, I may have missed the big green ‘Search The Wave’ button but I doubt it.

The Ugly

Scrolling: Its completely apparent to me now that everyone at the GooglePlex working on Wave has at least one 3280×2048 WQSXGA monitor. The only way these long waves make sense is if you make them Tall and Wide so as to reduce scrolling and the sheer amount of screen real estate they scream for is incredible. My pathetic little 15″ MacBook Pro was not up to the task of giving me a good view of the action.

Extremely Easy Thread Creation: I get it, it needs to be easy to use but does it need to be THAT easy to use? We spent a lot of time deleting empty threads when all someone was doing was trying to respond to a thread.

Extremely Difficult Thread Maintenance: 5 pixel click region to stay in a thread? This really made things disjointed for us as people failed to click in that 5 pixel boundary and Very Easily Created New Threads.

Inability to Reorganize Content: Once your cast yourself out of a thread because Its So Darn Easy To Create A New Thread  you must STAY OUT! Some drag/drop tools would be great here.

Take Aways and Ideas (haha, ideas? really? for google? sigh..)

The jury is still out but I think we like Google Wave and will use it again. With more experience under our belt it might actually make some sense and it did not truly detract once we wrapped our heads around it. With a bit of discipline I think our org will be successful Wavers soon enough.

So, in the spirit of a know it all, mostly anonymous and definitely uninformed blogger I will now proceed to tell Google what they should fix so that the Wave is easier to ride. Hold on to your pants because this anonymous and uninformed blogger is about to tell the uncountable Top Notch Product Managers and UX Developers who spent millions on design, research, usability analysis, market testing and prolonged beta-testing exactly what they need to do with the Wave!

Quick Search: As discussed above, I need to quickly search my wave and get a nice list of Google Like results. It would even be neat to do a narrowing search so that the threads that do not match your criteria disappear, one by one (think iTunes). That would be huge for navigability.

Index: Give me a thread index tree that I can collapse and expand and use as a quick nav for the wave. This would let me bounce around more easily. Bonus points: Let me give any node my own name so I can reference it more easily.

Bookmarks: Let me bookmark locations in the wave so I can anchor link back to them. Being able to click a ’star icon’ and name it ‘Status Report’ so I can quickly reference it without wearing out my 4 year old microsoft mouse would be fabulous.

Quick Links to Updates: The inbox nicely tells me ‘6′ if the wave has 6 new entries but it doesn’t tell me WHERE THOSE ARE. Once again my mouse-wheel suffers as I frantically scroll up and down scanning for the ‘new item’ indicator amongst 300+ posts.

Browser Tab Should Reflect Current Wave: There is a nice number in the browser tab. It usually reads ‘1′ and indicates that ‘1′ wave has updates. I was off doing other tasks expecting this to reflect the number of updates IN MY CURRENT WAVE (you know, like GMAIL and my INBOX) rather then the fact that my one and only wave was updated. I clicked back, wondering what was taking so long, and noticed that about 40 messages had scrolled by. Lets make this number reflect the number of updates in my open wave.

Hopefully the myriad of Google specialists who live and breath usability and probably possess PhD’s in ‘Waveology’  take my advice and make Wave a better tool for me. We’ll probably try it again with the knowledge gained and see if we can make it more sensible for us. My feeling is that the Wave is growing on the team and we’ll be doing a lot more with it soon.

In the spirit of duplicating information all over the internet I’ll paste my copy/paste implementation of the presented solution. It works well!

Step 1: Create the AjaxWizardButton class

Step 2: Create the buttonbar that will wrap these buttons and override the native buttonbar built into the Wizard.

Step 3: Alter your wizard html markup so you can override the native buttonbar

Step 4: Set your wizard markup output ID so that you can target it with Ajax calls

Step 5: Override the buttonbar call in your wizard class

Step 6: Bask in the glow of warm Ajax.

Pretty simple but the impact is fantastic. Using this code I was able to Ajaxify my wizard in well under 10 minutes.

Thanks Wicket!

‘Deployability‘.

WHAT? What is this “deployability” you speak of? Well, it is the key to a stable and reproducible system.

CPAN is great, mod_perl (and fastcgi) are , well, fast, but deploying the Perl stack is a nightmare of dependencies, OS incompatibilities and environmental hooks that require being set ‘just so’ for the entire shebang to function.

When I need to deploy across a mixed network of dozens app servers as well as get my app onto 20 developers dev tree’s, multiple staging, production and dev systems and keep everything moving smoothly perl does.. not.. help.

This is where a language like Java shines. The same app will run on windows, linux, mac os x and is almost entirely independent of its ‘environment’ outside the prerequisite (universal) java env vars. All libs are stored local to the app, it mostly ignores the OS level environment and the same libs run everywhere exactly the same (take that DBD::Oracle and your dumb client dependencies..) Yes, there is a big overhead of learning to be done, particularly coming from perl, but once your past that its pretty darn slick.

I’ve gotten around this ‘deployability’ issue with perl by creating one click/script build/deploy systems that pull CPAN modules from the project source control, build perl, apache and mod_perl from source controlled tarballs and strictly enforcing ‘The Environment Is Evil, Do Not Depend On It’ coding standards. This works really well but requires a lot of discipline and a TON of up-front work.

But what About CPAN/Debian/MyFavoriteDistro?!?!

And before you go touting CPAN as ‘all that and a stack of bricks’ let me highlight just a few incidents of lameness I have seen in a decade of perling..

• CGI.pm treats ‘undef’ differently in different versions in different circumstances. Yea, that hurts.
• The DBI Postgres drivers dramatically changed the internal SQL statement behavior and then DEFAULTED TO THE NEW BEHAVIOUR. (Can you say ‘Hello broken apps’? We sure did..)
• Storable changed its storage format which means different machines running different versions would corrupt data. Woops, thank goodness for a real multi-server test environment..
• on and on and on. This is just in the last 2 years or so.

CPAN was/is awesome but depending on it, or any uncontrolled release process, is stupid. Grab your modules, test them and then source control them. Don’t treat your linux distro like a source control system, its not, the Debian or Ubuntu guys don’t care about you. Case in point, deprecating Apache 1.3.x entirely and forcing mod-perl users to move to Apache 2.2. Sounds fine but when you have a 100,00+ lines of 10 year old perl code that have been grafted into mod_perl being forced to switch to mod_perl 2 is not happy or trivial. Sure, we can compile 1.3.x ourselves but that really defeats the point of a package management system..

Oh, and about those bajillion CPAN modules. It might be important to note

A) How many of these are just Base Features in most other languages.

B) The fact that most perl modules are factored down to very granular levels (Good Engineering Practice), increasing the module count.

I’ve found that a good 85-90% of my lib needs are covered in the core Java language + the apache-commons packages. Everything else is an edge case where either perl or java or ruby or whatever generally is going to have some sort of library that is half-tested but available and better then nothing..

Quantity is not Quality and Quantity is not a measurement of the level of support a language is getting.

Hold your nose, set your browser to ‘private browsing’ so no one knows you looked and check out the apache-commons and base java api’s, I bet you’ll find that it would provide you with a big slice of your CPAN requirements…

http://commons.apache.org/

I do love CPAN but a bajillion modules doesn’t mean anything really outside of bragging rights. And yes, there is some awesome cool Obscure Stuff in CPAN but it is dwindling as perl phases out and the next generation of languages start to arrive. Tried using some of that stuff from CPAN lately? Egads, most of it hasn’t been updated in 3+ years.

If you are building large apps in perl and are honest with yourself you should be nodding in agreement before telling me I am an idiot for using Java and not your favorite unstructured/untyped scripting language.

Summary

‘Deployability’, if you ever build a big system this will become one of the most important problems you face. If you don’t think this is true then you A) are not building large systems or B) don’t work in a multi-developer environment or C) have a manager/sysadmin/fellow coder that really loves you and shelters you from the harsh world of deploying apps outside of your dev tree.

If you don’t have a dev tree you aren’t building a real app. Sorry, its true. None of this applies to you, you can now switch back to vi and continue coding. Thanks for reading this far, be sure to USR1 your development environment when making changes so that your live users (mostly) don’t notice.

So, when you pick your next language think about its deployment mode and if its paradigm will really support what you need to get done. In the end I ended up moving to Java for most everything. Robust library support, 100% reproducibility in almost any environment, suitable performance characteristics and the best userland tools going. Yes vi is cool, I get it, you rock with vi. Congrats, I find it difficult to navigate a huge codebase in vi and the lack of tools support is insane, but that’s me, I am the Tools Using sort of monkey. Just about every single one of my developers is a vi bigot so I am used to being stared at.

That said, I am currently running a really rather very large perl codebase with tons of traffic and it is an absolute blast sometimes.

Apache Wicket provides an interesting out of the box component called a ‘Wizard’ that does just what you’d expect it to, create an easy to use Wizard workflow that you can insert into your page. It provides next, previous and cancel buttons as well as tracking the state between steps and enabling the ‘Finish’ button when appropriate. You can see the genericwizard in action here : http://www.wicket-library.com/wicket-examples/wizard/

The base markup that comes with the wizard is ‘adequate’ in that ‘it works’ but its complex table based structure is at odds with modern CSS based markup making it difficult to adapt at times. Luckily  (ok, by design..) its totally and completely customizable (as all good Wicket Things are) however doing so is a bit obtuse and generally left as an exercise for the reader. Here is a step by step for how you customize the look/feel of your wizard in Wicket. I won’t be explaining the base case, the link above details that. This tutorial won’t go “all the way”, if you really want to customize to the ‘nth degree its easy enough once you understand how the basics work.

When in doubt, download the wicket source and follow the bouncing ball, thats all I did. This quick tutorial also touches on how to customize a few other pieces of the Wizard process but the core of it is demonstrated in the first section.

You can download my Quickstart here: wizard-quickstart.zip

Create A Wizard without tables!

Lets create a Table-less Wizard! A Wizard is basically a Panel that extends ‘Wizard’, lets call ours “YourCustomWizard” and build out its base methods, assign it a model and add a few steps. In the Real World your steps are probably going to be dynamic. If you want a quick tutorial on that let me know but its pretty easy to get through.  Note that the HTML on the right has NO TABLES. You are now in control of your own CSS destiny.

Put simply (thanks Nathan!) “To override the default markup, make a markup file using the same basename as your custom Wizard class (e.g., “YourCustomWizard.html”) with the following wicket:id attributes: form, overview, header, view, feedback and buttons. With that knowledge you can stop reading. Continue on for code samples.

Your welcome. NOTE: The Old Fat Wizard is the html that ships with the Wicket source.

(/src/jdk-1.4/wicket-extensions/src/main/java/org/apache/wicket/extensions/wizard/Wizard.html)

Now that you have the base system of Custom Wizardry in place lets look at some more components you can customize.

What’s in a Wizard Step?

Lets first look at our example step “YourWizardStep()”. In our case this is a panel that extends the WizardStep class with a nice form that collects ‘elementvalue’ and displays two fields, ‘elementlabel’ and ‘elementdescription’. Its important to stop for a second and understand that I am using the same class for *all* of my wizard steps. You are not bound by this, you can create a multitude of wizard steps and add them to your wizard as you see fit. Your Wizard can do things like branch steps based on the previous set, force users to stay on a step and you can even embed a full wizard within a step for some Wizard On Wizard action.

Add an Overview

The overview is just a bit of markup that is placed in the element named ‘overview’. Nominally this is used to give the user an ‘overview’ of what this wizard is all about etc.  This is pretty vanilla but we are nothing if not complete.

Pressing My Buttons

The other thing developers usually want to customize are the buttons. You may need to add custom classes in order to Prettify them, you might want to use images. Buttons always need some help. The Wizard by default places buttons in a wicket element  component with the id ‘buttons’. To create your own ButtonBar you just need to … wait for it .. extend Panel and make sure you have some specific markup in the page to indicate to wicket what element to attach its actions to.

Easy! However if you look closely you will notice that One Of These Buttons is doing his own thing. We are attaching a button called ‘YourLastButton’ to the wicket element ‘last’. This is a custom button implementation as an example of what can be done. Note that you are not controlling the view of a button, just the controller. You’d do this if you wanted to add some customer behavior to a button or override one of the methods like isEnabled(). The most general case is to add an update workflow to the onClick() that updates the model somehow.

Summary

This is a basic introduction on how to implement your own Wizard markup in place of the mark-up that the wizard puts out. I had to get through this on a project because throwing the default table based layout into my pages was causing some serious issues. By creating my own extended Wizard classes I was able to totally customize the look/feel of the wizard and ensure pixel perfect design. In my case I needed to use NiceForms which I highly recommend. Particularly since you can now Customize Your Wicket Wizard..

You can do more with Wizards, much more. This tutorial is not meant to be comprehensive. Check out the Wizard API for a total rundown on what can be sub-classed. Take a look at the DynamicWizardStep and other classes, they really open the power of the Wizard. Also refer to the source for the base implementations. Open Source is an invaluable resource so Read The Source. The Wizard code is short and easy to understand since it uses the common Wicket constructs (panels etc). Don’t be intimidated.

Now, Go Forth And Wizardify.

There is a serious and mostly ignored split between Programming as a Trade and Programming as a Science.

There is a huge and legitimate need for people who can make a Core 2 Duo bleed by simply looking at it and thinking harsh and deep computer sciency thoughts. These folks tend to be on the science edge of the spectrum, you need them to be. You may be plumbing the depths of the universe, doing 15 degrees of relationships among a population of billions, inventing new and innovative compression algorithms, developing operating systems that run on graphics cards as parallel processing platforms or peering through an atom, these people are scientists. They make the tools that make the rest of us go. These folks don’t get paid tons but they also tend to go to work in big safe buildings with lots of locks and guards and almost never get fired.

Then there are the trade folks. Computer Science as a Blue Collar occupation. All of the Java, C, C++, perl, python etc folks who just want a paycheck, get enjoyment out of building things and occasionally, very occasionally, hit it big. Thankfully there are enough of the blue collar folks that the ‘hit it big’ ratio appears to be fairly high and thus encourages the rest of the interested to jump in and try their luck. But, for the most part, the blue collar folks are head down in the trenches on the hamster wheel of quarterly releases that never end.

Incidentally, these folks, just like auto workers, tend to get outsourced.

If you really want to get extreme, for the Blue Collar crowd, there is no longer a relevant need to understand Big O/Sigma/Theta and they really don’t care about sorting algorithms or how an operating system manages its virtual memory. Their ability to do good or bad is limited, incidentally just like the auto workers, by their tools compensating for their lack of knowledge/care (optimized compilers vs safety robots). The constraint of range that both the auto-worker robots and Blue Collar developer compilers/tools enforce is at the same time liberating as it is constraining. In some ways you can Do More Faster and in other ways you Just Cant Do As Much. But as long as those White Collar guys keep getting paid to make sure the tools compensate everything will be a-ok.

This is also why you don’t see a lot of Blue Collar guys doing embedded work, the toolsets are not engineered for safety so those tend to be more Sciency then Trade.

Anyhow, this ‘must have a college degree’ is an attempt to maintain software engineering as a White Collar profession for -all- of its members and maybe does us all a bit of a disservice when, truly in todays service economy, it has become Blue Collar in the western world.

Then again, maybe we’d all be better if everyone could make processors bleed by looking at them.. I just don’t know, but I do know that in my recruiting efforts seeing this split and understanding the requirements for a given position dramatically increases my success rate in hiring Right Fits for my team.

Your mileage may vary.

Its a very flexible way to manage your A/V distribution needs. However the real problem with it is controlling it, these matrix switchers are Old School and require Serial commands. Here is an example session over the serial port:


CI3O6T


That means Change Input 3 to Output 6 Take

Fun right? Well, maybe for me but my wife and children hate it.

A few years ago I wrote an interface for my Pronto remote that relied on Girder, a Windows based home automation package. Well, the pronto went belly up and the media PC was then replaced by an AppleTV and now, whenever we want to switch, I have to VNC to an old G4 laptop in the garage to issue the switching commands. Terrible..

So, this new category, AppleTV, is where I will document my quest to turn my AppleTV + my Harmony remote into a Matrix Switch Controller (on top of its normal duties). I have a roadmap prepared of all the tasks to accomplish in order to make this a reality. Some are these are done already but I will list them for historical purposes.

1. Hack my AppleTV. It needs to be SSH, fink (for svn and other utils) and USB enabled.
2. Get the AppleTV to recognize the USB-UIRT so that I can send it IR commands.
3. Get the AppleTV running java and jetty, so I can create a simple web UI to manage configurations
4. Convince the AppleTV to recognize and interact with the Keyspan USB->Serial port dongle
5. Get the USB->Serial Dongle and the USB-UIRT functioning on a USB hub
6. Get ribsu to receive commands from the USB-UIRT on appletv.
7. Get ribsu to flush its STDOUT so that I can exec it from java and work with the output in real-time
8. Prototype a simple control app via Jetty to prove that it all snaps together
9. Can I get wicket or some other jetty controlled app to do some AppleTV UI control so I can break the web barrier?

These items will prove it all fits together properly and will allow me to design my application. This app has some high level features, here they are:

1) JSON interfaces for all functions for future automation purposes.
2) Lightweight server process / highly visual and dynamic web app in javascript so the weak AppleTV doesn’t get burdened.
3) Ability to create configurations that can be referenced as a whole (EG: Switch DVD to Bedroom (CI2O5T) )
4) Ability to learn remote codes and then assign them to a configuration

That is the base minimum to allow my usage scenario of teaching the harmony remote how to do these things.

The future dreamy stuff would be to create some sort of plugin architecture so folks can extend it to do other things. Thats out a bit, right now I just need to be able to control the switcher..

Double-clickable JAR

You want users to double click the application, just like any other application. This is actually easy and well-documented, you just create a manifest.xml at the root of the app that looks like this:

Manifest-Version: 1.0
Sealed: true
Main-Class: gui.SyncApp

Replace ‘gui.SyncApp’ with the class that contains your main and your in business. Any recent JRE will launch this for you.

Bundled Libraries

Ok this one is a real bummer. Any reasonably modern Java app will have approx 35,431 JAR files that it requires. Unfortunately JAR files can not reference themselves internally in regards to the Class-Path for some esoteric reason that probably makes great sense on paper but utterly falls down in the Real World.

So when you do the normal ‘Export as JAR’ any bundled up JARs are useless! The answer is to unzip ALL of your jar libraries and bring the packages right into your app. Luckily Eclipse 3.4+ has a new export option “Runnable JAR”. This is a continuation of the very successful Fat-JAR project that has merged into Eclipse and basically decompresses everything for you and also pulls in all none code files for referencing as required.

SideNote: Fat-JAR is not OS X compat but the Export->Runnable seems to work great.

Embedded Databases and ORMS

For this particular app I merged the greatness of Derby (a pure java embedded database) with the fantastic Apache Cayenne ORM. This combination gave me great features in a nice tight package. The problem was how to deploy these things in a JAR file.

Problem 1: Data Source location

Where does the database live since its a file on disk? My app needs to run on Windows, Linux and Mac, all of which have dramatically different file system layouts. The answer here is:

System.getProperty("user.home") + "/myDbName";

This ensures that every OS does the Right Thing.

Problem 2: Tell the ORM where the database is

Now that I have a database how the heck do we tell Cayenne how to find it? Cayenne, like most Java frameworks, requires a static xml file for configuration. Unfortunately for me my database location is dynamic (based on the value of user.home) and the cayenne xml file is does not accept variables (like Ant does). After some great help from the listserv I simply implemented my own DataSourceFactory which turned out to be a lot less scary then I thought it would be.


public DataSource getDataSource(String location) throws Exception {

// All done!
return poolManager;
}


Now the app can dynamically connect to the proper database for its host OS file system.

First Run…

Next Problem! On first run the database does not yet exist but Cayenne assumes that it DOES exist. This makes Cayenne unhappy, confused and exception prone since connecting works, Derby autocreates the base database, but any real work fails since the database is empty! We need a way to create and populate the database on first run.

Easy enough, we just run some simple checks post-connect and if they fail we build out the tables required.

// Check if the schema is up to par
if (!checkSchema()) {
createSchema();
}


All checkSchema() does is attempt a simple query that should succeed if the table exists (even if it returns 0 rows). If it fails we create the table structures as required using various mechanisms and the createDDL function below. I chose Ant for this, Ant does things like this exceptionally well.

Headless Ants – Creating the table structure on the fly

Just because Ant does something well doesn’t mean that Real Humans can use it. No Real Human is going to execute an Ant script in order to use my cool little tool. This means we have to create Headless Ants that are automated. Easy enough! Ant, like all serious things Java, has an API.


public static void createDDL(String build, String db, String theDatabase) {
Project p = new Project();
p.init();
p.setProperty("dbFile", theDatabase);
File buildFile = new File(build);
File dbFile = new File(db);
System.err.println("File is "+buildFile.length());
ProjectHelper.configureProject(p, buildFile);
p.executeTarget("import-target-db");
}


All we do is pass this function the build.xml path, the db-schema.xml path and the database path. The Ant ddlToDatabase function takes care of the rest for us. Thanks Ant!

Ant Hates JARs

Here is where I really lost it, I simply could NOT coerce Ant into creating the database when running from the JAR file. In Eclipse everything was fine but from the JAR it would constantly bomb out because it could not find the build.xml file. After hours and hours of useless Googling and experimenting I finally discovered the following:

A) You CAN get a handle to the build.xml file using the ClassLoader
java.io.InputStream is = ClassLoader.getSystemResourceAsStream("build.xml");
B) However Ant has no idea how to deal with this, even when you load it as a URL instead of an input stream and pass Ant URL.getFile().

Solution? Copy this file to the users home.dir as .build.xml and pass THAT into the ProjectHelper. Delete when the operation completes.

I also had to do the same with the db-schema.xml file. Once these two files lived outside of the JAR Ant was happy, the database built and the app ran.

Conclusion

None really, once you know how it all snaps together its pretty easy stuff but getting there took a long long time. The docs are terrible and Google can’t find much on the topic that is anything more then other clueless developers begging for help.

Ant does suck in relation to JAR files although I am almost positive I could have done the entire thing in pure Java with no XML somehow. Possibly there is a better solution and way to get Ant to be JAR aware but given how horrible Java support for runnable JARs is I am doubting that this Better Way exists..

Bonus Points

I’ve used Install4J to create native OS packages (both JRE and non-JRE included versions). Fun!

Bummer Points

JNLP (java webstart) runs in a web sandbox so file system access is really difficult. They provide a simple API for file access but since my tools are mostly bundled up jars from other projects it proved exceedingly difficult to teach those how to use the new API for local file access (xml files etc). My solution here was to Give Up since I already had native app installers.

Recently he installed an Untangle server so that his users would get less Spam, he could start shaping traffic/dealing better with PeerToPeer traffic and other such things. He really likes the product and when it came time to add a few servers to his network we decided to virtualize them in VMWare.

The preferred configuration for this system was
VM1 : Untangle
VM2 : Web
VM3 : Mail

All well and good until you understand that Untangle is a Transparent Bridge, it needs to sit on the network between machines so it can manage all the traffic. Transparent Bridges are a nice piece of Layer 2 magic that look like a Firewall but are zero config and do not require network changes, NATs etc. They simply grab all incoming traffic on the external port, sorts and manages packets and then passes approved traffic/modified traffic onto the internal network. Likewise for outgoing traffic (swap external/internal). All “Internal” machines get External IP’s so you get the power of a traffic shaping firewall with the simplicity of a single network.

So, how does this work on a VMWare box? Easy and obvious but I want to confirm it works and works well.

Obtain a server with more then one network port. VMware Server will route packets between them even if they are not physically connected. So, in this example we put Untangles external network on ETH0 which was “connected” (configured really, its a VM) to speak to the outside network and then we configure its internal network to bridge to ETH1 which has no network connected.

Any protected VM’s just get configured to use the Physical ETH1 as their device. Give them an external IP address (its a transparent bridge, remember, not a firewall) and you will be off to the races.

Bonus, you can connect a physical network segment to ETH1 and Untangle in the VM will protect both the Physical and Virtual machines.

Its quite slick and I am very impressed, good work Untangle.