A Place for John on the Web » Software Projects http://www.siberian.org Just another WordPress weblog Sun, 21 Mar 2010 14:51:49 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 GXT/GWT, DTOs and Dozer : Managing the Exception rather then the Common Case http://www.siberian.org/2010/02/04/gxtgwt-dtos-and-dozer-managing-the-exception-rather-then-the-common-case/ http://www.siberian.org/2010/02/04/gxtgwt-dtos-and-dozer-managing-the-exception-rather-then-the-common-case/#comments Thu, 04 Feb 2010 18:56:12 +0000 John http://www.siberian.org/?p=230 For all of its amazing cleverness, GWT has one really difficult area, persistence. There are various projects and attempts to clear this are up but most are very young and have a ways to go. This gets even more complicated if your not, lets face it, using Hibernate. As a Cayenne fan I don’t use Hibernate so of course my path is even more complex then most. Persistent objects, in most popular frameworks, carry with them a lot of persistent baggage that doesn’t cleanly serialize across the wire.

Thankfully there is, as with most things software related, a pattern! The pattern in this case is to use a Data Transfer Object (DTO). A DTO is basically a serializable bean that mirrors the persistent object. This is an annoying duplication of work and obviously violates the DRY (Don’t Repeat Yourself) principle since you are essentially cloning the Persistent object as a basic java bean. The upside of it is that you can control exactly what gets into your DTO (and thus what is transferred to the client). Originally I was constructing these beans as required, doing a pretty ad hoc job of it and generally causing myself headaches.

My first attempt at streamlining this was to use the BeanUtils apache commons project. This library has some simple methods for copying properties between beans which, in the end, is what the DTO is all about. This worked well for common and primitive types that were shared (name and type) between the two objects but it fell over pretty quickly if the types mismatched. Additionally, there is no easy way to exclude or customize an individual field level map and no way to do one-way mappings (there is never a reason to map the ID from the Bean to the Persistent Object!)

What are my requirements and why didn’t BeanUtils not working?

  • BeanUtils is exceptions based: If it has a mismatched attribute it blows up spectacularly and you then have to treat those objects differently.
  • BeanUtils doesn’t have an exceptions mechanism: In my situation where I have enumerations everywhere this meant that many objects of mine are not eligible for BeanUtils. (GWT doesn’t easily support enumerations so I have to convert them to int before sticking the int in the DTO with a map of values)
  • BeanUtils doesn’t have an extensions mechanism: Once your object is out of the path of automation there is no way back, its a manual coding exercise.

There was a better way: Dozer

I had looked at Dozer a few times and it just seemed Scary and Large but in my desperation I started to dig around and discovered how addictingly easy it is to start with Dozer:

public static void setUser(UserBean bean, Pguser user){
     Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance();
     mapper.map(bean,user);
}

This is your base ‘BeanUtils replacement’ usage of Dozer. In my case this is part of a larger static class that is a big BeanFactory, eg: A bit file that has a bunch of static methods for converting beans to objects and objects to beans.

Fantastic! Now my conversions did not blow up in the general case where one object had a field that the other one lacked. However, it still exploded when types mismatched. Thats where the Dozer mapping comes in. Dozer allows you to create an XML file of exceptions that run as a part of the map(a,b) process. You don’t have to specific ALL fields in this mapping, just the differentials! Once I understood that it was happy-pants-time!

All you do is toss a file called ‘dozerBeanMapping.xml’ into your class path and Dozer automatically picks it up. No muss no fuss. Lets look at one of my beans in this file. In the below example I am basically excluded my enumerations from the mapping process. Note: you can do a LOT more then this, my goal here is to show how easy it is to use Dozer to manage your bean<->object exceptions.

<mapping>
   <class-a>com.pgi.common.Clientsettings</class-a>
   <class-b>com.pgi.webapp.client.dto.ClientsettingsBean</class-b>
   <field-exclude>
      <a>jobmode</a>
     <b>jobmode</b>
   </field-exclude>
   <field-exclude>
      <a>removemethod</a>
      <b>removemethod</b>
   </field-exclude>
   <field-exclude>
      <a>reportingmode</a>
      <b>reportingmode</b>
   </field-exclude>
</mapping>

But now how do I put my enumerations into my DTO? Two options really

For now I am putting this mapping in my factory method post auto-map

public static ClientsettingsBean getClientsettingsBean(Clientsettings settings){
   Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance();
   ClientsettingsBean theBean = mapper.map(settings,ClientsettingsBean.class);
   theBean.setJobmode(settings.getJobmode().ordinal());
   theBean.setReportingmode(settings.getReportingmode().ordinal());
   theBean.setRemovemethod(settings.getRemovemethod().ordinal());
   theBean.setId(DataObjectUtils.intPKForObject(settings));
   return theBean;
}
There is another way though, I can create a custom Dozer Converter that knows how to do this mapping for me. I am still working on this though, Enumerations in my case are tough as I have about 15 of them and they all have a custom methods that goes from int to enum. Since I don’t want 15 converters that do the same thing, just with a different class, I need to find a way to extend my enumeration from a base class that has this mapping.
With a base enumeration class (if possible, haven’t looked deeply yet) I can have one Dozer Conversion class that maps back and forth between any of my enumerations and the class that has that enumeration as an attribute. Should be fun.
An intriguing idea for the future would be a Cayenne template that auto-creates these DTOs, Factory and Mapping files for me as part of the Cayenne class generation process. I’d still need to copy them manually into my GWT *.client.* namespace but it would speed things up considerably.
]]> http://www.siberian.org/2010/02/04/gxtgwt-dtos-and-dozer-managing-the-exception-rather-then-the-common-case/feed/ 0 Ajax Wizards in Wicket http://www.siberian.org/2009/10/29/ajax-wizards-in-wicket/ http://www.siberian.org/2009/10/29/ajax-wizards-in-wicket/#comments Fri, 30 Oct 2009 03:42:00 +0000 John http://www.siberian.org/?p=174 Lately I’ve found that Wicket + Ajax == Very nice. More then nice its turns out its just a lot easier to build sites using Ajax and Wicket then it is to do it otherwise. Last night I found myself wanting to place a wizard into an entirely Ajax driven site. Luckily Google pointed me to the answer in this thread: Ajax Wizard Button in Modal Window.

In the spirit of duplicating information all over the internet I’ll paste my copy/paste implementation of the presented solution. It works well!

Step 1: Create the AjaxWizardButton class

  1. import org.apache.wicket.ajax.AjaxRequestTarget;
  2. import org.apache.wicket.ajax.markup.html.form.AjaxButton;
  3. import org.apache.wicket.extensions.wizard.IWizard;
  4. import org.apache.wicket.extensions.wizard.IWizardModel;
  5. import org.apache.wicket.markup.html.form.Form;
  6. import org.apache.wicket.model.ResourceModel;
  7.  
  8. public abstract class AjaxWizardButton extends AjaxButton {
  9.  
  10.         private static final long serialVersionUID = 1L;
  11.         private final IWizard wizard;
  12.  
  13.         public AjaxWizardButton(String id, IWizard wizard, final Form form,
  14.                         String labelResourceKey) {
  15.                 super(id, form);
  16.                 this.setLabel(new ResourceModel(labelResourceKey));
  17.                 this.wizard = wizard;
  18.         }
  19.  
  20.         public AjaxWizardButton(String id, IWizard wizard, String labelResourceKey) {
  21.                 this(id, wizard, null, labelResourceKey);
  22.         }
  23.  
  24.         protected final IWizard getWizard() {
  25.                 return wizard;
  26.         }
  27.  
  28.         protected final IWizardModel getWizardModel() {
  29.                 return getWizard().getWizardModel();
  30.         }
  31.  
  32.         protected final void onSubmit(AjaxRequestTarget target, Form form) {
  33.                 onClick(target, form);
  34.         }
  35.  
  36.         protected abstract void onClick(AjaxRequestTarget target, Form form);
  37. }

Step 2: Create the buttonbar that will wrap these buttons and override the native buttonbar built into the Wizard.

  1.  
  2.  
  3. import org.apache.wicket.ajax.AjaxRequestTarget;
  4. import org.apache.wicket.extensions.wizard.IWizardModel;
  5. import org.apache.wicket.extensions.wizard.IWizardStep;
  6. import org.apache.wicket.extensions.wizard.Wizard;
  7. import org.apache.wicket.extensions.wizard.WizardButtonBar;
  8. import org.apache.wicket.markup.html.form.Form;
  9.  
  10. public class AjaxWizardButtonBar extends WizardButtonBar {
  11.  
  12.         private static final long serialVersionUID = 1L;
  13.  
  14.         public AjaxWizardButtonBar(String id, final Wizard wizard) {
  15.                 super(id, wizard);
  16.  
  17.                 addOrReplace(new AjaxWizardButton("next", wizard, "next") {
  18.                         @Override
  19.                         protected void onClick(AjaxRequestTarget target, Form form) {
  20.                                 IWizardModel wizardModel = getWizardModel();
  21.                                 IWizardStep step = wizardModel.getActiveStep();
  22.  
  23.                                 // let the step apply any state
  24.                                 step.applyState();
  25.  
  26.                                 // if the step completed after applying the state, move the
  27.                                 // model onward
  28.                                 if (step.isComplete()) {
  29.                                         wizardModel.next();
  30.                                 } else {
  31.                                         error(getLocalizer()
  32.                                                         .getString(
  33.                                                                         "org.apache.wicket.extensions.wizard.NextButton.step.did.not.complete",
  34.                                                                         this));
  35.                                 }
  36.  
  37.                                 target.addComponent(wizard);
  38.                         }
  39.  
  40.                         public final boolean isEnabled() {
  41.                                 return getWizardModel().isNextAvailable();
  42.                         }
  43.                 });
  44.  
  45.                 addOrReplace(new AjaxWizardButton("previous", wizard, "prev") {
  46.                        
  47.                         protected void onClick(AjaxRequestTarget target, Form form) {
  48.                                 getWizardModel().previous();
  49.                                 target.addComponent(wizard);
  50.                         }
  51.  
  52.                         public final boolean isEnabled() {
  53.                                 return getWizardModel().isPreviousAvailable();
  54.                         }
  55.                 });
  56.         }
  57.  
  58. }
  59.  

Step 3: Alter your wizard html markup so you can override the native buttonbar

  1.  
  2. <html xmlns:wicket>
  3. <wicket:panel>
  4. <div>
  5. <form wicket:id="form" class="wicketExtensionsWizardForm">
  6.  
  7. <span wicket:id="overview"></span>
  8. <span wicket:id="header"></span>
  9.  
  10. <div wicket:id="view"></div>
  11.  
  12. <span wicket:id="feedback"></span>
  13. <div class="buttons">
  14. <span wicket:id="buttons"></span>
  15. </div>
  16. </form>
  17. </div>
  18. </wicket:panel>
  19. </html>
  20.  

Step 4: Set your wizard markup output ID so that you can target it with Ajax calls

  1.  
  2. theWizard.setOutputMarkupId(true);
  3.  

Step 5: Override the buttonbar call in your wizard class

  1.  
  2.         protected Component newButtonBar(java.lang.String id) {
  3.                 return new AjaxWizardButtonBar(id, this);
  4.         }
  5.  

Step 6: Bask in the glow of warm Ajax.

Pretty simple but the impact is fantastic. Using this code I was able to Ajaxify my wizard in well under 10 minutes.

Thanks Wicket!

]]> http://www.siberian.org/2009/10/29/ajax-wizards-in-wicket/feed/ 3 Customizing Wicket Wizard Markup http://www.siberian.org/2009/06/24/customizing-wicket-wizard-markup/ http://www.siberian.org/2009/06/24/customizing-wicket-wizard-markup/#comments Wed, 24 Jun 2009 08:04:23 +0000 John http://www.siberian.org/?p=77 Overview

Apache Wicket provides an interesting out of the box component called a ‘Wizard’ that does just what you’d expect it to, create an easy to use Wizard workflow that you can insert into your page. It provides next, previous and cancel buttons as well as tracking the state between steps and enabling the ‘Finish’ button when appropriate. You can see the genericwizard in action here : http://www.wicket-library.com/wicket-examples/wizard/

The base markup that comes with the wizard is ‘adequate’ in that ‘it works’ but its complex table based structure is at odds with modern CSS based markup making it difficult to adapt at times. Luckily  (ok, by design..) its totally and completely customizable (as all good Wicket Things are) however doing so is a bit obtuse and generally left as an exercise for the reader. Here is a step by step for how you customize the look/feel of your wizard in Wicket. I won’t be explaining the base case, the link above details that. This tutorial won’t go “all the way”, if you really want to customize to the ‘nth degree its easy enough once you understand how the basics work.

When in doubt, download the wicket source and follow the bouncing ball, thats all I did. This quick tutorial also touches on how to customize a few other pieces of the Wizard process but the core of it is demonstrated in the first section.

You can download my Quickstart here: wizard-quickstart.zip

Create A Wizard without tables!

Lets create a Table-less Wizard! A Wizard is basically a Panel that extends ‘Wizard’, lets call ours “YourCustomWizard” and build out its base methods, assign it a model and add a few steps. In the Real World your steps are probably going to be dynamic. If you want a quick tutorial on that let me know but its pretty easy to get through.  Note that the HTML on the right has NO TABLES. You are now in control of your own CSS destiny.

Put simply (thanks Nathan!) “To override the default markup, make a markup file using the same basename as your custom Wizard class (e.g., “YourCustomWizard.html”) with the following wicket:id attributes: form, overview, header, view, feedback and buttons. With that knowledge you can stop reading. Continue on for code samples.

Your welcome. NOTE: The Old Fat Wizard is the html that ships with the Wicket source.

(/src/jdk-1.4/wicket-extensions/src/main/java/org/apache/wicket/extensions/wizard/Wizard.html)

package wicket.quickstart;

import wicket.Component;

import wicket.extensions.wizard.Wizard;

import wicket.extensions.wizard.WizardModel;

public class YourCustomWizard extends Wizard {

/*

* Create a wizard with a default model

* and three custom steps. Each step

* is a form with some default data

* in the model for example purpose.

*/

public YourCustomWizard(String id){

super(id);

WizardModel model = new WizardModel();

YourObject theObj = new YourObject();

theObj.setElementdescription(“First item”);

theObj.setElementlabel(“Label me”);

theObj.setElementvalue(“my value”);

YourObject theObj2 = new YourObject();

theObj2.setElementdescription(“Second item”);

theObj2.setElementlabel(“Label me2″);

theObj2.setElementvalue(“my value2″);

YourObject theObj3 = new YourObject();

theObj3.setElementdescription(“Third item”);

theObj3.setElementlabel(“Label me3″);

theObj3.setElementvalue(“my value3″);

model.add(new YourWizardStep(theObj));

model.add(new YourWizardStep(theObj2));

model.add(new YourWizardStep(theObj3));

init(model);

}

/**

* On finish.

*

* Put any specific logic you want when the

* user hits the “finish” button eventually

* Here we print to error log and send them

* to the home page.

* Normally used for saving unclean objects

* to database if you did not do it in the WizardStep

*/

@Override

public void onFinish() {

System.err.println(“DONE WITH THE WIZ!”);

setResponsePage(Index.class);

}

/**

* On cancel.

*

* Put your cancel logic in for when users

* hit the ‘Cancel’ button.

* I send them to the home page

*/

@Override

public void onCancel() {

setResponsePage(Index.class);

}

protected Component newButtonBar(java.lang.String id){

return new ButtonBarPanel(id, this);

}

protected Component newOverviewBar(java.lang.String id){

return new OverviewPanel(id);

}

}

The New Lean And Mean Wizard

(YourCustomWizard.html)


<wicket:panel>

<div>

<form wicket:id=“form”class=“wicketExtensionsWizardForm”>

<span wicket:id=“overview”></span>

<span wicket:id=“header”></span>

<div wicket:id=“view”></div>

<span wicket:id=“feedback”></span>

<div class=“buttons”>

<span wicket:id=“buttons”></span>

</div>

</form>

</div>

</wicket:panel>

The Old Fat Wizard

(Wizard.html)

<wicket:panel>

<div class=“wicketExtensionsWizard”>

<form wicket:id=“form” class=“wicketExtensionsWizardForm”>

<table class=“wicketExtensionsWizardOuterTable”>

<tr>

<td valign=“top”>

<span wicket:id=“overview”></span>

</td>

<td valign=“top”>

<table class=“wicketExtensionsWizardInnerTable”>

<tr class=“wicketExtensionsWizardHeaderRow”>

<td valign=“top” class=“wicketExtensionsWizardHeader”>

<span wicket:id=“header”></span></td>

</tr>

<tr class=“wicketExtensionsWizardViewRow”>

<td valign=“top” class=“wicketExtensionsWizardView”>

<div wicket:id=“view” class=“wicketExtensionsWizardViewInner”></div></td>

</tr>

<tr class=“wicketExtensionsWizardFeedbackRow”>

<td valign=“top” class=“wicketExtensionsWizardFeedback”>

<span wicket:id=“feedback”></span></td>

</tr>

<tr class=“wicketExtensionsWizardButtonBarRow”>

<td valign=“top” class=“wicketExtensionsWizardButtonBar”>

<span wicket:id=“buttons”></span></td>

</tr>

</table>

</td>

</tr>

</table>

</form>

</div>

</wicket:panel>

Now that you have the base system of Custom Wizardry in place lets look at some more components you can customize.

What’s in a Wizard Step?

Lets first look at our example step “YourWizardStep()”. In our case this is a panel that extends the WizardStep class with a nice form that collects ‘elementvalue’ and displays two fields, ‘elementlabel’ and ‘elementdescription’. Its important to stop for a second and understand that I am using the same class for *all* of my wizard steps. You are not bound by this, you can create a multitude of wizard steps and add them to your wizard as you see fit. Your Wizard can do things like branch steps based on the previous set, force users to stay on a step and you can even embed a full wizard within a step for some Wizard On Wizard action.


package wicket.quickstart;

import wicket.extensions.wizard.WizardStep;

import wicket.markup.html.basic.Label;

import wicket.markup.html.form.TextField;

import wicket.model.CompoundPropertyModel;

import wicket.model.PropertyModel;

public class YourWizardStep extends WizardStep {

public YourWizardStep(YourObject object1){

setModel(new CompoundPropertyModel(object1));

add(new Label(“elementlabel”));

add(new Label(“elementdescription”).setEscapeModelStrings(false));

TextField theField = new TextField(“elementvalue”,new PropertyModel(object1, “elementvalue”));

theField.setRequired(true);

add(theField);

}

/*

*  This is called when the next button i clicked,

*  I assume you want to use your for for data gathering..

*  Normally you’d save the object the the database

*  Or just leave it in memory and save it in the Wizard onFinish()

*/

public void applyState(){

System.err.println(“Applying state!”);

setComplete(true); // Set this step as done, you should put custom logic here

}

}

<html xmlns:wicket>

<wicket:panel>

<fieldset>

<legend>

<span wicket:id=“elementlabel”></span>

</legend>

<span wicket:id=“elementdescription”>[Element Description]</span>

<br/>

<input type=text wicket:id=“elementvalue”>

</fieldset>

</wicket:panel>

</html>

Add an Overview

The overview is just a bit of markup that is placed in the element named ‘overview’. Nominally this is used to give the user an ‘overview’ of what this wizard is all about etc.  This is pretty vanilla but we are nothing if not complete.

package wicket.quickstart;

import wicket.markup.html.panel.Panel;

/*

* The Overview panel is displayed in the Overview div automatically.

* It serves no special purposes other then for the default wizard

* to display common information across pages.

*/

public class OverviewPanel extends Panel {

public OverviewPanel(String id){

super(id);

}

}

<html xmlns:wicket>

<wicket:panel>

Overview Panel

</wicket:panel>

</html>

Pressing My Buttons

The other thing developers usually want to customize are the buttons. You may need to add custom classes in order to Prettify them, you might want to use images. Buttons always need some help. The Wizard by default places buttons in a wicket element  component with the id ‘buttons’. To create your own ButtonBar you just need to … wait for it .. extend Panel and make sure you have some specific markup in the page to indicate to wicket what element to attach its actions to.

package wicket.quickstart;

import wicket.extensions.wizard.CancelButton;

import wicket.extensions.wizard.FinishButton;

import wicket.extensions.wizard.NextButton;

import wicket.extensions.wizard.PreviousButton;

import wicket.extensions.wizard.Wizard;

import wicket.markup.html.panel.Panel;

public class ButtonBarPanel extends Panel {

public ButtonBarPanel(String id, Wizard wizard){

super(id);

add(new PreviousButton(“previous”, wizard));

add(new NextButton(“next”, wizard));

add(new YourLastButton(“last”, wizard));

add(new CancelButton(“cancel”, wizard));

add(new FinishButton(“finish”, wizard));

}

}

<wicket:panel>

<div>

<input wicket:id=“previous” type=“submit” value=“previous” />

<input wicket:id=“next” type=“submit” value=“next” />

<input wicket:id=“last” type=“submit” value=“last” />

<input wicket:id=“cancel” type=“submit” value=“cancel” />

<input wicket:id=“finish” type=“submit” value=“finish” />

</div>

</wicket:panel>

Easy! However if you look closely you will notice that One Of These Buttons is doing his own thing. We are attaching a button called ‘YourLastButton’ to the wicket element ‘last’. This is a custom button implementation as an example of what can be done. Note that you are not controlling the view of a button, just the controller. You’d do this if you wanted to add some customer behavior to a button or override one of the methods like isEnabled(). The most general case is to add an update workflow to the onClick() that updates the model somehow.

jfdpackage wicket.quickstart;
import wicket.extensions.wizard.IWizard;
import wicket.extensions.wizard.IWizardModel;
import wicket.extensions.wizard.WizardButton;
public class YourLastButton extends WizardButton
{
private static final long serialVersionUID = 1L;
public YourLastButton(String id, IWizard wizard)
{
super(id, wizard, “org.apache.wicket.extensions.wizard.last”);
}
public final boolean isEnabled()
{
return getWizardModel().isLastAvailable();
}
public final boolean isVisible()
{
return getWizardModel().isLastVisible();
}
public final void onClick()
{
IWizardModel wizardModel = getWizardModel();
wizardModel.getActiveStep().applyState();
wizardModel.lastStep();
}
}

package wicket.quickstart;

import wicket.extensions.wizard.IWizard;

import wicket.extensions.wizard.IWizardModel;

import wicket.extensions.wizard.WizardButton;

public class YourLastButton extends WizardButton {

private static final long serialVersionUID = 1L;

public YourLastButton(String id, IWizard wizard) {

super(id, wizard, “org.apache.wicket.extensions.wizard.last”);

}

public final boolean isEnabled() {

return getWizardModel().isLastAvailable();

}

public final boolean isVisible() {

return getWizardModel().isLastVisible();

}

public final void onClick() {

IWizardModel wizardModel = getWizardModel();

wizardModel.getActiveStep().applyState();

wizardModel.lastStep();

}

}

Summary

This is a basic introduction on how to implement your own Wizard markup in place of the mark-up that the wizard puts out. I had to get through this on a project because throwing the default table based layout into my pages was causing some serious issues. By creating my own extended Wizard classes I was able to totally customize the look/feel of the wizard and ensure pixel perfect design. In my case I needed to use NiceForms which I highly recommend. Particularly since you can now Customize Your Wicket Wizard..

You can do more with Wizards, much more. This tutorial is not meant to be comprehensive. Check out the Wizard API for a total rundown on what can be sub-classed. Take a look at the DynamicWizardStep and other classes, they really open the power of the Wizard. Also refer to the source for the base implementations. Open Source is an invaluable resource so Read The Source. The Wizard code is short and easy to understand since it uses the common Wicket constructs (panels etc). Don’t be intimidated.

Now, Go Forth And Wizardify.

]]> http://www.siberian.org/2009/06/24/customizing-wicket-wizard-markup/feed/ 0 Latest GSpace doesn't really require GMail.. (GMail as an anonymous File Relay) http://www.siberian.org/2006/10/09/latest-gspace-doesnt-really-require-gmail-gmail-as-an-anonymous-file-relay/ http://www.siberian.org/2006/10/09/latest-gspace-doesnt-really-require-gmail-gmail-as-an-anonymous-file-relay/#comments Mon, 09 Oct 2006 11:48:41 +0000 John http://www.siberian.org/2006/10/09/latest-gspace-doesnt-really-require-gmail-gmail-as-an-anonymous-file-relay/ This is probably very obvious but I was installing the latest version of Gmail Space today – 0.5.1 – ( a FireFox extension for accessing GMail as a filestore in the browser, get it here: http://www.rjonna.com/ext/gspace.php ) and came across some interesting and most likely unintended functionality.

You can use your own arbitrary email address as GSpace storage. This latest release (and possibly others, have not tested) seems to communicate with GMail in a way that relays the data via the GMail SMTP servers and back and forth to your personal server. It appears actually to proxy the login over to the specified server, without any valid gmail credentials..

Pretty simple stuff actually, when you configure it, instead of giving it your gmail address simply give it your personal email address and password. GSpace communicates with GMail which then relays the mail to your personal account. When retrieving it does the same and somehow pulls the content back over. Note that the connection to the 3rd party server comes from GMail, not the connecting client.

My GMail account stays empty (since I never gave it my account id) but my personal email account gets the files as per the screencaps. The connection to the 3rd part mailserver comes from GOOGLE and not from my desktop PC.

Last oddity, if I delete the file from GSpace it does NOT remove it from my server but it is unavailable in GSpace (it must be keeping an internal counter?). If I delete from my email it STILL shows in GSpace(It seems to be caching the files locally).

Some security implications of this would be things like large scale piracy. Someone sets up an SMTP server, distributes the file drop information (login info for an email account on the mail server) and that data is transferred anonymously via GMail with no audit trail. Google can’t look for it since the data is NOT ON THEIR SERVER but thousands of people around the world can easily access the pirated data and be anonymized at the Google GMail SMTP gateway.

Anyone know why Google would be relaying data like this? Seems ripe for abuse… I may be missing something obvious but it feels wrong to me.

Some photos and logfiles


Webmail Contents
This is the contents of my PERSONAL email account (webmail interface)



Thunderbird Contents
GSpace files in Thunderbird via IMAP




Webmail Contents
Capture of the config screen for GSpace, note that its my PERSONAL account login information.


GSpace Contents
Screen capture of the contents of GSpace.


Here is my mailserver log. Note that the connection is coming FROM Gmail, not from my office ISP. My personal identity is not compromised in this workflow from a Gmail/GSpace perspective, I am totally anonymous to the 3rd party mailserver(my own in this case).


# host 64.233.166.178
178.166.233.64.in-addr.arpa domain name pointer py-out-1112.google.com.

9 17:35:14.00:Info:218318848: Adding (X-IP-stats: Incoming Last 0, First 244, in=51, out=0, spam=0)
17:35:14.00:Info:218318848: Adding (X-External-IP: 64.233.166.178)
17:35:14.00:Info:218318848: check friends
17:35:14.00:Info:218318848: friend: ok
17:35:14.00:Info:218318848: process exceptions
17:35:14.00:Info:218318848: friend: processed
17:35:14.00:Info:218318848: process centipaid
17:35:14.00:Info:218318848: Rename
17:35:14.00:Info:218318848: Hashed maildir path is (xxxx)
17:35:14.00:Info:218318848: UTOTAL: NEW 0xf53b5d0 nref=1 n=0 (global=231)
17:35:14.00:Info:218318848: fld_lock: (INBOX) msg.c:6995
17:35:14.00:Info:218318848: fld_lock_user siberian.org#siberian#_INBOX
17:35:14.00:Info:218318848: fld_slib_unlock_user now (siberian.org#siberian#_INBOX)
17:35:14.00:Info:218318848: fld_unlock_user siberian.org#siberian#_INBOX
17:35:14.00:Info:218318848: slib_unlock n=0 f=0
17:35:14.00:Info:218318848: UTOTAL: FREE 0xf53b5d0 nref=0 n=0 (global=231)
17:35:14.00:Info:218318848: msg: [4629097] Stored: 64.233.166.178 <479ffcd60610091739l4f8a9000y74058311dd1d29f7@mail.gmail.com> "Stored locally"
17:35:14.00:Info:218318848: msg: deliver locals done 1 Deliver locals
17:35:14.00:Info:218318848: msg: dsn send (delivered)

]]> http://www.siberian.org/2006/10/09/latest-gspace-doesnt-really-require-gmail-gmail-as-an-anonymous-file-relay/feed/ 1 Matrix Switcher Control Appliance (In progress) http://www.siberian.org/2006/07/05/matrix-switcher-control-appliance-in-progress/ http://www.siberian.org/2006/07/05/matrix-switcher-control-appliance-in-progress/#comments Wed, 05 Jul 2006 19:18:45 +0000 John http://www.avdistro.com/2006/07/05/matrix-switcher-control-appliance-in-progress/ EDITING IN PROGRESS

STARTUPDATE (Sept 8.2006)
Some interesting things have happened.

1) A LIRC module was released for Ruby and this will let me accept the raw codes and get rid of the totalitarian lird.conf file! Hooray! Now I can allow people to, via a RIA app (Flex) remap arbitrary IR codes to switcher activities!
2) Ruby actually works on my device! Having trouble packaging it up to return to the SH4 community but I’ll work on that more later.
3) I learned how to build RIA apps in Adobe Flex.

This means some progress is being made, just slowly. Lots going on as I put the finishing touches on the huge home remodel I did All On My Own With Help From My Dad.

oh, and now I don’t have to code in C. All Hail Ruby (this month at least).

ENDUPDATE

As my 2 regular readers know, I am deep into my Whole House A/V Distribution installation. Overall the results have been fantastic but there is one missing component, its just not turnkey enough. If anyone else had to maintain it they’d rip it off the wall and plug all the RG-6 into a splitter from the local Cable Co.

Thats not the goal here, the goal is to build a Distribution system to stand the test of time, one that Ozymandias himself would be proud of. Wait. Bad example. So, the latest quest has been for a small appliance that can act as the switching system.

There are any number of these out there but they are all limited in one way or another, until now that is! JJPlus (http://www.linux.jjplus.com/) recently released this cute little box that has NAND RAM onboard, accepts USB storage external or internal and has multiple network ports AS WELL AS a serial port! That makes it perfect for my scheme. Should be easy enough right?

Not really, see, since I haven’t had hardware to work on I’ve been playing ‘product manager’ in my head and thinking up all sorts of crazy features. The goal is to make this thing usable when I am no longer the owner of it. Towards that, here is my current feature list:

Dynamic Configuration of IR codes

  • Accessed via a web-based wizard as well as a Java SWT app
  • Allows for configuration of switch (add/remove inputs/outputs) and assignment of IR codes via learning

PROBLEM – LIRC is great but its typical ‘Linux’ stuff meaning ‘the people who use it use it one way’, I need LIRC with an sql back-end and no .conf file. I don’t know much about it yet, it probably does this already since lircd can spit out raw codes on socket.

PROBLEM - JJPlus box is very low resource. Thats ok, I’ll write my CGI’s in C and use really lite XML.

Plug-in architecture for switching languages

  • Each switch manufacturer uses a different protocol, although they are all text based. We’ll need to be able to drop new protocols in whenever required.
  • XML communications to Java App

Obvious, buy why a Java app? See next point ZeroConf implementation to support Java app configuration – This is the key ingredient. Launch the java app, it finds the switch controller and your off to the races. So thats the list today. I think its a nice, doable, medium term project that I can get cracking on. I haven’t coded in C in -years- (who codes in C now? No one I know but then again I hang out with perl nerds and java snobs). It should be a blast and I’ve been wanting to build a -useful- embedded system for quite awhile. DISCLAIMER : I Know That There Are More C Coders Out There Then Sand On A Beach. It Was A Sarcastic Remark. Some of My Best Friends Code In C, regularly! From a user perspective its great, plug power, ethernet and null modem to the box, power it up, launch the java tool, add your inputs and outputs and for each combination assign IR controls. Viola! Truly distributed media switching that was dynamically configured by someone who could care less what an autopatch is, why its connected with a null modem connector and how levels work. PS – Why do I care that my HTPC handles it today? Well, my HTPC tends to CRASH or be REBOOTED BY MICROSOFT on a regular basis. Thats cool, they are protecting me from Evil, I know this but it makes things get screwy with Girder and the IR sub-systems when it happens to often. Its also lame to have this big 3ghz P4 system required to issue 6 character codes over the serial port. Its also just way cooler to have a little 1″x4″x8″ solid-state box doing all the magic, thats what my inner nerd tells me. Finally, I am on a quest. For the last 18 months everyone I know tells me I am crazy, that my system is to complicated (even though my wife can use it with ease) or that its to expensive. This is the final piece of the puzzle to complete the vision and prove, once and for all, that I have built the Budget A/V Switching System To End All Budget A/V Switching Systems. Thats not to hard, its the only one going. Turns out A/V Switching systems are not that popular..

]]> http://www.siberian.org/2006/07/05/matrix-switcher-control-appliance-in-progress/feed/ 0 Webmin Virtual Host Manager http://www.siberian.org/2006/07/05/webmin-virtual-host-manager/ http://www.siberian.org/2006/07/05/webmin-virtual-host-manager/#comments Wed, 05 Jul 2006 19:18:02 +0000 John http://www.avdistro.com/2006/07/05/webmin-virtual-host-manager/ **UPDATE**

Dead project, leaving this be for historical purposes. Webmin has perfectly good vhost capabilities now.

Update
Now requires the perl module YAML. Why? I moved to a new host and the Storable data was not byte compatible. YAML is a nice way to represent data structures in a portable, parsable and human readable way. It was a 4 line change and it made my life easier. It’ll make your easier as well. Ok, maybe not, but it works better this way, trust me:)

Download the 1.1 Version that requires YAML


Download the 1.0 Version that requires Storable


There are about 4,353.32 Virtual Host tools for webmin so of course I had to write my own! What does mine do differently?

  • It uses Apache mod_vhost_alias to simplify deployment and management of sites.
  • It automatically gives each domain a restricted FTP account using Pure FTP
  • It supports unlimited alias per virtual host.
  • Everything is managed via the Webmin interface

I use this to support ~100 virtual sites on a server and it has really freed up my time in a significant way. Mod_Vhost_Alias is highly recommended, its a nice way to have virtual sites created by simply creating a directory. When apache gets the request it checks for a directory using the hostname (www.crackcocaine.com for example). If it finds it then a vhost is dynamically created with that as the DocumentRoot.

Currently tested on Linux and FreeBSD with no issues, its just perl after all!

Read on for details

All you need to do to get this to work is :

  • Install Pure FTP and configure it for Virtual Users (follow Pure FTP docs)
  • Install Apache with mod_vhost_alias (follow Apache Docs)
  • Create a dynamic vhost that looks like this (change items in bold). Note, you can add whatever options you wish enabled for the Vhost in this config file. Things like server-side includes etc.
UseCanonicalName Off VirtualDocumentRoot /path/to/your/vhosts/%0
LogFormat "%V %h %l %u %t \"%r\" %>s %b \"%{Referer}i\"" vhost_common
CustomLog /path/to/your/weblogs/VHOST_ACCESS vhost_common
ErrorLog /path/to/your/weblogs/VHOST_ERROR   /path/to/your/vhosts/*>
AllowOverride All
Options All
  • Load the vhost.wbm into webmin. It will appear within ‘Others’, you can move it where you want it.
  • Configure the module by telling it
    • Where to store its virtual host database (text file writable by webserver)
    • What the webserver username is (for directory creation)
    • Where the PureFTP user file is (for ftp login updates)
    • The mod_vhost root from the apache config file (to create the new directories)
  • Voila! Thats it, create a virtualhost and try it out. You’ll need to map your DNS entries to the IP of your vhost of course :)

You can install it directly from webmin from from this link : Virtual Host Manager
Any questions? Drop me an email or make a comment. Let me know if you use it, I am always curious!

]]> http://www.siberian.org/2006/07/05/webmin-virtual-host-manager/feed/ 0