Understanding the Benefits of XForms

Blogger: Kurt Cagle

XForms is something of an odd duck. Originally intended simply as a modularization of the HTML components so that they could better work in a more XML oriented environment, the XForms specification very quickly morphed into the foundation for a considerably more sophisticated application, albeit one that had a few ... idiosyncrasies.

For instance, unlike most other applications where the data model was bound into objects and object models, XForms defined its data models as blocks of XML, and the controls, rather than individually holding values that the application could set or retrieve via script, actually were bound to the data model - change a value in the model, and the control changed, change a control value, and the underlying model automatically reflected that change. Throw in bindings using XPath expressions, a purely declarative structure and working with models that could be loaded from the server or submitted to the server without forcing a reload of resources, and you could perhaps be forgiven for thinking that XForms was a language designed by committee (which, to be honest, is pretty much what happened).

W3C Standards are themselves a little odd, with the XML standards being particularly notable for this. In most cases, standards represent a consensus view about what the existing state of the industry looks like, only with the rough edges of controversy smoothed away in smoky, back-room deals (the truism of politics and sausage also applies to the creation of standards - one should never look too closely into how any of them are made). The W3C XML standards, however, involved a technology that itself had not existed until the W3C formally established it, though it did of course have the twenty five year old precedent of SGML to fall back on.

What emerged after XML, on the other hand, has bordered on the surreal. XSLT took a template matching approach to transformations and XML processing that was powerful but hardly intuitive (especially if you tended to be dubious about the power of recursion). XPath provided an odd notation for referencing the various parts of a given XML structure, while the recent completion of XQuery did the same thing for whole collections of XML documents. Even XML domain formats like SVG got into the act, representing a radical break in the depiction of vector graphics that seemed completely at odds with the instruction oriented approach that predated postscript.

All of these technologies (as well as XForms) represented the rethinking of computational tasks in light of a semantically neutral meta-language and a declarative approach to programming.  Not surprisingly, the tools that emerged out of this bear remarkably little resemblance to more procedural imperative tools. Similarly, the XML community has always been somewhat smaller and ... well ... quirkier than its imperative counterparts in other languages, so it is perhaps not all that surprising that XForms has gained relatively short shrift compared to AJAX, Ruby or other web technologies.

However, there are a number of signs that XForms is beginning to establish itself as a legitimate tool for businesses and developers:

• The XForms specification has gone through both a 1.0 release, which provided the first generation of XForms tools a stable standard on which to build, and a 1.1 working draft that represents some significant rethinking of 1.0 problem spots is now relatively stable and will likely become a formal recommendation this year.
• XForms implementations exist for Mozilla Firefox as a plugin that's been built in conjunction with the primary build, and this implementation now supports most of the 1.0 specification and several important pieces of the 1.1 specification. Third party support for an Internet Explorer is provided by FormsPlayer.
• Server-based XForms implementations by Orbeon and Chiba provide ways of hosting XForms on any platform that supports basic AJAX capabilities, with both increasingly appearing as bundled clients with a variety of XML Database servers ... and both boasting a significant customer base.
• XForms is now appearing as an embedded systems on mobile devices from companies such as PicoForms and Yahoo, and is also appearing as the embedded forms engine for the OpenOffice.org suite.
• The combination of XForms, REST and XQuery (a collection increasingly known collectively as XRX) is gaining prominence as XQuery itself becomes more widely accepted and more XQuery implementations appear on the market.
• Finally, as organizations move towards the creation of industry vertical XML taxonomies, the need for sophisticated and versatile tools for rendering this (highly malleable) content is rising. XForms is ideally suited for these types of applications.

All of these factors indicate that XForms as a technology is beginning to become competitive, especially for those applications dealing with large, complex data structures. However, while this may show the viability of such applications, it doesn't necessarily indicate why XForms as a technology should be interesting to you or your organization. To better understand that, it's worth appreciating that a radical shift has been taking place quietly in the web application space for some time, a shift best embodied by the acronym MVC: Model/View/Controller.

Models have been at the heart of application development for some time, but typically the models involved are conceptual in nature, and tend to revolve primarily around the objects that are interact within the application and their correlated processes.

For instance, consider the typical insurance application, which in most cases actually tends to be a number of related applications. One part of this application is the sale of insurance policies, each of which usually has a number of potential sub-parts with fairly significant data acquisition requirements per policy or sub-policy. A second part of the application is the submission of a claim against such a policy, where a number of different people interact with the system: an agent records the initial claims from a customer (often going from screen to screen to handle whether the claim is against a small vehicle, a large vehicle, a house, or some other insured entity), a claims adjuster who compares the initial claim against the actual damages found, and consequently makes annotations or changes content and makes recommendations, and a claims manager, who either approves or disproves the claim, typically on the basis of external conditions, legal precedent and customer prior history.  All of these people may in fact be utilizing the same basic record, but what they do with it differs dramatically; not surprisingly, such applications, for all that they may seem perfect for use on the web, often tend to be based on standalone apps because the complexities of setting up a website to manage this is just more work than its worth.

As it turns out, however, all of the actors above are in fact working on the same basic data model that in turn can be broken down into a few (fairly complex) collections of entities - a collection of policies, a collection of claims, and a collection or rules for determining if the claims will be paid out (there are also a couple of additional actor collections which determine permission action for various users, but the role that these play is comparatively minor and can be filled in as appropriate). Thus, in terms of a model, the primary objects defined in this case are policies, claims, and rules.

One of the central concepts of data modeling is that what you are attempting with a model to determine the minimal necessary set of data that needs to be maintained in order for an application to work. The configuration of this data can be called the state of the model at any given point in time, or an instance of the model. In the XML space, this condition can be restated by saying that a given model can be expressed as a set of XML structures, each of which holds some critical aspect of a model (for computational sake, there are times that it is easier to decompose certain portions of a model as separate XML instances than it is to maintain all of the state in a central document).

One implication of this, however, is that if the instances do in fact accurately represent the state of the model at any given time, then it is possible to conceive of a layer (called a controller) that can map the model to some distinct presentation or view. One such view may show a report providing the details of the claim in a read-only fashion. Another view may provide an editor that lets you make changes to the model (and that render changes in the model as some change in the user interface). If you indicate that a claim is being made for a boat, for instance, then only those parts of the claim editor that deal with  marine vehicles are deemed relevant and displayed, while those dealing with houses or cars are hidden from the user.

In this example, relevancy is a significant aspect of a model that can be incorporated as a specific property in the model through the use of some form of constraint ... which is in fact exactly what the controller is supposed to control. Thus the role of the controller is to mediate constraints on the model, while the role of the view is to provide an interface between the model and the producer or consumer of instances of that model.

The Model/View/Controller (MVC) paradigm has been around for quite some time, but in general has been found more in the breech than in the observation. Most contemporary applications work upon a component basis where each component maintains its own (largely inaccessible) model, and any attempt to build a more cohesive large-scale model from this usually involved pushing the model into abstraction. The difficulty with abstract models is that they are extremely hard to enforce, especially when the controller portions are defined weakly if at all.

However, for a number of reasons, XML based web applications lend themselves strongly to MVC type applications. XML, by its very nature, makes it possible to establish a virtual object model without having to specifically the precise implementation of each subordinate object in the model. It is property based, rather than process based; this makes it easy to separate out the concerns between model and controller. XML has no intrinsic semantics (nor even an explicit requirement upon schema), meaning that it becomes the right and privilege of the application designer to determine the bindings on a given element in the model, not the framework's. Finally, because XML can be readily streamed over network connections, you can create complex distributed models at comparatively little impact (and in most cases, no impact) to the functioning of the application.

This is precisely what XForms does. XForms is an XML namespace layer that is meant to be bound into other namespaces - XHTML, SVG, Open Office, DocBook or DITA, proprietary namespaces, it really doesn't matter. An XForms application thus usually defines a model with one or more instances in it that hold sway within the scope of the particular page, graphic, or related context, all defined by the use of either local XML documents or imported XML streams. The application in question may also define constraints within that model called bindings that determine the relevancy, validity, value, or accessibility (read/write state) of given nodes within that model).

Once these are defined, the view defined by the XForms application in turn consists of various input (and output) components, each of which links via XPath to one or more nodes in the model. Sometimes these linkages are simple - a given selection box could be linked to the type of claim (auto, house, marine, etc.), such that by changing the value in that box you also change the type of claim. Other linkages are more complex - a repeat element will link to a set of elements which represents all boats that are covered by the claim, where the individual properties of each boat represent simple links. Perhaps most important, if a constraint in the model forces a change in the type of claim, then the select box will automatically (without user input) also change its value to represent this new state.

Put another way, these controls are bound to the model, which is bound to the control. The user seldom if ever needs to write any imperative code to accomplish these changes. More significantly, while the data model instance can be changed to reflect the changes introduced from this view, at any given time, the model itself is internally consistent. Indeed, it is even possible within XForms to keep an application from submitting an XForms model to a server (or persisting it locally) if the model instance in question doesn't have all of the right information (such as someone making a claim without adding the specific  thing claimed damaged or stolen).

This kind of design approach has some huge implications:

• Especially by controlling relevance, you can replace a large number of "screens" in a traditional server-based application with a single XHTML+XForms document that shows only the parts of the user interface that are relevant at any given time in the entry process.
• You can prevent model instances with incomplete state from being persisted to the server, or you could use an XForms interface to clearly differentiate between draft documents which aren't valid and processed documents that are.
• A given XHTML+XForms document can be broken down into distinct sub-pieces that are then compiled together as a single document when needed. This means that it becomes far easier to change portions of a document without having to replace the entire application.
• You can utilize XForms implementations in most browsers, so that changing your applications generally does not require extensive downloads and upgrades of the XForms-based application.
• XForms based applications work especially well in queued message environments: multiple agents can send new claims to the server where they are submitted in a message queue and can consequently be processed independent of the agent's own editing processes. Submissions that require clarification or are otherwise provisional rejected for re-editing would then be sent back to the user via notification systems (such as atom feeds).
• In most XForms implementations, it is possible to customize the look and feel of a given control, sometimes dramatically. For instance, in the insurance claim for an auto collision, it may be possible to define different types of collision intersections as graphics, then record the path of both vehicles as a set of lines from a graphic drawing interface that gets translated into a set of coordinates in a given field. The underlying XForms model and controller remains simple, but the components in the view can be made as sophisticated as necessary.

The principle drawbacks that XForms has at this stage are generally those that any fairly new technology has, and as such need to be considered in that light when evaluating XForms. Specifically,

• the developer community is small, albeit is growing at a fairly dramatic pace. This will likely continue as a number of XForms pilot projects in various industries come to fruition, and the demand for XForms developers continues to gain steam,
• because of the close (and obvious) coupling between XForms and XQuery, as more systems implement robust XQuery based systems, the architectures for supporting XForms on the server will also increase. XQuery implementations have risen dramatically in the last year after the publication of the W3C XQuery recommendation (significantly, this same coupling also has meant that every XQuery developer has in turn either also also become or worked in conjunction with an XForms developer),
• in general, the XForms technologies are currently produced by tier 2 or tier 3 companies, or exist as open source projects of the same magnitude. Given the synergies between XQuery and XForms, it is likely that many of these companies or projects will be absorbed by one or more tier 1 companies over time,
• XForms 1.0 has some issues that have limited the complexity of the applications built against it. XForms 1.1, now in Candidate Recommendation status, resolves most of these issues, while still remaining backwards compatible with 1.0, and it will likely become a full Recommendation by June 2008. Discussions for an XForms 2.0, which incorporates many of the innovations brought with XPath 2.0 and XQuery 1.0, are currently underway, and will likely convene as a full working group upon publication of 1.1.

What this means is that the technology has both reached a point of stability and has developed a set of complementary technologies that are also stable enough to make XForms a good tool for exploration via pilot projects. It is debatable whether the technology is stable enough for the full deployment of enterprise applications at the moment, though the trends indicate that this will be case by the end of 2008 or mid-2009 at the outside.

XForms represents one area where Burton Group recognizes the potential for significant growth in both market presence and technological adoption. It is no panacea (though few technologies are, even those that may be advertised as such) but especially in increasingly sophisticated XML data applications it is a logical, compelling, and attractive alternative to complex applications that become bogged down because of server application complexity.

Comments

[Dan McCreary]

Kurt,

This was an excellent article. I agree that the shift to MVC in the browser (not just on the server) is a fundamental paradigm shift in application architecture. I have been using XForms and XQuery for over a year and I have seen huge productivity gains by using the same XML model in the client and the server. I think the XRX application stack has a excellent future because of its elegent simplicity. Something that the Ruby community has taught us all. And as you point out, the MVC architecture is really central to this stack.

- Dan McCreary