Application Platform Strategies Blog

News from OSBC

Blogger: Joe Niski

I’ve spent the last couple of days at the Open Source Business Conference, now in its seventh (or eighth) year. As far as I cold tell, the roughly 800 attendees were about evenly split between open source projects seeking funding, venture capitalists, and lawyers - but organizer Matt Asay said this year was notable for about 40% of attendees representing enterprise IT. This would indicate that enterprises are starting to get serious about understanding the implications of using free open source software (FOSS), and proactively managing it.

As a developer who’s used FOSS for over a decade and worked hard to get executive buy-in, I find this a very welcome indicator. As an industry analyst with a report on FOSS detection and governance tools due to publish in late April, I’m running to stand still regarding the evolution of FOSS business models and governance tools.

I learned of two governance resources that might not make it into my my upcoming report, both with heavy involvement from HP. The first is FOSSBazaar, an open community for sharing governance best practices. HP is a founding partner, along with Coverity, DLA Piper, Google, the Linux Foundation, Novell, Olliance Group, OpenLogic and SourceForge. HP has also released its internally-developed FOSSology detection and governance framework under the GPL (v2). Although there's something poetic about a FOSS solution for FOSS governance, as with all new projects, we'll need to wait and see how a community develops before forming an opinion on what FOSSology means for commercial vendors already in this solution space.

I also heard a few interesting data points. According to a survey of conference attendees conducted by North Bridge Venture Partners, there’s a strong belief that the current economic downturn will be good for FOSS in general – to quote a panelist, “the absence of money is good for innovation.” If anything, there’s some concern that a flood of poorly placed investments in FOSS-based startups could have a negative effect. I think this is good news for developers, users and enterprise IT.

SOA Expectations

Blogger: Anne Thomas Manes

Todd Biske recently responded to my post from March 9 (Looking for SOA success stories) with a valuable discourse on Setting SOA Expectations followed by another post yesterday on Organizing for SOA. I'm still noodling on the organization topic, so I'll save that for a later post, but I would like to respond to the comments Todd made regarding SOA expectations.

Todd said:

I’ve always believed that the changes associated with SOA adoption were a long term effort, at least 5 years or so, but it was very surprising to see Anne only find one success story, although further comments indicated she had only talked to 7 companies. 1 out of 7 certainly feels right to me. Things became even more interesting, though, with some comments on the post which indicated that Anne’s definition of success was, “Has the initiative delivered any of the benefits specified as the goals of the initiative?” She indicated that every company said that their goals were cost reduction and increased agility. My question is how did they measure this?

First I want to point out that our research methodology looks for patterns and trends rather than statistics. We've interviewed 12 companies so far, which was our minimum target. We expect to interview another half dozen or so. Obviously, this small sample size is not going to produce any kind of statistically relevant numbers. But we are identifying plenty of patterns. And that's our goal. We're identifying the issues that cause breakdowns. And the situation is remarkably similar at most of companies.

I agree with Todd that a SOA initiative is a long term effort. I typically put it at 20 years rather than 5 years. So I'm actually pretty stunned at the enormous success my one success story has achieved in just 4 years. And I'm not sure that it's appropriate to attribute all of its success to SOA. In addition to the SOA initiative, the company executed a very significant IT transformation effort that involved reorganization, a radically modified incentive program, and adoption of agile methodologies. All factors have contributed to its success.

Todd points out that "success" is relative. Perhaps. But every company that Chris and I have interviewed has told us that the goals of the SOA initiative are to reduce costs and increase agility. And our first question when they tell us this is always, "How do you measure agility?". Most companies that we've interviewed don't have a satisfactory answer. (Unspecified metrics is a key trend.)

I have absolutely no doubt, though, that my one success story has attained these goals: The business units that pay for new applications are eager to extol the virtues of the new IT group and its ability to deliver new systems quickly and at a fraction of the previous costs. In fact, the VP of sales said to the CIO, "Whoa, slow down. We can can't handle all these improvements to the CRM system."

I'm anxious to find other success stories and compare them with this one so that we can identify trends. One company provides an anecdote. I need at least three companies to be able to identify trends. I'm scheduled to interview someone tomorrow that claims to be successful. I'm hopeful.

But let me get back to the heart of Todd's post, where he says:

Rather than starting out with a goal of “cost reduction” or “increased agility”, start out with a goal of “create a service portfolio.” This sets the organization up for an appropriate milestone on the journey rather than exclusively focusing on the end state. Without interim, achievable milestones, that end state will simply remain as an ever-elusive pot of gold at the end of the rainbow.

I completely agree that definition of reasonable goals (as well as metrics for measuring success) is an important aspect of a SOA initiative. But I'm not convinced that "cost reduction" and "increased agility" are bad goals -- they just need to be set up as long term goals. I don't think "create a service portfolio" is an appropriate goal. (It's a milestone rather than a goal.) The SOA team won't be able to justify continued funding just by building a service portfolio. A goal needs to represent demonstrable value to the business (i.e, cost reduction, faster time-to-market, improved employee efficiency, etc.)

Nonetheless, the IT team needs milestones. It needs a roadmap with well defined deliverables to measure progress on its SOA journey. So "creating a service portfolio" is a good milestone. And based on the trends we've seen, "creating a service portfolio" is not an easy milestone to attain.

Joe B. on Data Principles

Blogger: Chris Haddad

 

Burton Group is fortunate to have Joe Bugajski on board as a Senior Analyst covering Data Access Strategies. Joe is a seasoned data architecture veteran who recently was vice president of Global Standards and Interoperability at Visa. Reporting to Visa’s Global CTO, Joe was responsible for worldwide interoperability of VisaNet data and applications and global service standards for transactions processing and payment data management. 

For his first Burton Group research report, Joe is writing about .NET 3.5 Support for Data Services: LINQ and ADO.NET. To put Microsoft's data technologies in context, Joe has created a data principle evaluation framework. The framework reviews capabilities in the following areas:

• Data Assets: Increase their value
• Enterprise Architecture: Support technology standards, goals, and plans
• Software Development: Promote code quality, reliability, simplicity, and security
• Development Management: Meet business objectives

While explaining the data principles framework would require an entire research report (hopefully one Joe will write soon), I have distilled out key evaluation criteria for this blog entry:

Data Assets: Increase their value

• Assure fidelity of data typing

• Preserve (business) semantics

• Understand reliability of data semantics

• Understand the quality of the data

• Map business semantics to actual semantics

• Minimize data movement

• Understand stored form of data and query logic

• Define information preserving data format conversions

• Calculate data access volumes

• Measure instantaneous rate of change of data values

• Measure total cycle time from access to persistence

• Understand persistence needed for disaster recovery

• Meet information steward’s requirements for data access

• Maintain principle of least access

• Secure private and sensitive data during access and use

EnterpriseArchitecture: Support technology standards, goals and plans

• Minimize code and architectural complexity

• Minimize total number of suppliers and contracts

• Minimize total number of active data formats

• Promote enterprise architecture plans

• Support multi-tier maintenance services

• Achieve compliance with internal technical standards

• Maintain semantic interoperability

• Preserve and enhance value of metadata stores

• Provide for data usage traceability and auditability

• Support nonfunctional requirements; e.g. security architecture

• Maximize continuity of architectural styles

• Maximize code and services reuse

• Maximize use of existing data stores

Software Development: Promote code quality, reliability, simplicity and security

• Minimize costs and time required to deliver code

• Minimize language complexity

• Minimize code complexity

• Minimize context switching (time lost in obscurities)

• Minimize coding errors

• Achieve closest match to functional requirements

• Meet nonfunctional requirements

• Promote code portability

• Promote reliable code walk-through

• Support frequent builds

• Support regression testing

• Optimize use of working prototypes

• Maximize unit testing

• Maximize code quality

• Maximize code readability

• Maximize ease of maintenance

Development Management: Meet business objectives

• Minimize number of coding styles across projects

• Support budget and time constraints

• Improve capability to size projects

• Promote growth of developer knowledge and skills

• Minimize personnel turnover

• Minimize requirements churn

• Avoid increasing scope

• Minimize the number of surprises

• Maximize developer efficiency and effectiveness

• Maximize ability to reassign personnel

• Achieve organizational objectives

How well do these criteria match your data technology evaluation and incubation process? I look forward to reading Joe's report detailing how LINQ, ADO.NET, and the Entity Framework rate against the framework.

How to improve IT Influence?

Blogger: Chris Haddad

 

After agreeing on technical best practices and best-of-breed infrastructure capabilities, many conversations turn circular when the focus shifts towards building a coalition for change. I am reading a book outlining "powerful influence strategies that can be combined to make change inevitable".  The book, 'Influencer', is a compelling transformation manual.  A full review can be found at the 'Business Book Review' website and on the RockStar blog. More information on the authors can be found at the VitalSmarts website and the Influencer blog. If you have great IT ideas and architecture plans, but can't influence change, the book is  highly relevant and a worthwhile read.

REST is about Resources

Blogger: Anne Thomas Manes

The folks at InfoQ have been doing a nice job putting together information about REST. SOA Lead Editor Stefan Tilkov posted A Brief Introduction to REST in December, which explains REST in simple terms and provides some basic examples of the model. For those looking to delve more deeply into the REST architectural style, I recommend reading Mark Baker's article, Hypermedia in RESTful applications. As Mark says at the start of the article, "Hypermedia as the engine of application state" (HATEOAS) is arguably the most important constraint in REST.

The most recent REST article posted on InfoQ is a follow-up article by Stefan, entitled Addressing Doubts About REST. He starts off by addressing the issue I hear most often from developers who inherently resist the notion of constraining an application to a uniform interface: "REST may be usable for CRUD but not for *real* business logic". Stefan explains how GET, PUT, POST, and DELETE can be used to accomplish a variety of functions that deliver more value than just create, read, update, and delete (CRUD) of data, and he shows an example of using GET to call a calculation function. But I don't think he effectively dispels this concern. And that's because he doesn't go the next step to talk about resource models.

REST is fundamentally about resources. Every "thing" that anyone might ever want to access or manipulate is a resource. And what that means, in practical terms, is that an application's interface is defined by the resources that it exposes (i.e., its resource model). Thinking in terms of resources rather than objects and methods requires a fundamental shift in application design. This is the biggest challenge that the typical OO developer is likely to have in adopting REST. If you're working on a RESTful application, and you find that you just can't support the functions you need without tunneling an RPC through POST, then you should go back and reassess your resource model.

Joe Niski is right now working on a revision of our report on Ruby on Rails, and one of the coolest features in Rails 2.0 is its inherent support for REST via the resource_controller plug-in. It automatically generates a resource model that exposes the application's data model as RESTful resources. See the InfoQ article by Rick DeNatale for more information, Rails: Resource_controller Plugin Put Controllers on a Diet.

Rails 2.0 is a really nice framework for building RESTful web applications, but the Rails sweet spot is focused on relatively simple applications based on relatively simple data models performing basic CRUD operations. If the application requires more complex processing, Rails is not such a good option. It requires a lot of complex coding to do complex things. So this brings us back to the big issue raised by Stefan: What to do if the application needs to do more than simple CRUD?

The answer is to use an abstraction layer. Keep in mind that a RESTful application's interface is defined by its resource model. Also keep in mind that the resource model does not need to map one-to-one to methods in the application's object model. And here's where many supposed "REST" frameworks tend to fall down. Most service frameworks invariably map the application's object model directly to its resource model. This approach often leads to non-RESTful POX services that violate RESTful principles. For example, a framework might map a setter method to a GET operation:

GET http://example.com/products/setPrice?item=widget&price=10.95

This is an extreme no-no. A GET operation should never update a resource. But how is a framework supposed to know whether a particular object method is "safe"?

That's why it's a good idea to start designing a RESTful application by defining its resource model. You need to ensure that you can expose all the required capabilities of the application through resources that support the uniform interface (GET, PUT, POST, and DELETE). From there, you can map the resource methods to code that implements the required capabilities.

 

XML and Modeling

Blogger: Kurt Cagle

Data modeling is a big thing at Burton Group - a significant amount of the airtime expended in the ether around the virtual water cooler is devoted to teasing out the way that models interact, the best language for expressing such models and what characteristics best define a good model. In a way this isn't surprising - many of the people within the organization are former application or systems architects, and as such have a common belief that nonetheless is one that application developers don't necessarily share: before you write a single line of code, you should have a reasonably deep understanding of what particular piece of the real world you are attempting to model in that code.

Modeling, for the record, is hard. It is in a very real sense an attempt to predict the future by trying to determine, for a give set of problems, what the final application that solves those problems will look like. What makes this particularly difficult in many settings is the fact that these predictions often have to be made before real intangibles - what development environments and languages the systems will be built in, whether people with the adequate skills to program the pieces can be found, whether new technologies will emerge that will eclipse what is currently under develop and so forth - can even be made.

HP/Systinet finally published GIF

Blogger: Anne Thomas Manes

Way back in April 2005, Systinet (now a part of HP) announced the Governance Interoperability Framework (GIF), which proposed to define standard formats and practices to enable interoperability among SOA infrastructure ecosystem products (service platforms, mediation systems, and management systems) via UDDI. At the time Systinet recruited about a dozen SOA infrastructure vendors to participate in a partnership  to implement support for GIF in their products. Most of these vendors implemented at least some support for GIF by mid 2006. But the program remained a proprietary partnership. Other registry vendors (e.g., Infravio and SOA Software) were not invited to participate in the program. I was very disappointed with the closed nature of the effort, and I railed repeatedly on Systinet to publish the specs in a public forum.

Now, nearly three years later, HP has finally published the Governance Interoperability Framework Reference specification. I've only given it a quick glance, but it appears to provide enough information to enable other UDDI-compliant registry vendors/users to configure their systems to act as a GIF hub. It also provides enough information for other infrastructure vendors to use GIF to exchange service metadata, policy, and management information.

The publication is not quite as open as I'd like -- you must register with HP and forfeit private information to gain access to the document. But at least it does not require acceptance of a license agreement or signing any type of partnership agreement. And the Copyright Notice is quite benign:

Copyright Notice
Copyright © 2008 Hewlett-Packard Development Company, L.P.
DISCLAIMER OF WARRANTEES. USER ACKNOWLEDGES THAT THE SPECIFICATION MAY HAVE
ERRORS OR DEFECTS AND IS PROVIDED "AS IS." HEWLETT-PACKARD MAKES NO EXPRESS OR
IMPLIED WARRANTIES OF ANY KIND WITH RESPECT TO THE SPECIFICATION, AND
SPECIFICALLY DISCLAIM THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, EVEN IF THAT PURPOSE IS KNOWN TO HEWLETT-PACKARD.
LIMITATION OF LIABILITY. HEWLET-PACKARD SHALL NOT BE RESPONSIBLE FOR ANY LOSS TO ANY THIRDS PARTIES CAUSED BY USING THE SPECIFICATION IN ANY MANNER
WHATSOEVER. HEWLETT-PACKARD SHALL NOT BE LIABLE FOR ANY DIRECT, INDIRECT,
SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER BASED ON CONTRACT,
TORT OR ANY OTHER LEGAL THEORY, ARISING OUT OF ANY USE OF THE SPECIFICATION OR ANY PERFORMANCE OF HEWLETT-PACKARD RELATED TO THIS SPECIFICATION. USER FURTHER ACKNOWLEDGES THAT THE SPECIFICATION IS PROVIDED FOR EVALUATION PURPOSES ONLY, AND USER ASSUMES ALL RISKS ASSOCIATED WITH ITS USE.

GIF defines a model that supports information exchange based on UDDI (v2 or v3, but v3 is recommended to enable subscription capabilities), WS-PolicyAttachment, and WS-Management. It supports information exchange related to service management, policies (constraints, configuration, and capabilities), dependency relationships, consumers and providers, property information, and management metrics. All-in-all, quite a valuable document.

Impressions from GITA 2008

Physical infrastructure - roads, cities, bridges, sewer and power systems, that sort of thing - is not exactly at the top of the list of most people's ideas of exciting subjects, but they are nonetheless as necessary a part of daily life as food production and transport, oil production and laying of network cable. The GITA 2008 Conference - also known as Geospatial Infrastructure Solutions - has showcased that in very stark detail this year. Seattle once again showed off its reputation as one of the rainiest cities in the country, but the dark clouds and gloom outside served only to highlight the dark clouds and gloom on the inside.

XML - my bailiwick - is mostly about systems and information flow, and over the years I've come to develop a particular sense about systems theory in general, especially when it touches on IT. Yet the people here are system thinker's system thinkers, the ones that not only play SimCity but offered advice to its game designers, and frankly, they're very nervous right now.

Looking for SOA success stories

Blogger: Anne Thomas Manes

My friends over at ZapThink recently posted an interesting ZapFlash newsletter that has sparked another heated debate (i.e., permathread) on the Yahoo! Groups service-orientated-architecture discussion list.

Ron Schmelzer's missive last month was entitled "Why Service Consumers and Service Providers Should Never Directly Communicate". The article is directed specifically at "integration-centric techies" and (once again) attempts to explain the fundamental difference between SOA and integration. (Ron, Jason, and I have been beating this drum for a long time.) As Ron says, "to correct your misunderstanding, I need to speak your language: the language of integration technology."

Ron characterizes the typical integration-centric approach to SOA as EAI 2.0. It's a better form of EAI than in the past, but it's still fundamentally focused on integrating application silos (EAI) rather than dismantling the silos (SOA). Ron also does a nice job positioning the role of an ESB in EAI 2.0 vs SOA.

Nonetheless, I come away from this article with a sense that the technology discussion is irrelevant. (The article got a similar response from the SOA discussion list, also. I especially liked Stu Charton's initial response where he pigeon-holed the discussion as one related to "technical services architecture" and then went on to say, "Technical services architecture helps enable business agility to the same degree that Enterprise JavaBeans did.")

Don't get me wrong -- I concur with Ron's recommendation to use virtualized proxies and dynamic bindings between service consumers and service providers. (I don't recommend using a registry at invocation time every time to locate the proxy, though -- that should be a fall-back mechanism to be used only when you can no longer find the proxy.)

But Ron implies that the only way to achieve SOA is to use an advanced WS-* infrastructure that relies on WS-Addressing. As far as I'm concerned, a SOA infrastructure needs to be middleware-independent, not just vendor-independent. The virtualized proxy and dynamic binding mechanisms should work with any middleware -- not just with WS-*.

But let me get back to my point -- this technology discussion is irrelevant. As Chris Haddad mentioned in this blog last month, we're conducting an extensive research project using a methodology we developed based on the Contextual Design process. And I think I've become a bit jaded from the interviews I've conducted thus far. It has become clear to me that SOA is not working in most organizations.

I've talked to many companies that have implemented stunningly beautiful SOA infrastructures that support managed communications using virtualized proxies and dynamic bindings. They've deployed the best technology the industry has to offer -- including registries, repositories, SOA management, XML gateways, and even the occasional ESB. Many have set up knowledge bases, best practices, guidance frameworks, and governance processes. And yet these SOA initiatives invariably stall out. The techies just can't sell SOA to the business. They have yet to demonstrate how all this infrastructure yields any business value.

More to the point, the techies have not been able to explain to the business units why they should adopt a better attitude about sharing and collaboration--which is the fundamental cultural shift required for SOA to succeed. The pervasive attitude is "What's in it for me?" As one of my interviewees said, "Altruism is not an enterprise strategy".

Thus far I have interviewed only one company that I would classify as a SOA success story. This company reorganized IT around functional capabilities (rather than business units) and established strong positive and negative incentives that encourage people to adopt a better attitude toward sharing. I'm beginning to think that this is the only path to SOA success.

I'd really like to find other success stories. If you have a successful story to tell, please contact Chris or me.

 

Understanding the Benefits of XForms

Blogger: Kurt Cagle

XForms is something of an odd duck. Originally intended simply as a modularization of the HTML components so that they could better work in a more XML oriented environment, the XForms specification very quickly morphed into the foundation for a considerably more sophisticated application, albeit one that had a few ... idiosyncrasies.

For instance, unlike most other applications where the data model was bound into objects and object models, XForms defined its data models as blocks of XML, and the controls, rather than individually holding values that the application could set or retrieve via script, actually were bound to the data model - change a value in the model, and the control changed, change a control value, and the underlying model automatically reflected that change. Throw in bindings using XPath expressions, a purely declarative structure and working with models that could be loaded from the server or submitted to the server without forcing a reload of resources, and you could perhaps be forgiven for thinking that XForms was a language designed by committee (which, to be honest, is pretty much what happened).

W3C Standards are themselves a little odd, with the XML standards being particularly notable for this. In most cases, standards represent a consensus view about what the existing state of the industry looks like, only with the rough edges of controversy smoothed away in smoky, back-room deals (the truism of politics and sausage also applies to the creation of standards - one should never look too closely into how any of them are made). The W3C XML standards, however, involved a technology that itself had not existed until the W3C formally established it, though it did of course have the twenty five year old precedent of SGML to fall back on.