Looking for SOA success stories

Blogger: Anne Thomas Manes

My friends over at ZapThink recently posted an interesting ZapFlash newsletter that has sparked another heated debate (i.e., permathread) on the Yahoo! Groups service-orientated-architecture discussion list.

Ron Schmelzer's missive last month was entitled "Why Service Consumers and Service Providers Should Never Directly Communicate". The article is directed specifically at "integration-centric techies" and (once again) attempts to explain the fundamental difference between SOA and integration. (Ron, Jason, and I have been beating this drum for a long time.) As Ron says, "to correct your misunderstanding, I need to speak your language: the language of integration technology."

Ron characterizes the typical integration-centric approach to SOA as EAI 2.0. It's a better form of EAI than in the past, but it's still fundamentally focused on integrating application silos (EAI) rather than dismantling the silos (SOA). Ron also does a nice job positioning the role of an ESB in EAI 2.0 vs SOA.

Nonetheless, I come away from this article with a sense that the technology discussion is irrelevant. (The article got a similar response from the SOA discussion list, also. I especially liked Stu Charton's initial response where he pigeon-holed the discussion as one related to "technical services architecture" and then went on to say, "Technical services architecture helps enable business agility to the same degree that Enterprise JavaBeans did.")

Don't get me wrong -- I concur with Ron's recommendation to use virtualized proxies and dynamic bindings between service consumers and service providers. (I don't recommend using a registry at invocation time every time to locate the proxy, though -- that should be a fall-back mechanism to be used only when you can no longer find the proxy.)

But Ron implies that the only way to achieve SOA is to use an advanced WS-* infrastructure that relies on WS-Addressing. As far as I'm concerned, a SOA infrastructure needs to be middleware-independent, not just vendor-independent. The virtualized proxy and dynamic binding mechanisms should work with any middleware -- not just with WS-*.

But let me get back to my point -- this technology discussion is irrelevant. As Chris Haddad mentioned in this blog last month, we're conducting an extensive research project using a methodology we developed based on the Contextual Design process. And I think I've become a bit jaded from the interviews I've conducted thus far. It has become clear to me that SOA is not working in most organizations.

I've talked to many companies that have implemented stunningly beautiful SOA infrastructures that support managed communications using virtualized proxies and dynamic bindings. They've deployed the best technology the industry has to offer -- including registries, repositories, SOA management, XML gateways, and even the occasional ESB. Many have set up knowledge bases, best practices, guidance frameworks, and governance processes. And yet these SOA initiatives invariably stall out. The techies just can't sell SOA to the business. They have yet to demonstrate how all this infrastructure yields any business value.

More to the point, the techies have not been able to explain to the business units why they should adopt a better attitude about sharing and collaboration--which is the fundamental cultural shift required for SOA to succeed. The pervasive attitude is "What's in it for me?" As one of my interviewees said, "Altruism is not an enterprise strategy".

Thus far I have interviewed only one company that I would classify as a SOA success story. This company reorganized IT around functional capabilities (rather than business units) and established strong positive and negative incentives that encourage people to adopt a better attitude toward sharing. I'm beginning to think that this is the only path to SOA success.

I'd really like to find other success stories. If you have a successful story to tell, please contact Chris or me.

 

Comments

[Duane]

I think your comment of only one successful SOA raised some eyebrows, though I suspect it may reflect on what you believe an SOA is and is not.
The reason way I see it, SOA is about a higher level of abstraction, where I can engage business functionality at a level where little is known about the actual implementation. Therefore, it behooves the business to specify what capabilities make sense to be shared as a service. That is, the same functionality is needed across the organization and if we abstract the fundamental elements that would make it useful, then it can be effectively shared (reused).

Consequently, I suspect it is somewhat difficult (based on my experience) to get people to think in terms of that level of abstraction. There seems to be a tendancy to drive it down to system functionality that is "wrappered" so that I don't know the implementation details, but I know that at end the customer database is updated. Though this is powerful, it is basically the Object-Oriented layer of separating the interface from the implementation and should be the modis operandi of could system design, but does not raise itself to the level of being an SOA abstraction.

So, I suspect that you are right, there may be lots of people doing things they call SOA and I'm sure they are doing things more service-based than they did before, but it terms of organizing around an SOA approach to architecture / the business, there are probably few success stories. I do hope you hear of more than one.

D.M.

[Anne Thomas Manes]

Duane -- my measurement of "successful SOA" is simple: Has the initiative delivered any of the benefits specified as the goals of the initiative? Every company I have interviewed has said that the goals of the initiative are to reduce costs and increase agility. Only one company of the seven that I have interviewed has achieved these goals.

[Duane Meade]

Anne,

I agree with the goal to reduce costs and increase agility. What I guess I was trying to say is if you don't raise the level of abstraction where "use" of the services can be very flexible because of the stability of the interface and also reduce costs because you have decreased the coupling (and thus dependencies between services), then it is likely you will not achieve the benefits of SOA. I can't decide that all band music doesn't sound good, if my only experience is with the 5th grade band at my kid's school.

[Steve Rdzak]

I think most businesses and the IT that support them do not have a good understanding of the conceptual model of their business. Many can't decouple their thinking from the application and database models used in their daily work.

To get to successful SOA you have to get both the business and the IT that support them to buy into delivering IT capabilities matching the conceptual model of the business.

But getting the two camps to collaborate on what that model is and governing projects to deliver shared capabilities aligned to that model is the biggest hurdle to overcome in most businesses today.

I think the value curve for SOA is going to be much slower to be realized than it was for Object-Orientation because of the fact that it really is way of thinking that is foreign to both business folk and technical folk.

S.R.

[Mike Kavis]

The problem is many people don't really understand SOA. The problem with SOA is not SOA, it is people. I wrote a short post on this topic.

http://blogs.ittoolbox.com/eai/madgreek/archives/why-doesnt-anyone-understand-soa-23102

[Anne Thomas Manes]

Mike,

I appreciate your comments, but I don't think that BPM is the "killer app" that will save SOA. In your post you describe "SOA" as a means to enable integration with legacy systems. Certainly legacy encapsulation can be a great enabler for BPM, but it doesn't make for an effective SOA strategy.

A successful SOA initiative focuses on reducing complexity, refactoring systems into services that implement core capabilities, and eliminating redundancy. It focuses on defining common patterns that apply to all business units and setting up software factories that enable the delivery of new products in a fraction of the time it used to take.

The goal of a SOA initiative should be to get the business units to look at the app dev group as an enabler, not a hinderance. To reach the point where the business says to the app dev group, "slow down -- you're giving me too much, too fast".

BPM is a valuable initiative in its own right, and organizations that fully embraced BPM typically realize a lot of value. But, as you say, BPM requires that you have services to assemble.

The reason why most SOA initiatives fail is that they can't reach the point where they have a portfolio of services. And that's because the business units aren't willing to sponsor the creation of services.

Catch 22.

Anne

[Eric Roch]

SOA is too much in the hands of the techies in most companies (I am one of course). I am afraid the business is about to pull the plug on a lot of these SOA technical exercises.

http://blogs.ittoolbox.com/eai/business/archives/soa-business-value-23123

[Mike Kavis]

I agree with what you said which is why I say BPM is the killer app that makes the business care about SOA.

As I stated in this article http://blogs.ittoolbox.com/eai/madgreek/archives/selling-soa-a-true-story-18349
if you align SOA with a key business initiative you can get the support (and funds) that you need to make SOA a reality. The business doesn't understand SOA, nor should they. But they do understand BPM and we sold them on the fact that SOA allows the business to receive the value of BPM without the expense of IT having to blow up the legacy systems (which would take years). Our business understands SOA as a tool to extend the life of our legacy systems (legacy rejuvenation as Zapthink calls it) and BPM as the tool that eliminates waste, improves employee morale and customer satisfaction, and increases revenue.

Without linking SOA to BPM, we never would have been able to justify the expense of SOA. I know, because I had been trying for years.

[Mike Kavis]

Hopefully this link explains my point better.

http://blogs.ittoolbox.com/eai/madgreek/archives/why-cant-i-sell-soa-to-the-business-23135

[pawel lubczonok]

SOA is not a technology - it is abstraction and architecture. Also, I believe that SOA is overhyped and it will be superceeded by another type of technology. The current approach is to package old systems to participate in services bus. This enables archic systems that should be scrapped to survive. Of course, this means that existing groups behind these systems preserve their jobs and power as well as consultants earn from 'helping to implementing SOA' - another vested interest. So, in a way SOA can cause retardation of progress. It is just too difficult to get SOA to work as you have to get so many pieces (and people) to be organised and connected (all have to agree) that complexity is too great and benefits will come too late to be accepted. Tools and approaches to get SOA right are also inadequate.

The development of systems and architectures based on semantics/knowledge are the future. One can see now the beginings of interest in semantics (again!) - semantics web, ontologies etc. It is true that these are still in very early stages of evolution. However, in systems that are based on semantics, SOA should just happens behind the scenes just like various protocols happen behind email system. Once one operates on a knowledge level rather than information level a lot of things can become different and most importantly simpler. Of course, one must agree with the SOA objective to make things work together.

Pawel Lubczonok

[James West]

I have been and am continuing to promote SOA, to business and IT, as a way to get everyone on the same page process-wise and identify common services across business units. As Enterprise Architect with 25-years in the software development experience, that's valuable regardless of technology.

From a business standpoint, there is some buy-in (head nodding) and perceived value but most, unfortunately, are either overwhelmed with their own duties and/or think SOA is just another technology initiative that will soon be replaced with SOB.

Realize the that IT industry is its our worst enemy. Only months after a new strategy/technology is promoted, something else is promoted as superior. In spite of all the effort towards "open", "standard", "inheritance", the ego and profit of the creator still dominates the software landscape.

The majority of Corporate America's budgets, systems and staff simply can't react effectively to these ever-changing ideologies not knowing which has sustaining technology, fit and support. As such, most action is the action of "wait and see".