Application Platform Strategies Blog

SOA is about Architecture

Blogger: Anne Thomas Manes

Thanks to Jeff Schneider for pointing this article out to me.

It's been more than 7 months since I published the "SOA is Dead" post, but responses are still coming in. Most recently (yesterday), Dan Woods published an article on Forbes.com entitled "Reviving SOA". I'm pleased to see that Dan read beyond the first paragraph, and he understands the core message of my post (i.e., "SOA has been disappointing and that services should be a key focus"), he nonetheless fell into a common trap of misinterpretation, as I described in my follow-up post, "SOA Obituary: Misinterpretations and Perceptive Enrichment."

Dan falls into the first and largest camp, "Big SOA is dead, but little SOA will survive." He also has a touch of Camp 2, "REST will fix everything." He claims that "REST" is now more popular than SOAP. By "REST" I can only assume that he means plain old XML (POX) over HTTP, because I can assure you that the REST architectural style is much less understood than SOA. And a recent survey of Burton Group clients (mostly F500 companies) clearly indicates that REST is not yet a popular topic among large enterprises -- as much as I try to promote it.

Dan recommends an incremental approach to SOA: Just build services as you need them on a project-by-project basis, and at some point in the future you can go back and consolidate the services you've built and somehow derive some architectural consistency. Unfortunately, this strategy invariably leads to Just a Bunch of Web Services (JABOWS) because the future consolidation step almost never happens. The result is too many services, too many moving parts, and too many brittle connections. Systems wind up being more expensive and more fragile than ever before. #FAIL

I've only encountered one company to date that has succeeding using this method -- and I must point out that they maintained strong architectural governance of all services as they were built.

In particular, Dan misinterpreted my comment, "If you want spectacular gains, then you need to make a spectacular commitment to change." He took this as a recommendation, when it was really intended as a warning.

One of the biggest reasons that SOA "died" is due to unrealistic expectations. The hype has suggested that all you had to do was deploy an ESB and use web services and miraculously you would gain reduced costs and increased agility. The truth, though, is that architectural improvement does not come for free. It requires a concerted effort to improve the application architecture. If you aren't willing to make that concerted effort, you won't gain spectacular results. 

The Bottom Line:

The highly-touted benefits of SOA derive from architectural improvement, not from any particular technology. But architecture is hard. And it requires a fair amount of organizational maturity to accomplish. Organizational maturity is a function of five organizational dynamics:

• Governance: Establishing policies, guidelines, and decision-making authority
• Management: Enforcing policies and directing day-to-day operations
• Leadership: Inspiring people to perform
• Skills: Ensuring people know how to do what they need to do
• Practices: Establishing structure that helps people do what they need to do

Attempting to do SOA without also attempting to strengthen these five organizational pillars will invariably result in failure. Hence my warning: If you want a spectacularly successful SOA initiative, you must first address these organizational and cultural issues. Strengthening these organizational dynamics invariably requires a spectacular commitment to change.

VMWare fuels SpringSource's stratopheric journey

For the last year, the SpringSource team has been intently focused on bridging the gap between development and operations. The team has also been moving quickly to re-tool their platform to become a Cloud-native. Cloud-native platforms are built to exhibit cloud computing characteristics (i.e. self-service, elastic and scalable, virtualized, and dynamic) compared to the 'earthborn migrant' runtime stacks which are simply traditional application platforms hosted on cloud infrastructure without application container and framework modifications.

Cloud-native platforms enable users (including non-developers) to customize pre-built templates. They offer abstractions for workload granularity that lighten the developer’s burden for many scalability and operational concerns. While Cloud-native platforms will attempt to transparently address scalability and elastic provisioning/de-provisioning, the platform will assume that an application is already horizontally scalable, or can be redesigned to conform to Cloud application architecture restrictions that facilitate a scalable solution. Moving to Platform as a Service forces architects to fundamentally re-examine their application architectures.  Tackling parallelism and single points of bottleneck (usually database access) will become paramount.

With a new programming model, developers can approach solving problems that are resistant to traditional application architecture practices.  Democratizing the process of building massively scalable web applications and delivering high-quality applications in a superfast time to market are two examples that are becoming mainstream concerns. SpringSource, a framework innovator, is well-positioned to address the programming model evolution.

Attenuating the feedback loop between VMWare's vSphere and SpringSource's application servers will enable finer grained application footprints and timely elastic scaling. Imagine the ability to dynamically provision a specialized stack (i.e. container, framework, application meta-data and code) to handle a discrete unit of work (e.g. web request, transaction, resource query), intelligently pool the stack, and tear it down when not needed. SpringSource's investment in OSGI, inversion of control, lightweight containers, and application management interfaces establishes a necessary foundation to re-think and shrink run/manage away from monolithic systems (e.g. hardware virtual machines, application servers, and applications) and towards discrete business services.

Service models: Building services that add value, not follies

Blogger: Richard Watson

Should I use WS-* or REST? Should a service provide access over HTTP, MOM, or XMPP? These are the wrong questions for architects to ask when first conceiving a service. By concentrating on how to build, we lose focus on what to build.

Debates about whether to use REST or WS-* interface styles are seductive. But, these are the wrong questions to ask first. Interface style and middleware decisions should be based on consumer requirements. The primary service design principle emphasized in a service model is maintaining a clean separation of capability ('what') from interface concerns ('how'). If we are arguing at all we should be arguing about what services to build, and why.

What is a service? This question cannot be answered in the abstract. Service modeling is an entirely contextual activity. 

Despite their rich history, service design practices remain nascent. The identification and definition of sharable and reusable services is still more art than science. In this paper (Burton Group subscribers only) published today, I examine the neglected art of service modeling. When people hear "model" they immediately think of formalism, notation, and tools. My definition of service modeling is not a formal model or modeling notation. It's thinking about the answers to these questions in a repeatable way.

Can you answer these questions? Do you have places in your service development process where these questions are answered in a consistent way? Service modeling is a set of software architecture practices for providing us with a way to get at answers to these questions and document them in a way that's valuable for everyone who has a stake.

Near where I live, in Dublin, is an extensive park called St. Anne's Park. It's part of what was the Guinness estate (yes, that Guinness!). This is a Greek Temple, built in the park as tea house by the Guinness family. It’s has very ornate mosaics on the floor, and is almost perfect classical design.

But, it's a Folly. Follies are buildings, or parts of buildings that have no purpose other than as an ornament; they are purpose-built as ornaments; they are often eccentric in design or construction; there is often an element of fakery in their construction.

The Guinness family built this because they were wealthy, just because they could. We can't afford to do that in our organizations any more. What we build must be relevant, driven by the right business context, and play its part in a portfolio. As you can see from its state of repair, there's no great motivation to maintain the tea house. It’s not valuable any more, if it ever was...

Put simply, if you create services without a service model, then you are creating bespoke application integration interfaces and not doing service oriented architecture. If context is not driving you to create the right services, then they are most likely not adding value to your applications architecture, they are making it worse.

Let me know what service modeling means to you. If you’re going to the Burton Group Catalyst conference next week in San Diego, I'll be giving a session on service modeling on Wednesday afternoon in the "Whither SOA: What’s next?" topic.

Navigating the Myriad SOA Standards and Specifications

Blogger: Anne Thomas Manes

This just in:

"Members of OASIS, OMG, and The Open Group announce the publication of a new white paper, 'Navigating the SOA Open Standards Landscape Around Architecture.'  The paper was produced to help the SOA community at large navigate the myriad of overlapping technical products produced by these organizations with specific emphasis on the “A” in SOA; i.e., Architecture.

This joint white paper explains and positions standards for SOA reference models, ontologies, reference architectures, maturity models, modeling languages, and  governance. It outlines where the works are similar, highlights the strengths of each body of work, and touches on how the work can be used together in complementary ways. It is also meant as a guide to users of these specifications for selecting the technical products most appropriate for their needs, consistent with where they are today and where they plan to head on their SOA journeys."

Join me at "SOA: The Night of the Living Dead"

Blogger: Anne Thomas Manes

Calling all Boston-based SOA zombies!

My friends, Ron and Jason, over at ZapThink, will be hosting a soiree to discuss the death of SOA.

The venue: MJ O'Connor's Irish Pub on Columbus Ave in Boston, on July 23. Pundits joining me at the event include:

• Ron Schmeltzer
• Jason Bloomberg
• Dana Gardner
• Brenda Michelson
• Sandy Rogers

Total cost: just $29, and that includes food and drink.

For an even better rate, use discount code PRTDISC

Click here to register.

This Imperial Middleware will be operational as planned

Blogger: Richard Watson

Dum Dum Dum Dum, Dum Dum Dum, Dum Dum Dum.

KURIAN*

Lord Phillips, this is an unexpected pleasure. We're honored by your presence.

PHILLIPS**

You may dispense with the pleasantries, Thomas. I'm here to put you back on schedule.

Thomas turns ashen and begins to shake.

KURIAN

I assure you, Lord Phillips, my men are working as fast as they can.

PHILLIPS

Perhaps I can find new ways to motivate them.

KURIAN

I tell you, this middleware will be operational as planned.

PHILLIPS

The Emperor does not share your optimistic appraisal of the situation.

KURIAN

But he asks the impossible. I need more men.

PHILLIPS

Then perhaps you can tell him when he arrives.

KURIAN (aghast)

The Emperor's coming here?

PHILLIPS

That is correct, Thomas. And he is most displeased with your apparent lack of progress.

KURIAN

We shall double our efforts.

PHILLIPS

I hope so, Tom, for your sake. The Emperor is not as forgiving as I am.

<fades>

Commander Kurian will be relieved that it seems the Oracle Fusion deathstar will be operational as planned, as Anne reports here. So, let's award the middleware team 10 out of 10 for execution. Ramming acquisitions into a (excuse the pun) coherent set of products is tough engineering and marketing work.

Product strategy is a different matter. Burton Group's position on heterogeneity (not homogeneity) as a driver for an application platform strategy is well known. We see opportunities for Rebel incursions.

Also, technically, I don't (yet) see any influence of the BEA microkernel architecture surfacing in other parts of the suite. This is a pity, because promoting a modular architecture like OSGi really helps developers digest parts of the stack they need and remove the bits they don’t. The WS02 carbon project and Paremus Service Fabric are pioneers for this type of architecture.

Miko Matsumura has also picked up on the cinematic sweep of Oracle's advances through the middle(earth)ware market.

* Thomas Kurian - Senior Vice President, Oracle Fusion Middleware.

** Charles Phillips – President, Oracle.

Thoughts on Oracle 11g Announcements

Blogger: Anne Thomas Manes

By invitation, I went to a special NDA analyst briefing last week on the Oracle 11g announcements. As I tweeted at the time, I found it very refreshing to hear a strong and definitive strategy underlying the announcements. It's a four-part strategy:

Complete: Oracle wants to be a one-stop shop. You can buy everything you need: applications, software infrastructure, tools, databases, management, and hardware infrastructure from Oracle.

Integrated: All components of the complete platform are designed (or perhaps refitted) to work with each other.

Best-of-breed: Each component in the integrated, complete platform is a credible, competitive product in its own right.

Hot-pluggable: The environment is standards-compliant, so, if you desire, you can replace a best-of-breed Oracle component with a comparable standards-compliant component from another vendor.

Unfortunately, it's a specious strategy.

Yes, the Oracle environment is complete, integrated, and composed of (for the most part) standards-based, best-of-breed offerings. But if you take advantage of the "hot-pluggability" feature, you break the "integrated" benefits of the environment, which derive from the common development and management systems (JDeveloper and Enterprise Manager). But Oracle has deliberately limited the scope of these products to work only with Oracle-supplied platform components.

As alluring as the one-stop shopping strategy is, organizations must learn to just say "no".  The reality is that no one has an entirely homogeneous environment. Oracle claims that Enterprise Manager supports end-to-end business process monitoring, but the concept breaks down if the process includes a .NET service or a third-party COTS application. A better solution is a management strategy that embraces diversity.

Diversity in IT systems is a fact of life. The trend toward heterogeneity is only going to increase as organizations take advantage of cloud computing or implement ebusiness collaboration systems.

As for the specifics of the announcements: Two things I really liked:

• Integration of TopLink and Coherence -- you can now use the Coherence distributed data caching system within TopLink
• Integration of Collaxa and Fuego runtime engines -- a single engine can now run both BPEL scripts and BPMN models

Don't fixate on service reuse

Blogger: Richard Watson

Measuring service reuse alone is not the right way to determine the value of a service.  SOA advocates tend to fixate on service reuse, and fill business cases with return on investment calculations based on reuse projections.

"Reusability is always overpromised."

This is a comment from the CIO of a Fortune 500 company interviewed by Burton Group. It reflects the risk in fixating on reuse. The memory of his application architects claiming that reuse is virtually guaranteed by object-oriented design has not quite dimmed. Likewise, applying service oriented principles promotes reuse, but does not guarantee it.  Reuse is a complex issue, as much cultural as technical.  As Jeremy Meyer eloquently says [i]:

Reuse is about people and education, not just architecture ... The truth is that even the most beautiful, elegant and re-usable architecture, framework or system will only be re-used by people who: a) know it is there, b) know how to use it, and c) are convinced that it is better than doing it themselves.

A service may never be reused, but still be used to create value in other ways: by being adaptable and less costly to maintain, reducing redundancy, increasing security and compliance through consistent enforcement of policies, to name just a few other desirable outcomes. Exclusive focus on reuse blinds us to these other outcomes.
I've had some great conversations recently with clients about calculating reuse. It's a tricky calculation.  Most reuse savings formulae look something like these:

Savings = Cost of Implementation - Cost of Integration
or
Savings = services * reuse * change
or
Savings = sum ( f (cost of integration at consumer, cost of change to service provider for new consumer) ) over #reuse, where f is "some function of".

Reuse savings are problematic because stated like this, as the client I spoke to says, "if I've saved all this money, where is it?"  Of course, it's deferred expenses, but that doesn't sound quite so valuable.  These formulae also overstate the value of reuse over time.  Some kind of decay function is needed to take account of a discount in the 'Cost of Implementation' value.  I'm out of college too long and not crazy enough to attempt the math for that decay function, I just know it exists. I'm not convinced it's always a decay function either.  If a service can take advantage of network effects then its reuse benefit over time actually rises.  And unlike radioactive decay, the value of the service can be refreshed occasionally, say when a changes to reporting regulations dictate a different set of rules and the changes required can be made in an isolated place instead of across the board.  But that gets us back to the value of service "use" not "reuse".

  

[i] Jeremy Meyer. In "97 things every software architect should know", ed. Richard Monson-Haefel.  Sebastopol: CA: O'Reilly. 2009.

News Flash: Insourcing Fosters Relationship

Blogger: Kirk Knoernschild

A recent article on TechTarget claims that IT insourcing is on the rise. In other words, organizations are beginning to bring their recently outsourced IT functions back in-house. The article cites a few motivating forces for this change. Organizations have more control, can respond more quickly to business change, and are leery of outsourcing going forward due to unpleasant past experiences. The articles cites on of the reasons for outsourcing - it was cheaper - may not actually be true. While outsourcing may be cheaper short-term, the long-term cost may be more. Not an epiphany, for sure.

But what really caught my eye was the following quote from the article:

There's a sense of intimacy with the business users that's hard to achieve with outsourcing.

That's a rather profound statement. It took outsourcing for us to realize that collaboration with business users is a critical success factor on an IT initiative? Wow.

IT Labor

Since we're on the topic of outsourcing.... Another popular reason why organizations outsource is due to the IT labor shortage. Many claim there is a lack of qualified IT candidates. This reason is also the foundation of the H-1B program. Not so fast. I've never actually believed this was entirely true. My good friend Randy once said, "There's also a shortage of $2.00 per six pack micro-brews." If you're interested in IT labor, this Lou Dobb's video might interest you.

Conspiracy Theory

I take my car to a certified GM dealership, even though it's a bit more expensive than going to the hometown mechanic. Why? Because I know they won't use secondhand or refurbished parts. I also know they have the specialized equipment and expertise to work on my model vehicle. But if the GM dealership charged a 50% premium for the same level of service I could get at my local auto shop, I'd have no motivation to take my vehicle to the dealership. Now, apply this analogy to corporate IT.

If corporate IT charges a 50% premium over an outsourcing provider, but doesn't provide any strategic or competitive advantage to the organization, what's the motivation to use corporate IT? Instead, it's sound fiscal strategy to outsource. In other words, why should I pay $100 an hour for crap when I can get the same crap for $75?

For now, let's ignore the crap (that's a topic of discussion for another day), and instead focus on cost. $75 an hour may seem cheaper than $100 an hour, but only when you fail to factor in the number of hours it takes to complete a project. And the challenges inherent to distributed development, contract negotation with outsourcing providers, and more will impact delivery, which will increase time, which will drive up cost. So now, we're starting to realize that outsourcing isn't necessarily cheaper. Don't be surprised to see aggressive legislation on the table soon asking for an increase to the H-1B cap. Wouldn't it be ideal to have the best of both worlds? Cheap labor that's co-located! Really? Is it possible there's a causal relation between IT failure, motivation behind outsourcing, and the H-1B program?

Disclaimer: I'm not claiming that everything we deliver is crap. But there is no shortage of evidence illustrating some of the challenges we've had in delivering IT projects.

Disclaimer: I don't claim that we should abolish the H-1B program. However, there is evidence that H-1B visa holders are paid between $12,500 and $20,000 less than citizens. That's not the purpose of the program.

Cloud Computing is "depressingly" similar to SOA

Blogger: Anne Thomas Manes

David Linthicum wins the "grumpy architect" award for today.

He posted a rant on Sys-Con Media entitled, "Getting the Links Straight Between Cloud Computing & SOA". It's well-worth the read. In it he references an ancient (Feb 17, 2009) article by Stacey Higginbotham entitled, "HP Confines the Cloud for Enterprises," in which she wrote up HP's strategy announcement on Cloud Computing.

In particular, Dave was ranting about this comment:

But aside from some vague nods to the benefits of accessing information in the clouds (such as with web-based email) most of HP’s detailed talk of clouds in the first webinar was depressingly similar to the idea of service oriented architecture. HP offered clouds as merely a means to deliver IT as a service inside the enterprise.

Well, Duh!

Alright. I admit it. It's all my fault. I proclaimed that SOA is dead, and people who can't be bothered to actually read the blog post (or any of my follow-on blog posts), took it to mean that we shouldn't bother doing service orientation anymore.

I tried to be really explicit in my last blog post, "SOA: It's Dead, Jim," but perhaps people still didn't get it:

"SOA" as a term has lost its luster, but "SOA" as a practice is essential for all organizations going forward.

As I said in the original obituary:

Service-orientation is a prerequisite for rapid integration of data and business processes; it enables situational development models, such as mashups; and it’s the foundational architecture for SaaS and cloud computing.

The network is a bit overrun with articles talking about the intersection of SOA and Cloud Computing, so it really should be obvious by now: Cloud computing requires SOA.

Note, though, that the exigencies that require we do SOA will do nothing to revive it in the eyes of business people that control its funding. SOA is still dead: business people believe that SOA is expensive, and it doesn't deliver, and they don't want to allocate any more money to it this year. (In case you missed it, this is what I mean when I say, "SOA is dead".)

Our challenge as architects is to figure out how to do SOA without a big pile of money to fund our efforts. Fortunately, SOA doesn't need to cost a lot. It just requires good architectural practices. SOA is about design, not about technology.