Blogger: Anne Thomas Manes
Follow me on Twitter: @atmanes
I had the great pleasure to participate in the crafting of the SOA Manifesto last month. Thomas Erl organized the effort. Fifteen of us gathered at the second annual SOA Symposium in Rotterdam, with two others contributing remotely. It was a diverse group of people representing vendors, consultants, analysts, and journalists. All but one of us are published authors. Some members of the group are big proponents of WS-* and ESBs, while others are big proponents of REST. All of us have lots of hands-on experience with SOA. (See the bottom of the page at the SOA Manifesto site for the list of authors of the SOA Manifesto.)
Our goal is to raise awareness and realign perspectives about SOA.
We modeled the SOA Manifesto after the Agile Manifesto. The SOA Manifesto begins with a preamble, followed by a set of value statements, and ending with a set of guiding principles.
Preamble
The preamble establishes the context for the manifesto and asserts what we believe to be the intents and purposes of SOA:
Service orientation is a paradigm that frames what you do.
Service-oriented architecture (SOA) is a type of architecture
that results from applying service orientation.
We have been applying service orientation to help organizations
consistently deliver sustainable business value, with increased agility
and cost effectiveness, in line with changing business needs.
A number of people have chided us for stating the obvious in the first paragraph, but our contention is that many people miss this most obvious point. Many people think that SOA happens simply by using a particular type of technology or product (e.g., WS-* or an ESB). But SOA is not about technology. SOA is entirely about design. You can't achieve service-oriented systems unless you apply the service orientation paradigm during system design.
The second paragraph speaks to the fundamental intent and purpose of SOA: to consistently, quickly, and cost-effectively deliver solutions that generate sustainable business value.
Value Statements
The value statements represent priorities.
Through our work we have come to prioritize:
Business value over technical strategy
Strategic goals over project-specific benefits
Intrinsic interoperability over custom integration
Shared services over specific-purpose implementations
Flexibility over optimization
Evolutionary refinement over pursuit of initial perfection
That is, while we value the items on the right, we value the items on the left more.
The bracketing qualifiers are really important here: The value statements represent prioritized values. Certainly the values expressed on the right side of each statement are important, but when faced with a hard decision between the two, designers should favor the value on the left. So let's look at each value statement.
Business value over technical strategy
This value statement asserts that the primary goal of service orientation is to generate business value. This goal trumps all other considerations, including technical strategy and adherence to the service orientation paradigm itself. In other words, you want to apply service-oriented principles when they are appropriate, but you must recognize that sometimes they are not appropriate for the needs of a specific project.
Strategic goals over project-specific benefits
This value statement asserts that system designers should always consider strategic business and technology goals when implementing a project. Decisions made for the project should not violate or subvert those strategic goals. This value statement is somewhat in conflict with the first one, and that’s intentional. Designers must balance delivering tactical value now with pursuing strategic value that’s better for the organization as a whole.
This value prioritization is missing in many organizations, and it's responsible for many SOA initiative failures. It's the "you can't see the forest for the trees" challenge. SOA requires an enterprise perspective even though implementation always occurs at a project-by-project level.
Intrinsic interoperability over custom integration
This value statement asserts that systems should be designed with interoperability in mind. Although custom integration is often required, a better approach is to build services with open, standard interfaces that support the needs of many. Keep in mind that interoperability is not simply a function of using standard protocols. I'm sure you've seen countless SOAP interoperability horror stories. Intrinsic interoperability is a function of semantic compatibility more than anything else. If you have to stick a transformation function into a communication path, you don't have intrinsic interoperability.
To some degree, I'm surprised that we were able to get the ESB vendors to agree to this value statement. If you build services that support intrinsic interoperability, you really don't need an ESB. Of course you might use an ESB as a service container (not just as a mediator), in which case you can use the ESB to expose an intrinsically interoperable service. I'll just point out, though, that RESTful services are more intrinsically interoperable than other types of services.
Shared services over specific-purpose implementations
This value statement asserts that it’s typically better to use a service that already exists rather than to build a new service that exactly matches a project’s needs.
I find this value prioritization is also missing in many organizations. Unfortunately, the impediment to using shared services is a cultural issue that's very hard to overcome. Many organizations suffer from chronic "I'm special" and "not invented here" syndromes, both of which often represent trust issues. This Geek & Poke cartoon exemplifies the problem:
You may notice that we use the term "shared" rather that "reusable" in this value statement. We had pretty strong consensus among the authors to avoid the term "reusability." We prefer "use" rather than "reuse". This value proposition primarily recommends avoiding redundancy. i.e., if you already have a service that implements the required capability, don't build another one.
Reusability is a tricky subject. A few years ago, I thought that service reuse was fundamental to the SOA value proposition. And I've certainly seen many SOA business plans that use projected savings from service reuse as the basis for their ROI. But I've backed away a bit from that focus on reuse. Designing for reuse is expensive, and many services will never be reused. And that's okay. Going back to the "strategic goals over project-specific benefits" and "intrinsic interoperability over custom integration" value statements, designers should always consider whether the services they are building could have value to the organization beyond the scope of the individual project. If so, they should design the service for reuse by concentrating on getting the granularity right and by adopting corporate standard data models that enable intrinsic interoperability. From my perspective, though, the true value of SOA comes from improved usability and maintainability of the application portfolio, as articulated in the next value statement.
Flexibility over optimization
This value statement asserts that designers should consider the needs of the future when designing systems. Optimization is clearly a desired outcome, but business requirements invariably change, and built-in flexibility reduces total cost of ownership and ultimately increases the system’s value.
Evolutionary refinement over pursuit of initial perfection
The last value statement asserts that time-to-value followed by continuous improvement is preferable to delayed deployment in pursuit of completeness and perfection. Although every designer would like to produce a perfect solution the first time, such perfection is not realistic because requirements always change. It’s better to deliver less-than-perfect value quickly and evolve the solution over time in response to real use.
Guiding Principles
We recommend that SOA practitioners consider these principles when pursuing service-orientation. I should point out that we aren't trying to define "SOA principles" here. Thomas Erl has done a great job describing the service-orientation paradigm, SOA principles, and service design patterns in his SOAbooks series. Most of these principles pertain to the business and cultural mindset required to be successful with SOA. Only a few pertain to technical design.
Respect the social and power structure of the organization.
Every SOA initiative is different, and the initiative has to work within the business and cultural constraints of the organization.
Recognize that SOA ultimately demands change on many levels.
As we assert in the preamble, the service orientation paradigm frames what you do. Adopting SOA requires changes to the way the organization funds and executes software projects. It requires new levels of collaboration across business and technical boundaries. It often requires new roles and responsibilities within the organization. It invariably requires changes and additions to governance precepts. It often requires a modification to established incentive systems to promote good behavior. Although you can start small and execute a Guerrilla SOA campaign, at some point you need to enact organizational and cultural changes to succeed.
The scope of SOA adoption can vary. Keep efforts manageable and within meaningful boundaries.
In other words, don't bite off more than you can chew. The scope of "manageable and within meaningful boundaries" varies depending on the culture and maturity of your organization. One of the most important SOA planning efforts is analyzing and determining what those boundaries are.
Products and standards alone will neither give you SOA nor apply the service orientation paradigm for you.
In other words, SOA is something you do, not something you buy. The only way to achieve service-oriented systems is by applying service-oriented principles during system design.
SOA can be realized through a variety of technologies and standards.
In other words, SOA is not dependent on any particular set of technologies or standards. i.e., you don't have to use WS-* or an ESB to build service-oriented systems. You can build SOA systems using "plain old HTTP", CORBA, MQ, and other middleware.
Establish a uniform set of enterprise standards and policies based on industry, de facto, and community standards.
Enterprise standards and policies provide the foundation for intrinsic interoperability. It doesn't so much matter which standards you use, but once you have established them, you want to convince people to use them consistently.
Pursue uniformity on the outside while allowing diversity on the inside.
We haggled quite a bit about this principle. It started out as a very simple assertion to "embrace diversity". As any large organization knows, it's extremely difficult, if not impossible, to maintain a completely homogeneous environment. Heterogeneity is a fact of life. And yet many organizations persist in trying to align their systems with a single vendor strategy. (I have many client dialogs seeking advice about "which one platform/middleware/ESB/product/etc should we use" -- and my advice is always to recognize no one product is likely to support all requirements. Most organization employ multiple platforms and products. Stop trying to fight it. Embrace diversity.)
But while heterogeneity is inevitable, it also causes usability and interoperability headaches. In order to achieve intrinsic interoperability, you want to minimize the number of standards, protocols, and data models that service consumers have to deal with. Hence the previous principle recommending a uniform set of enterprise standards and policies. So we decided the best way to articulate this principle is to present it as a strategy to balance uniformity and diversity.
"Outside" refers to the interfaces that systems and services expose to the outside world. Uniformity reduces complexity, increases usability, and promotes intrinsic interoperability. But external uniformity does not require internal uniformity. Effective use of encapsulation and abstraction allows and enables heterogeneous coexistence.
Identify services through collaboration with business and technology stakeholders.
In order to deliver services that generate business value, service designers need to get input from the people that understand the business goals of the service. Likewise business people must trust in technology people and give them the latitude to develop services in such a way that the services can best support the needs of the business as a whole. This principle reflects the sentiments of all the value statements.
Maximize service usage by considering the current and future scope of utilization.
This principle supports the 4th, 5th, and 6th value statements (shared services, flexibility, and evolutionary refinement).
Verify that services satisfy business requirements and goals.
This principle supports the 1st and 2nd value statements. The fundamental goal of SOA is to deliver business value, but don't allow tactical business requirements to impede or derail strategic business goals.
Evolve services and their organization in response to real use.
This principle supports the 1st, 5th, and 6th value statements. Only real use can verify whether a service delivers its intended value. As Moltke the Elder said, "No plan survives contact with the enemy." Inevitably, requirements evolve. Services must adapt in response to real use. In some cases, modifications occur inside the service. In other cases you may need to refactor or reorganize the services to support revised or new requirements.
Separate the different aspects of a system that change at different rates.
Separation of concerns (SoC) is a fundamental design principle of all software engineering. It is the motivation behind structured programming, object-orientation, aspect-orientation, refactoring, decomposition, and, likewise, it is the foundation of service-orientation. SoC reduces software complexity and improves comprehensibility. It promotes traceability within and across artifacts and throughout the software lifecycle. It facilitates reuse and adaptability. It also simplifies component integration. Most importantly, it improves maintainability.
Reduce implicit dependencies and publish all external dependencies to increase robustness and reduce the impact of change.
This principle reflects the loose coupling SOA design principle. Loose coupling refers to the degree of flexibility between system components. Implicit dependencies increase the degree of coupling that exists between system components and therefore reduce flexibility. Publishing and abstracting dependencies reduces coupling and enables services to be moved, scaled, managed, and evolved without breaking existing connections. Loosely coupled systems eliminate the need for interoperating parties to understand a priori the internal details of each other’s IT systems.
At every level of abstraction, organize each service around a cohesive and manageable unit of functionality.
This principle reflects the fundamental encapsulation design principle. In SOA, the fundamental unit of encapsulation is a service (hence the name, "service-orientation"). Ensuring that services are cohesive and manageable is achieved by applying service design principles such as service abstraction and service autonomy.
The phrase "at every level of abstraction" also reflects the service composability principle. That is that services can be composed to create higher-level services. The point is that designers and developers must apply the same service abstraction and service autonomy principles to composite services as they do to low-level services.
[Updated 11/22/09: fixed a couple of typos (Thanks Brian) and added the paragraph under the cartoon on "use" vs "reuse" in response to Herbjorn's comment.]
