ESB-oriented architecture: The wrong approach to adopting SOA

Document options

Print this page

E-mail this page

Discuss
Watch these demos

Integrate new tools and architectures into your environment -- fast!
Rate this page

Help us improve this content
Level: Introductory
Bobby Woolf (bwoolf@us.ibm.com), WebSphere SOA and J2EE Consultant, IBM
17 Aug 2007
This article examines projects organized around building an enterprise service bus (ESB). It describes why a project with no SOA goals is a bad idea, and it explains what to do instead to properly adopt SOA.
An increasingly common request from clients is to complete a project that does not use service-oriented architecture (SOA) as a whole, but instead implements only an enterprise service bus (ESB) architecture. Such an ESB-oriented architecture is easy to envision, but its success is difficult to measure. What clients requesting such projects do not understand is this: An ESB-oriented architecture does not produce business value. A project based on ESB-oriented architecture needs to be made into one based on SOA-oriented architecture to help ensure that it successfully delivers business value.
SOA is based on business requirements. It aligns IT and business so that IT systems work the way the business does, helping to ensure that IT produces business value. See the IBM whitepaper IBM SOA Foundation: An architectural introduction and overview (seeResources) for more details.
The primary goal of SOA is to align the business world with the IT world in a way that makes both more effective.
IT departments that use IBM products and services to build IT systems might have limited understanding of their businesses’ requirements. For an engineer accustomed to precisely planning how a system will work, the way the business works can seem unplanned and random. Explanations appear inconsistent and unworkable, and the needs of business users sound unrealistic and ever-changing. Business requirements become an urban myth, rumored to exist within the organization, but somehow melting away under the light of scrutiny.
From this perspective, aligning IT and business sounds impractical. It seems the business does not know what it wants. Its processes (such as they are) defy automation. Efforts to automate processes become frustrating and pointless.
What engineers understand is technology. Technology does not require fanciful wish lists of requirements; it just requires code. Code does not complain about being difficult to use, and the compiler does not change its mind day-to-day about what it really wants. Code either runs or it doesn’t. And if it runs today, it will run tomorrow.
Technology is easier for engineers to grasp, and it is more satisfying. It also happens to be the main thing most enterprise software companies sell. ESBs are technology, and they connect other technologies.
By contrast, SOA is complex, while ESBs are easy to understand. ESBs do not need any of those annoying business requirements; only technology requirements. ESBs are precise. They are based on standards: data formats, wire protocols, XML, IP, HTTP, SOAP, JMS, JAX-RPC, JAX-WS, and so on. SOA could get stuck in analysis paralysis forever, but an effort to build an ESB could actually get some real work done.
This is often referred to as a connect everything to everything project. The client has many parts—applications, computer systems, data centers, departments, subsidiaries, outposts, partners, and customers—and the parts do not talk to each other. The left part has no knowledge of what the right part is doing. One part has data that another part needs, and these two parts need to work together. If only all the parts were connected together, they could all work together. And unlike the futility of trying to understand what the business’ requirements are, connecting everything to everything is a problem that can be solved, because the solution is technology. If the IT department is a hammer, then an ESB is the nail of SOA.
The mindset is often: “We don’t know what else we need, so for now we will just build an ESB.” But is this really any different than the approach of “You start coding, I’ll go find out what they want”?


Back to top
Readers often summarize this desire to connect everything to everything with a single phrase: enterprise service bus. So what do they mean when they say they want an ESB? What do they mean by ESB? Do they necessarily call it an ESB?
Often clients like to rename that first word in ESB. Instead of enterprise, they describe another organizational unit, such as corporate, department, or government. Sometimes they describe what it will be used for, such as procurement or payroll. Or they describe what it will transmit, such as product or order. Even if what clients want is a corporate product procurement service bus, do not be fooled by the words preceding service bus. Those clients want an ESB. They sometimes even describe it as “An ESB but ….”
The client’s focus is really on the last part: the bus. In a technical topology that includes a bus, everything connects to the bus, and therefore to everything else using the bus. The bus is the main avenue of communication between the parts. Communication between applications, and even between computers on a network, is usually accomplished using messaging: a series of discrete packets of information. One resource that gives a great overview of this is the book,Enterprise Integration Patterns , which calls such connectivity a message bus.
Clients often do not think too much about the service part of ESB. An xSB, enterprise or otherwise, is for invoking services; otherwise, it is simply a message bus. Service invocation means one application telling another application what to do, and the other application doing it and usually sending back a response to report the result.
So, if what the client wants to build is an xSB, what are the services? “The services can be whatever it turns out they need to be,” the IT staff might say. This is the most definite indication yet that the project is purely a technology one. Implying that the services are irrelevant says that the applications using the bus are irrelevant, how the applications use the bus is irrelevant, and the application integration requirements (much less business requirements) are irrelevant. The xSB will work for anything. An ESB being architected for an SOA can initially ignore many of these service requirements, because they will become apparent as the SOA is fleshed out. But an ESB without an SOA has no services, and therefore it is just a bus.
A project to build just a bus can be considered an IT field of dreams project. Like Kevin Costner’s character in that baseball movie, the IT department is taking the attitude that "if you build it, they will come." If you build the bus, people will build SOA applications around the bus. The problem with the field of dreams approach is that, unlike in Hollywood, in the business world there is no guarantee they will come. If and when people do build SOA applications, they might not like what you built, so you might have to rebuild a lot of it anyway before they can use it. Even if they do eventually use it and love it, that is a lot of delayed gratification, which the IT department might find difficult to endure while waiting for the big payoff at the end of the movie.
If you create a ESB far before the services come, you may suffer the pain of no services to be plugged into or recreation a bus to accommodate the services.


Back to top
What is wrong with an ESB field of dreams? Remember the part in the middle of the movie where Annie wants to divorce Ray? (If not, it takes up the whole second act, where everyone thinks Ray is a fool.) Your project will go through a period like that too, and the project sponsor probably does not want his employer to divorce him!
The problem is this: An ESB by itself produces no business value. An ESB is a means to an end, not the end itself. An ESB is like the electrical wiring or plumbing of an SOA. Plumbing does not produce value; sinks with water coming out of their faucets do. Wiring does not produce value; lights, especially lights connected to switches, are valuable. A road is not valuable except that it enables you to get from one point to another. An ESB without an SOA is like a road from someplace nobody is located going to other places nobody wants to be. People might eventually want to go to those places, but in the meantime, the road is all cost and no benefit.
ESB-oriented architecture is inherently flawed in that it builds connectivity no one might ever want to use. The business does not derive additional value until systems connect to each other and are working together. Until then, the ESB is just cost with no benefit. It might make the IT department feel good because they have built something, but it will not make the business feel any better, because the business is not accomplishing anything it couldn’t have already accomplished without the ESB. The ESB becomes the equivalent of a human appendix for the IT department, a vestigial organ within the topology of deployed applications.
Rather than the IT field of dream’s slogan of “if you build it, they will come,” a more appropriate slogan comes from Extreme Programming (XP): “You aren’t gonna need it.” This slogan is shorthand for a very practical principle:
Always implement things when you actually need them, never when you just foresee that you need them.
This principle—don’t build it until you need it—is the opposite of the IT field of dreams. Rather than building it because you hope that someone will want it, do not build it until you know someone wants it. Then you can make sure to build what they want, not what you think they might eventually want. And you will not incur the costs of building it until you are also ready to reap the benefits of having built it. This principle is just a good business philosophy, and it applies to the IT department as much as any other parts of the business.


Back to top
To review, some good principles for any software development are:
Do not build it until you need it.
Build what creates business value.
Align IT and the business.
Instead of following an ESB-oriented architecture, follow an SOA-oriented architecture, and build the ESB as part of the SOA. In other words, do SOA—just as described inIBM SOA Foundation for creating and integrating applications that embody the SOA foundation architecture.
With this approach, you develop an ESB as part of developing the SOA. You will discover services based on business needs. Each service requires not only providers and consumers, but also a channel in the ESB to connect the two. That channel implements the service interface just like a provider (but acting as a proxy), including message formats for service requests and responses that enable remote invocation (such as interprocess communication) of the service. Differences in the consumers’ and providers’ service interfaces, message formats, scope, and qualities of service can be bridged and facilitated by mediations. All of this is the core of ESB design, and none of this can be done without knowing the services the ESB invokes. And knowing those services requires knowing the services in the SOA that will use the ESB.
In this light, connecting the applications is the easy part. Connecting their business functionality is the much greater challenge. That cannot be achieved by building only an ESB.


Back to top
Clients often want to build only an ESB because that involves a technology challenge without the need for messy business requirements. Building just an ESB becomes an IT field of dreams, where IT builds an ESB and then hopes some SOA will come along and use it. Such an ESB-oriented architecture loses the benefits of SOA. It does not create business value. In fact, it incurs cost without reaping immediate benefit. And it does not align IT and the business. The better alternative to ESB-oriented architecture is SOA-oriented architecture. Do not build an ESB by itself; build it as part of an SOA, preferably one that fits the SOA Foundation architecture that IBM recommends.