"Sometimes I lie awake at night, and I ask, 'Where have I gone wrong?' Then a voice says to me, 'This is going to take more than one night.'"—Charlie Brown.
This may well be the plight of many CIOs and IT directors today. Companies big and small have invested time, effort, and money over the past few years hoping to realize the SOA (service-oriented architecture) dream. Numerous SOA implementations have resulted in varying degrees of disappointment. And many companies are coming to realize that SOA is more complex to implement than expected, requiring a laser-like focus on all aspects of enterprise data and a deeper shift in organizational culture than any previous technology wave has demanded.
As we all know, an architecture oriented around services is not new. A few precursors to SOA, such as CORBA and DCOM, have already successfully bridged disparate applications using a loosely-coupled services-oriented approach. What is new about the SOA wave is that SOA is not simply about services. The emergence of the Internet and XML has opened the floodgates for data exchange. The software industry has lined up behind common data exchange format (XML) and Internet transport protocols like it never has before. Thus, a surge of well-accepted and open standards have emerged that enable the promise of SOA: to support flexible configuration of business processes, reduce operational costs, enable dynamic discovery of services, and provide seamless integration between applications, departments, and trading partners.
Unfortunately, much to the disappointment of enterprises and technologists, these lofty expectations from SOA are not being fulfilled. This is not because the promise itself is false, but because most SOA implementations today are experimental in nature. The good news is that valuable lessons can be learned from an SOA experiment, lessons that can help transform it into an enterprise-grade implementation that realizes the full potential of SOA.
Before we explore the challenges that lie in the path of SOA, let's step back and reexamine the goals of an organization embarking on an SOA implementation.
To achieve process visibility and flexibility
The worldwide SOA wave is definitely a well-justified one; SOA has come to represent an amalgamation of several radical changes in distributed computing. Organizations always invest heavily in technology to stay ahead of their competition, and SOA provides that breakthrough opportunity. At the same time, organizations have also been refining their business processes to squeeze out every last bit of competitive advantage. The emergence of business process management (BPM) promises continuous process improvement and never-seen-before collaboration between businesses and IT groups. SOA is the umbrella under which organizations are making a concerted effort to gain complete visibility into their data and processes, perform continuous improvements, and exercise fine-grained control in an effective and transparent manner.
To break down silos
The second goal of SOA is to break down application, department, and trading partner silos—built through years of software development—that addressed ever-evolving business requirements and technology improvements. Having been assigned their own IT budgets, departments have built applications to address their immediate needs, with little visibility into projects around them that might have overlapping functionality. The result: numerous legacy applications that leave IT departments struggling to reconcile duplicate information, and bits and pieces of business processes strewn across hundreds of applications. An SOA promises to break down these silos and let organizations gain better visibility into their data and processes.
To manage better data
Organizations not only want to manage data better, they also want to manage better data. It is important to ensure that the data being produced and consumed across the organization and its trading partners is clean, reliable, secure, well-governed, and fast. One of the goals of SOA is to provide a composite data services platform with a unified set of components for data access, quality, transformation, governance, and caching, among many other data-centric services.
A typical composite data service is an assembly of well-behaved and well-understood atomic activities and business rules deployed as a more functional service. For example, a composite data quality service for customer contact information might sequence an address validation service and an email validation service, and add the business rules to dictate what happens when either service reports an error. Composite data services themselves are deployed and managed in a metadata repository so that they are transparent and easily governed.
To reuse services
A related goal of SOA is to effectively manage and reuse enterprise services and data. Services developed by one group in an organization can be used by any other group within or outside the organization if they are published and described in a standards-based format in an accessible registry. When data and services reside with their owners and are shared by consumers as they need it, operational costs associated with their maintenance and management are reduced. Reuse is one of SOA's biggest incentives.
To align organizational goals
Another goal of SOA is to align the business and IT groups behind the organization's goals to better aid the development of flexible and configurable business processes. Historically, business and IT domains have maintained quasi-independent charters to improve the organizational bottom line. Business groups try to streamline business processes for competitive advantages, and IT departments try to use technology adoption for implementing these processes. The traditional mode of operation has the business group identifying the process changes and then "tossing them over the wall" for IT engineers to implement. Neither group has much visibility into the other group's function. BPM is a facet of SOA that tears down the business-IT divide by enabling continuous process improvement through modeling, simulation, execution, and monitoring in vocabularies that are shared and understood by both business and IT departments.