Saturday, June 20, 2009

Agile SOA - Part III

This is the third and final blog in this series. In Part I, I opened with the thought that the real challenge of SOA is changing the way you look at the IT investment of your business to be more from the perspective of software architecture. In Part II, I discussed that the best approach to software architecture is Agile due to the evolutionary nature of software. In this final installment I'd like to close with how an Agile approach to software architecture - based on thinking of SOA in terms of composite design patterns implemented as part of your application architecture through the use of embedded frameworks will help you better realize the benefit of an Agile approach.

In the quest for enterprise architecture discipline, many of us are re-creating our futures out of our past by modeling it through a static, black-box approach. If your business' enterprise architecture can only be described in a power-point slide deck, using boxes and connecting lines - then you haven't taken the time to truly understand the collective learning from the previous generation of hardware-centric architects - and are doomed to make the same mistakes they made. SOA makes behavior a first-class design concern, and pushes static representations of architecture to a "behind-the-scenes" supportive role.

A business' enterprise architecture is a living organism, much like the business it supports, and at it's core should be a software architecture - modeled through a dynamic, white-box approach. We should be striving to describe this software architecture through a means closer to a linear, episodic series of video clips. Each episode should demonstrate an encapsulated portion of the system in a white-box manner - demonstrating the scripted pattern in which these components interact (think of a football play being viewed from an overhead camera with primary and optional flows for the different players on the field). Episodes should be linked together in a linear-fashion through the use of rule-based pathing (think of a football game as a composition of individual plays - where the individual plays are woven together with an over-arching strategy that capitalizes on both an individual team's strengths and an opposing team's weaknesses). It is in this way that a business strategy can best be decomposed into a set of composite processes that capitalize on the competitive advantage of a business within an industry full of competitors.

Many of you who are reading this may think that I am just another advocate of a Business Process Management (BPM) middleware-centric approach to SOA. In fact, I am not. Using my analogies above, I believe the Process Manager pattern is a good fit for demonstrating the scripted interaction of local components. I emphasize the word local here because I do not believe Process Manager is a good architectural pattern for interactions of distributed components. Hohpe and Woolf describe the Process Manager pattern as follows in their Enterprise Integration Patterns book: "Using a Process Manager results in a so-called hub and spoke pattern of message flow." Similar to database architecture, the hub and spoke architecture of a traditional process manager will not scale well horizontally. Horizontal scalability is key to distributed computing, especially within today's cloud hardware architectures.

If you still think that the Process Manager pattern is best for managing the scripted behavior of local components, then you have to take a long, hard look at the BPM middleware at the core of most commercial-off-the-shelf (COTS) vendor SOA suites. BPM middleware today is built around the Web Services Business Process Execution Language (WS-BPEL) standard - which describes a standard way in which many, distributed Web Services can be orchestrated within the context of a single business process. If the Process Manager pattern should not be used for distributed components, then the link between Web Services and today's BPM middleware should come in to question. If the Web Services standard is removed from BPM middleware, then the whole WS-BPEL standard fails as a foundation for implementing the Process Manager pattern, and COTS BPM products in-turn fail as a foundational component of your SOA infrastructure.

As an alternative to BPM middleware, I prefer a lightweight solution, such as the one described in the following InfoQ article by my colleague and friend, Oleg Zhurakousky: Workflow Orchestration Using Spring AOP and AspectJ. This article demonstrates how to build and orchestrate highly configurable and extensible, yet light-weight embedded process flows within your software architecture using Aspect Oriented Programming (AOP) techniques. Oleg's use of Spring AOP, a proxy-based AOP mechanism, to address functional concerns such as instrumenting a process with activities while using Aspect, a byte-code weaving AOP mechanism, to address non-functional concerns such as activity navigation and transition is, in my opinion, the perfect way to implement the scripted interaction of local business logic components within the Process Manager pattern.

So if the Process Manager pattern is not the right foundational pattern for your distributed SOA, then what is? I believe the right foundational pattern for your SOA is the Staged Event Driven Architecture (SEDA) pattern first published by Matt Welsh, et. al., from the Computer Science Division of the University of California, Berkeley in 2001. In SEDA, applications consist of a network of event-driven stages connected by explicit queues. Using my previous football analogy, think of the network as the whole football game, composing a number of event-driven stages which represent the individual play. The network architecture underlying SEDA is intrinsically better at supporting the horizontal scalability demanded by distributed SOA computing.

Within the context of SEDA, a stage is a self-contained application component consisting of an event handler, an incoming event queue, and a thread pool. As described above, that self-contained application component could itself be a composite scripted interaction of local application components. The introduction of a queue between stages decouples the execution of application components by introducing an explicit control boundary - which allows for their distribution. The SEDA model constrains the execution of a thread to a given stage, as a thread may only pass data across the control boundary by enqueuing an event. Introducing a queue between two modules of an application provides isolation, modularity, and independent load management. Because stages interact through an event-dispatch protocol instead of a traditional API, it is straightforward to interpose proxy stages between components for rules-based pathing, performance profiling, and/or testing & debugging.

One of the best implementations of the SEDA pattern is Spring Integration, described in the following InfoQ article by Joshua Long: Getting Started with Spring Integration. In an answer to the "Why Spring Integration?" question, Joshua says it best with the following quote: "Because Spring Integration is so lightweight (you deploy the Spring Integration server with your application; you don't deploy your application to Spring Integration) and so focused on the development life cycle (XML schemas to facilitate configuration, POJO-friendly APIs, and strong integration with the Spring Framework and JEE), you'll find Spring Integration a better fit than a lot of other ESBs." Spring Integration supports a pluggable transport layer, where a lightweight, highly scalable Message Broker, such as Apache ActiveMQ, can provide support for the network architecture that allows for the horizontal scalability of the Spring Integration solution.

So why all of these tools from SpringSource (Spring Framework, Spring AOP, AspectJ, Spring Integration, and ActiveMQ)? Traditional Enterprise Integration products promote a proprietary development and deployment model that requires a steep, costly organizational learning curve to successfully adopt. In addition, the more successful you are at adopting these development tools and deployment models - the more locked in to those proprietary products you become. The SpringSource mantra is Eliminating Enterprise Java Complexity. Using their standard Java development tools and deployment models, SOA can be incrementally adopted in a lower risk, more agile way - led by the Java developers and systems analysts you already have using the tools they already know (and love). The end-result of this incremental adoption is simply a re-factored version of your current Java and .NET business applications. Finally, the Spring Framework, which serves as the foundation for all of these tools, ensures both the tight integration and full portability you've come to expect from any SpringSource solution.

As I stated in the very beginning of this series, SOA is an architectural pattern - not an expensive suite of software products. A pattern, by definition, is the encapsulation of a complex, dynamic system into a reusable component. Patterns are meant to describe, through a white-box approach, best practice ways for you to build YOUR solution - they are not meant to be solutions in of themselves. Patterns therefore lend themselves to be best implemented by lightweight, embeddable frameworks that serve to support your solution, not heavy, commercial-off-the shelf products that aim to control it. The integrated Spring components, backed by the unwavering commitment to simplicity of the SpringSource company, provides you with the Agile SOA solution you need to support the evolutionary nature of your business.