The problem with SOA is that it has always been too abstract. The SOA manifesto that was signed late 2009 confirms this. No longer is it a set of technologies or even a set of standards, it is just simply the architecture that arises from applying service orientation which is defined as building modular systems via evolutionary refine to achieve business goals. This definition is a bit too abstract for me. Astonishingly, that’s as good a definition you can find in the manifesto.
So I decided to dig deeper, possibly I can gleam some knowledge by go through patterns discovered in practice and documented in the book “SOA Design Patterns” by Thomas Erl. This is a massive book with over 800 pages, the patterns in the book can also be found in www.soapatterns.org. The problem I have with almost all SOA books is that SOA is discussed in a manner that reveals little differentiation from any other distributed processing model.
The first hundred pages of the book covers introductory material covering SOA and Design Patterns. There’s nothing new here that you can’t find in other books on the subjects. So let’s dive straight into the meat of the book, the Design Patterns themselves.
I’m a big fan of Design Patterns, however I just abhor it when authors define a Design Pattern that is an obviously implied by the domain you are defining patterns for. For example, if we take the Object Oriented Programming (OOP) domain, Polymorphism is not a Design Pattern, it is an attribute of OOP. When I see these kinds of patterns, its an indicator to me of the lack of rigor in vetting out these patterns.
In the original GOF book, the OOP design patterns are categorized into 3 sets these are Behavioral, Creational and Structural. In this book the categories are Service Inventory, Service Design and Service Composition. The “Service Inventory” category is the most difficult to grasp simply because it is too abstract and its definitions are very weak. The Service Design category covers concerns revolving around the design of services by itself. The Service Composition category covers concerns that cover how services are composed together and how they interact. Service Inventory however seems to be describing services at a meta-level. That is, how would one describe services
The book is a very difficult read because it avoids the use of more concise terminologies commonly used in other computer science texts. Furthermore, it employs pattern names that although sound familiar, can lead to a lot of confusion. In my attempt to understand the book, I will be relating the Design Patterns of this book to more commonly understand computer science terminology.
The first category of patterns named “Service Inventory Patterns” covers ways in which Services are to be described. It can be a bit confusing discussing ideas in a meta-level, or in other words attempting to describe how you describe things. That’s the main flaw of this section in that it is not made apparent as to what is being talked about.
Chapter 6 covers “Foundational Inventory Patterns”. These patterns is simply recording and categorizing services. The “Enterprise Inventory Pattern” says that you should recorded in an inventory, said inventory can be further categorized into different domains (i.e. “Domain Inventory Pattern”) and to various interacting layers (“Service Layers”). Each service can be normalized to minimize overlap in functionality (i.e. “Service Normalization Pattern”) and making sure to avoid redundant logic (“Logic Centralization Pattern”). A standard protocol (“Canonical Protocol Standard”) and standard schemas (“Canonical Schema”) may be defined in the inventories. Nothing really informative in this chapter, its all about book keeping. Maintaining a Meta data repository to track a systems artifacts is nothing new, I personally would have condensed this as “Meta Service Pattern” and just shoved in all the different aspects into a single pattern.
Chapter 7 covers “Logical Inventory Layer Patterns” which in my opinion simply talks about the kinds of services that may be implemented (really just another categorization). That is one can talk about Utility, Entity and Process focused services.
Chapter 8 covers “Inventory Centralization Patterns”. In general Processes, Schemas, Policies and Rules (which incidentally are all meta-data) can be positioned in a central location so as to avoid duplicate and inconsistent definitions. I would have just called this “Source of Truth Pattern”.
Chapter 9 covers “Inventory Implementation Patterns”. Which would mean something along the lines of how you would implement ‘meta data’. Unfortunately I fail to see the logic behind why the patterns in this chapter are collected in this category. The category seems to consist mostly of patterns involving the sharing of compute resources across multiple services. The first pattern “Dual Protocols” doesn’t really belong here, it really is about support more than one protocol for a given service. I would in fact rename this a “Service Virtualization” pattern. “Canonical Resources” is about providing standard interfaces to compute resources. “State Repository” is about providing a utility service for storing service state. “Stateful Services” is well about Services that maintain their own state. “Service Grid” is some kind of service fabric that provides high scalability and fault tolerance for services that require states. I don’t know why this is a pattern, it seems to be more of a technology. “Inventory Endpoint” is a kind of service that acts like a facade to multiple services. “Cross Domain Utility Layer” provides utility services than span multiple domains. Though this pattern seems to be a replay of a previously mentioned layering pattern.
Chapter 10 covers “Inventory Governance Patterns”. Which would mean manage ‘meta data’. “Canonical Expression” states that there should be a standard way for defining contracts. “Metadata Centralization” states that there should be a registry to store services for discovery. I would rename this pattern as “MetaData Discovery” to disambiguate itself from the “Inventory Centralization Patterns”. The key point here is that meta-data should be discoverable by the services within the system. “Canonical Versioning” states that there is a standard way of defining versions of services, this pattern in fact is ambiguous with a later pattern that describes the idea that there should be a language for versioning.
The next set of chapters covers Service Design.
Chapter 11 covers “Foundational Service Patterns”. The problem I have with this chapter is that it talks about fundamental concepts which is apparently is difficult to differentiate from taking about meta-data. In other words, if I can describe my vocabulary then I am in essence defining the foundations of what I’m describing. The chapter attempts to include patterns that one would assume as being all too obvious. For example, “Functional Decomposition” pattern states that a problem can be broken down into smaller problems. The inclusion of this kind of pattern is just plain simple absurd. There is “Service Encapsulation” which has a misleading name, but it is about designing existing logic as a service that can be used outside of its original context. Which again Erl continues to state the obvious through complex pattern definitions. Finally there are two patterns “Agnostic Context” and “Non-Agnostic Context” patterns which is all about identifying multi or single purpose services. This chapter seems completely pointless in my opinion.
Chapter 12 covers “Service Implementation Patterns”. This is when finally there is some meat to the bones. However, some of these patterns here a miscategorized in that they are more about Service Composition, for example “Service Facade”, “Redundant Implementation”, “Service Data Replication” should be in the Service Composition category. In fact, it would have made better sense to categorize these patterns and those in chapter 9 under “State Handling Patterns”. “Partial State Deferral” pattern is indeed an implementation detail of service in how it manages its runtime state. I personally am a bit ambivalent about SOA design patterns that concern themselves with resource optimization. These kind of patterns belong elsewhere. “Partial Validation” pattern permits services to focus on what’s relevant in data and ignore the rest. This is a very useful capability that supports both versioning and interoperability. “UI Mediator” pattern is likely the most unique pattern I’ve found in this book. It is about providing a mechanism to support receiving timely feedback to a user on the progress of a service execution.
Chapter 13 covers “Service Security Patterns”. This is a very coherent category in that it restricts itself with the concern of handling security of services. This is probably one of the better chapters, and the interesting coincidence is that none of the patterns are written by Erl. In fact, as a rule of thumb, patterns that were written by someone other than Erl tend to be of more valuable. I in particular have high regard to the patterns written by David Orchard, these are non-obvious and quite insightful. However, Erl however a times creates a pattern that appears to be a duplicate of Orchard’s pattern (i.e. Version Identifier) and does a very poor job at presenting it (i.e. Canonical Versioning).
Chapter 14 covers Service Contract Design Patterns. This actually is a good categorization however I would have chosen a different name. I would label it “Contract Coupling” patterns. Decoupled Contract – States that a contract should be decoupled from its implementation. This is actually a practice a good practice worth emphasizing.
Contract Centralization – States that all access to a service is through its contract. A better name would be Service Encapsulation. Contract Denormalization – This seems counter to chapter 7 service normalization. The pattern states that redundancy in contract may be required to reduce demands on consumers. These kinds of patterns that appear to be in conflict with other patterns are actually the ones that can be quite insightful. However I would rename this as “Contract Redundancy”. Concurrent Contracts – Were a service defines different kinds of contracts depending on target consumer, again running counter to Service normalization. Finally, a very interesting pattern “Validation Abstraction” – where Validation logic is made portable from the service contract.
Chapter 15 covers Legacy Encapsulation Patterns. This is yet another bad chapter which covers the “Legacy Wrapper” pattern which clearly is the same as “Service Encapsulation”.
“Multi Channel Endpoint” pattern which is intended support multiple user access channels (ex. laptop, mobile, etc.) which again is expounding on the obvious. Services are meant to be shareable across multiple contexts, is it not blindingly obvious the multi access channels would share the same service? Finally there’s the “File Gateway” pattern which is the same thing a “Protocol Bridging” that is described in a later chapter. This chapter makes me wonder as to the target audience level of technical sophistication.
Chapter 16 covers Service Governance Patterns. “Compatible Change” pattern written by David Orchard discuses how to change a contract without affecting legacy consumers. A good example of a well written and insightful pattern. The same goes with the next pattern “Version Identification” which describes the need to define a Version vocabulary that identifies the compatibility constraints between versions. “Termination Notification” pattern is another pattern that is all to easy to forget. There should be a mechanism for contracts to express service termination information. This is then the point where this chapter turns for the worse. “Service Refactoring” pattern which is an obvious consequence consequence of “Service Decoupling” described previously. “Service Decomposition” pattern, well this is just a refactoring technique and the same goes for the “Proxy Capability” pattern. Now my head is beginning to hurt. Where you find the “Decomposed Capability”, with the following description “How can a service be designed to minimize the chances of capability logic deconstruction?”. I’ve got simply little patience left to figure what is meant here. The same goes with the “Distributed Capability” pattern. I’ll likely make another effort some other day.
The next section covers Service Composition Patterns, which discusses patterns on how to compose existing services with each other.
Chapter 17, a chapter that bordering on the absurd. The “Capability Composition” pattern is about composing services out of other services. I’m a bit confused here, for all this time I had thought that services by definition were composable. “Capability Recomposition” which again is obscured with a description like “How can the same capability be used to help solve multiple problems”. This is just plain and simple service instantiation. I can’t see how this is non-obvious. The book seems to repetitively recast well known computer science concepts as patterns and furthermore recasts these as entire chapters. The entire chapter can be summarized in one sentence “Services can be composed of services and Services can be instantiated and invoked in multiple contexts”. I’m beginning to get the feeling that Erl has trouble understanding basic concept like ‘instantiation’.
Chapter 18 covers “Service Messaging” patterns, Hohpe “Enterprise Integration Pattern” provides a much better treatment of this subject area and I refer you to his excellent book if your interested in this. To be brief, the following patterns are discussed: “Service Messaging”, “Messaging Metadata”, “Service Agent”, “Intermediate Routing”, “State Messaging”, “Service Callback”, “Service Instance Routing”, “Asynchronous Queuing”, “Reliable Messaging” and “Event Driven Messaging”. The Author again tries to re-define a common word, he defines the “Service Messaging” pattern which essentially is “Asynchronous Communication” as a pattern.
Chapter 19 covers “Composition Implementation Patterns”. I find the title to be confusing, I simply don’t understand what ‘Implementation’ is meant in this context. The chapter covers a incoherent collection of patterns: “Agnostic SubController”, “Composition Autonomy”, “Atomic Service Transaction” and “Compensating Service Transaction”. I would think that the appropriate title for this could be “Scope of Work Patterns”. It is as if Erl structures his chapter by creating combinations of the words “foundational” and “implementation” without giving any thought as to what they mean.
Chapter 20 covers Service Interaction Security Patterns which covers an interesting collection of patterns not written by Erl.
Chapter 21 covers Transformation Patterns. The chapter covers the “Data Model Transformation” pattern and the “Data Format Transformation” pattern which were written by Erl and the “Protocol Bridging” pattern written by Mark Little. The latter pattern is clearly the generalization of the former two.
In summary, the SOA Design Patterns book isn’t structured with the same rigor and coherence as other Design Patterns books. The content is unusually wordy and repetitive. There are a lot of diagrams but a majority of them provide little insight. The book takes well known concepts in computer science and regurgitates them as design patterns essentially taking what is obvious and making them obscure. Despite the poor quality of most of the book, its saving grace is that there are but a few patterns that have been submitted by contributors that are of a high quality.
However considering the pervasively poor quality of SOA books in general, I’m going to say it is one of the more valuable SOA books. Even if this bar is extremely low, this is of the few SOA books where you can indeed find some true nuggets of wisdom. (The book’s website has a lot more interesting patterns that weren’t published with the book) However, you have to dig very hard and long to find them because the map that is provided can is deliberately obscuring and more of a hindrance than an aid. Read it only if you know what to look for.
So you don’t have to waste your own valuable time, I’ve collected a reference list of patterns from the book that are of some value. Included is a quick explanation of my own and an alternative and hopefully more concise name.
- Canonical Protocol – Always convenient to standards on a common protocol to reduce bridging costs. Uniform Protocol.
- Dual Protocol – Supporting more than one protocol increases the number of compliant clients. Virtual Service.
- Canonical Expression – Specifications about meta-data should be standardized to avoid the cost of translation. Canonical Metadata Language.
- Metadata Centralization – SOA systems should support some kind of discovery of metadata services. Metadata Discovery.
- Partial Validation – Services should support non-strict validation of messages. Non-strict Validation.
- UI Mediator – Provide a capability to receive timely feedback when monitoring a service’s execution.
- Exception Shielding – To ensure security the implementation details of an exception should be hidden from a client.
- Message Screening
- Trusted Subsystem
- Service Perimeter Guard
- Partial State Deferral
- Contract Denormalization – Redundant specifications are something necessary to reduce coupling. Redundant Contract.
- Validation Abstraction – A language for input validation should be introspect-able to permit flexibility in where validation is performed. Introspect-able Validation.
- Compatible Change – Service contract changes can be performed in a way to support backward compatibility.
- Version Identification – A versioning vocabulary should reveal the compatibility constraints between different versions of a services. Versioning Constraints.
- Termination Notification – A service should have a mechanism to express its availability.
- Messaging Metadata – There should be a mechanism to parse information about a message without having to read the entire message. Message Envelope.
- Intermediate Routing
- State Messaging – Conversational State can me stored in message. Conversation State Messages.
- Service Instance Routing – Communication between services may be routed using logic that is dependent on the content of the message. Content Based Routing.
- Asynchronous Queuing – Clients need not require the temporal availability of the services it requires. Asynchronous Communication.
- Reliable Messaging – Clients should not have to manage the reliable delivery of a communication to its destination.
- Event Driven Messaging – A service may not require the knowledge of the identity of its clients. Publish and Subscribe.
- Compensating Service Transaction – Actions performed by services should be undoable.
- Data Confidentiality
- Data Origin Authentication – A mechanism for discovering the provenance of data in essential. Non-forgeable Provenance.
- Broker Authentication – An intermediate broker may be required when there is no trust between two interacting services. Trust Broker.
- Protocol Bridging – SOA should allow the inclusion of a protocol mediator to translate communication between services. Protocol Mediator.
Everything else not listed here is most likely to consist mostly of fluff and best be ignored. Do let me know however if I mistakenly ignored a good design pattern.