High Time to Get Rid of the Stack

One of the biggest mental crutches that is preventing people from truly comprehending and building composable distributed services is the notion of the stack. The stack is so prevalent in computing. It is everywhere, every microprocessor and almost every programming language supports this construct. It is the basis of the procedural call.

What if you had a variant of the conventional procedure call? That is one that doesn't by default return to the original caller. This concept is called a Continuation. Languages based solely on continuations are called "Stackless". A continuation is a means of capturing the control flow of the program and manipulating it. Some authors like to use the phrase "Reify the control flow". However, think of it as defining an Object to represent the control flow.

Gregor Hohpe has an interesting article "Correlation and Conversations" that shows that in our quest to abstract the invocation mechanisms in asynchronous environments we end up with a situation that looks like a call stack. A situation we wanted to avoid in the first place with asychronous messaging.

Gregor Hohpe's proposed strategy was to use a "Conversational Identifier", he writes:

When we look at message exchanges between systems in a broader context we realize that Request-Reply is just the most basic form of a Conversation between multiple systems. A conversation is the coordinated exchange of messages between two or more partners. The coordination is distributed as each participant in the conversation has a model of the conversation that relates to this participant's specific role in the conversation.

Conversations are a generalization of Request-Reply, furthermore it is a kind of message that is exchanged by multiple participants.

Dragos Manolescu in his paper "Workflow Enactment with Continuation and Future Objects" he writes:

Over 20 years ago Kowalski argued that separating the what, which specifies the “knowledge to be used in solving problems,” from the how, which determines “the problem solving strategies by means of which that knowledge is used,” will make programs more readily adapted to new problems, thus improving modifiability.

In his paper he shows how the use of continuations reify the control flow. He describes the results as the "trampolined style":

The trampoline (i.e., a loop) drives the entire computation by bouncing from one continuation to the next. In the context of micro-workflow, the trampolined style replaces the chain of message sends with a single message send from the trampoline to the executing activity node.

David Chappel in his book "Enterprise Service Bus" writes about "message itineraries":

The details of the itinerary are stored as XML metadata and carried with the message as it travels across the bus from one service container to the next.

Conversations, trampolines, message itineraries are all the emodiment of the concept of a distributed continuation. If we examined the architecture of the Web, that is Representational State Transfer (ReST), and looked at the constraint of stateless servers:

We next add a constraint to the client-server interaction: communication must be stateless in nature, such that each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server.

Clearly, the client is driving control flow. In fact the client contains all the information required for the server to proceed. The client is in fact the trampoline.

We have now come full circle, in the architecture of the web the client is the container of the continuation. In ESB it is the message itinerary (aka the Conversation) that is the continuation. In workflow it is the trampoline (i.e workflow enactment engine) that is the continuation. All these technologies revolves around the reification of control flow.

The blind spot is that most attempts at abstraction try to hide control flow. That is why you have Object Oriented Programming where control flow is encapsulated. This is in fact interesting in that the main benefit of OO was to bring down the complexity of GUI development. This is done by patitioning the massive state space of a GUI. Individual objects have ownership of part of the state space, responding to changes in the form of event handlers.

The explicit definition of control flow is lost and is dispersed among multiple event handlers. This happens because the partitioning of state is based on presentational aspects. That is the objects in a GUI are Panel, Buttons, Fields etc. The objects of a reified control flow are Sequence, Loop, Condition etc. These partitionings clash and you end up with a structure favoring presentation over control flow.