How Process Orchestration Alleviates Automation’s Two Biggest Roadblocks
As part of Solutions Review’s Expert Insights Series—a collection of contributed articles written by industry experts in enterprise software categories—Bernd Ruecker, the co-founder and Chief Technologist at Camunda, explains how to use process orchestration to alleviate some of the most significant roadblocks automation faces.
One of the core benefits companies gain from digitization is the ability to generate efficiencies or gain a competitive advantage by automating business processes. Executives roundly agree on this issue. Recent automation market research shows 95 percent of executives surveyed say process automation drives more business productivity, and 93 percent say it improves business growth. Moreover, more than 90 percent say they plan to increase investments in process automation implementations over the next two years.
At the same time, companies are struggling to get all of their automation projects under control. Nearly three-quarters of executives in the same survey agree that real-world, mission-critical processes are difficult to maintain. While companies invest more in automation, they face challenges in automating processes end-to-end. They need more coordination of multiple automated tasks of very different natures. This can be achieved through new technologies that master the art of process orchestration. The biggest roadblocks to solving process orchestration are process complexity and endpoint diversity. We’ll explore each of these next.
Automating a simple sequence of steps isn’t rocket science, but very few business processes are, in reality, that simple. End-to-end processes are complex. They don’t follow a linear pattern; instead, they involve numerous tasks running in parallel, must wait for events or business conditions, or be able to interrupt activities at any time if necessary. Such processes are not only harder to implement but also hard to scope and define with multiple stakeholders.
The second obstacle in process automation is the variety of endpoints. Every process consists of a series of tasks that need to be orchestrated. Every job may require integrating a different endpoint—a person, team, or department, another internal or external system, IoT, and physical devices.
Integration of various endpoints is not only an integration challenge—it can also lead to more operational problems and complexity that need to be mastered and tamed. From a system perspective, while microservice architectures might solve some integration challenges, every enterprise must connect internal or external legacy systems for processes to be efficient. The same spectrum of complexity applies to human and device-related process integration and orchestration.
Process Orchestration: The Conductor Sets the Pace
As processes become more complex and more endpoints vie for attention, organizations need the ability to coordinate those processes holistically. They need to be able to handle all the technical challenges during execution, like dynamically synchronizing concurrent tasks, correlating events and messages or timeouts, and also adapting and managing the integration of various endpoints. This forms the backbone of process orchestration.
Process orchestration can be imagined as a digital conductor that sets the pace for all instruments (or people, systems, and devices) and tells them when and how to perform their tasks. Like a conductor in the orchestra needs a sheet of music, the process orchestrator also requires a process model. Like music sheets, this is best expressed in a standard language such as Business Process Model and Notation (BPMN), which is visual and understandable by stakeholders, but also directly executable by orchestration software. This allows for the most complex or sophisticated processes to remain cleanly managed and executed effectively at speed and scale.
Endpoints can be integrated via connectors: interfaces that standardize data exchange between the overall process and the endpoints. Connectors can be developed for almost any software system—whether programmed “out-of-the-box” by their publishers or custom-built—and allow the integration of legacy systems, microservices, and more. Connecting endpoints incorporates more than just systems, often including manual tasks and even interchange with connected devices.
When it comes to scaling business processes, optimizing them is imperative. Because even if one or the other inefficiency does not catch the eye on a small scale, it can make a significant difference on a larger scale. Which path is most commonly invoked by process instances? Where are the most significant delays? Bottlenecks and other weak points in a process can be identified particularly well using heat maps as part of process orchestration. What can or should be automated next? What gains can be made from integrating more related tasks?
The graphic above shows an ordering process in online retail modeled as a BPMN diagram. It may still look comparatively simple, but it already orchestrates many endpoints and contains some examples of advanced workflow patterns.
The task between placing the order and sending the invoice is carried out in parallel for each order item. Each of these parallel processes can prematurely terminate the entire process if, for example, an item is unavailable or the customer cancels the order. The time-based escalation after the invoice has been sent, which sends a reminder after ten days, is also an excellent example of a task that can be automated by orchestrating the process.
Coordinating processes without process orchestration is also possible with a lot of custom programming at the cost of great initial and even greater subsequent efforts. Maintenance, changes, and extensions of custom process logic will need to be carried out manually in multiple places and will be more error-prone overall. Conversely, process orchestration can continuously monitor processes to identify bottlenecks and performance issues.
Achieving Full Automation Potential
Many business processes require the cooperation of several different endpoints and are complex and dynamic in their logic. The mere automation of individual tasks is a step in the right direction, but this does not yet deliver the full automation potential. This is where process orchestration comes into play; like a conductor, it coordinates all the components in the execution of their tasks, sets the pace, and enables harmonious interaction from the beginning of a process to the end to drive significant efficiencies and fast gains.