Balancing the Heat: System Dynamics and Backpressure in Software Queues
With a sprinkling of Elixir for good measure
In the 1950s, MIT professor Jay Forrester introduced System Dynamics, a framework for understanding how complex systems—whether in industry, biology, or even software—function through the interaction of individual parts and feedback loops. He arrived at this concept, in part, through his work on servomechanisms, which use feedback to regulate systems much like a thermostat controls temperature. This same thinking can be applied to modern software queues, for example, with the help of Elixir’s GenStage, where balancing workloads and managing flow through backpressure are critical to keeping systems efficient.
As we interrelate System Dynamics with queuing, we’ll cover:
Stocks, flows, & rates
Delays & nonlinearity
Feedback loops: reinforcing & balancing
Backpressure
Now, let’s set the scene table!
The Kitchen as a System
Imagine a restaurant kitchen. Orders come in from the dining room, and each chef or station (similar to stages in GenStage) is responsible for a part of the cooking process. The key to running an efficient kitchen, just like a software queuing system, is managing how orders flow through it and ensuring accumulated orders don’t bog it down.
In System Dynamics, we think about such systems in terms of stocks, flows, rates, and feedback loops.
From Order to Plate: Stocks, Flows, and Rates
In our kitchen, the stock represents all the orders waiting to be cooked. These are equivalent to tasks or events sitting in a queue, waiting to be processed. The flow represents orders moving from one cooking station to another. And the rate represents how quickly these orders transition from station to station.
From a System Dynamics perspective, when a queue is under heavy load, delays build up when the inflow (demand for new orders) is higher than the outflow (completed dishes) and so the stock (queue) grows. Managing this inflow is critical to avoiding show-stopping delays.
Under Pressure: Delays and Nonlinearity
In both kitchens and software systems, delays happen when tasks take longer than expected to process. For example, if too many orders arrive too quickly, the time it takes for each dish to reach the table increases, leading to unhappy diners. In other words, as orders pile up, stations in the kitchen become bottlenecks, creating a feedback loop which slows the entire workflow. This is an example of nonlinearity: a small increase in work can lead to a disproportionate slowdown, unless there's a way to manage the rate of flow.
Let’s explore nonlinearity feedback loops during the lunch rush!
Handling the Rush: Feedback Loops and Backpressure
Left alone, nonlinearity introduced by delays contributes to a reinforcing loop which can quickly overwhelm the kitchen. As more orders pile up, delays increase, leading to slower service. This is the dreaded feedback loop that every busy kitchen—and every overloaded software system—seeks to avoid.
To manage this, backpressure mechanisms can be put in place to relieve the system of having to process a potentially overwhelming number of orders. For example, in the kitchen, this could be represented as waiters slowing down their rate of order taking, a shorter, quicker-to-prepare menu may be provided to new diners, or other employees could even help out in the kitchen! All of these options help relieve demand on the kitchen, our stand-in for a queue.
This balancing loop, one that takes advantage of backpressure strategies, helps keep the system stable, avoiding the long delays that come from overwhelming the queue. Elixir’s GenStage, again, comes in handy here with its baked-in implementation for enabling demand to be communicated, much like a chef telling the waiters to reduce their order taking.
Queues Through a System Dynamics Lens
A queue is an essential tool for handling workflows at scale, providing enormous benefits. Paired with efficient tools like Elixir, it can accelerate outcomes in a well-built system. Even more, when combined with knowledge from other disciplines, like System Dynamics, you can gain a deeper understanding of data flow, feedback loops, and backpressure mechanisms to maintain efficient systems.
And you never know where System Dynamics thinking may come in handy—from kitchens, to cities, to software, to manufacturing!