Optimizing your System: Don't get Distracted by Individual Metrics

Several years ago, I had as great mentor who recommended the book The Goal, by Eliyahu Goldratt (also available as a graphic novel these days). I was chatting with someone recently about their team struggling, because their product team wasn't refining and prioritizing the backlog fast enough to keep up with developer capacity, which reminded me of this book.

The core of the book is on optimizing the right things and not get distracted by other things, such as how busy/productive every individual is in the organization. Instead, the focus should be on the primary goal, which for many companies would be financial performance. At a systems level, think about how fast value or work is flowing through the whole system, not how hard each individual is working. To optimize that overall value flow, Goldratt focused on looking at the whole system view. In the book, he illustrated this systems-level optimization through the story of story of boy scouts on a hike. When faster kids lead the hike, the group spreads out and then have to wait for the slowest hiker to catch up. Over time, the leader realizes he should put the slowest hiker (named Herbie) in front, and then further realizes he should move things from the slowest hiker's backpack to everyone else, to optimize the overall system speed.

When you read this, it's quite obvious. However, in the professional context, we often are very focused on individual activities instead of looking at the broader picture. Think about streams of work, and think about how to optimize the flow through the system.

The book focuses on five techniques to optimize the system, after you understand the system you're optimizing:

1. Identify the Constraint: Find the role, machine, task, or person with the biggest pile of work in front of it
2. Exploit the Constraint: Ensure the constraint is running as effectively as possible (e.g., remove other priorities from that employee)
3. Subordinate Everything Else: This is the hardest step for managers -- non-constraints must slow down to match the speed of the constraint. If they work faster, they just create pile-ups (instead, they could contribute to other projects or help the constraint)
4. Elevate the Constraint: If you still need more capacity, invest money in increasing the constraint's capacity (e.g., hire another person, outsource)
5. Repeat (Prevent Inertia): Once the constraint is fixed, it is no longer the bottleneck. The bottleneck has moved somewhere else. Continually repeat this process to further optimize.

Here's an old-school 9 minute video illustrating the lesson:

The Theory of Constraints was also a significant part of Gene Kim's great book The Phoenix Project, which applied this systems-level optimization work to modern DevOps-enabled software engineering (see a past blog post DevOps can be Interesting).