Deming’s Wisdom for Staff+ Engineers: A Recap of My Talk at StaffPlus London 2024

• August 25, 2024

In June 2024, I had the privilege of travelling from Auckland, New Zealand, to speak at StaffPlus London - an unforgettable experience. Speaking at such a high-profile event, surrounded by so many talented Staff+ Engineers, was truly special. What made this event particularly memorable wasn’t just the prominence of the event but the opportunity to share something that has become deeply important to me: how the timeless teachings of W. Edwards Deming remain as relevant today as ever in the constantly evolving world of technology.

Stepping into a Distinguished Engineer role last year pushed me to explore leadership in new ways. It was during this time that I discovered Deming’s work, which provided me with clarity and direction. His insights have shaped how I think about creating environments where engineers can thrive and continually improve.

For those interested, my slides from the talk are available here, and you can find the video of the talk here. It may still be paywalled, although I hope it will be accessible soon. In the meantime, below is a summary of the key points I covered during the talk.

Why Deming?

W. Edwards Deming, known as the father of Total Quality Management, is best known for his work in post-war Japan, where his teachings on quality and systems thinking transformed industries. Deming wasn’t merely concerned with improving processes; his focus was on changing how we think and lead.

For me, discovering Deming’s work felt like finding a key piece of the leadership puzzle. His emphasis on continuous improvement and pride in workmanship deeply resonates with my own values. But what really struck me was how much Deming’s views on leadership aligned with what I had already believed. One of his quotes that I agree with wholeheartedly is, “To manage one must lead. To lead, one must understand the work that he and his people are responsible for.” This idea is central to effective leadership – ensuring there is a deep understanding of the work happening across the organisation and the challenges engineers face. Curiosity plays a vital role here too. By staying curious, continuously asking questions, and exploring how things work, one is better equipped to provide the right guidance and support. Deming’s philosophy reinforced this, showing that leadership is about more than oversight; it’s about being in tune with the system, the people involved, and the processes that drive improvement.

Deming to DevOps

Deming’s influence can be traced through modern methodologies such as Toyota’s Production System (TPS), Lean Manufacturing, Agile, and DevOps. His teachings laid the foundation for continuous improvement, reducing waste, and optimising processes, all of which are central tenets of Lean and Agile. In the software world, these principles evolved further into DevOps, where collaboration and system optimisation are critical for improving quality and speed.

Deming’s System of Profound Knowledge

At the heart of Deming’s philosophy is his System of Profound Knowledge, which consists of four key areas:

1. Appreciation for a system – Recognising that organisations are socio-technical systems, where nothing happens in isolation. Changing one part can have unintended consequences elsewhere, so we must optimise systems holistically.
2. Knowledge about variation – Distinguishing between common cause (inherent in the process) and special cause (due to external factors) variations is key. Common cause issues require systemic changes, which fall under management’s responsibility.
3. Theory of knowledge – Questioning why we know what we know, grounding actions in facts and context. Operational definitions ensure clarity and consistency in our understanding.
4. Psychology – Refers to the human aspects of work. Intrinsic motivators and purpose are essential for creating productive and enjoyable work environments.

For Staff+ Engineers, these concepts are invaluable when tackling complex systems. Whether we are improving processes, enhancing collaboration, or driving innovation, Deming’s principles guide us toward a more holistic understanding of the challenges we face.

Deming’s 14 Points for Management

Deming’s 14 Points for Management serve as a practical guide for leaders striving to create better systems and environments. While all the points are valuable, a few stood out to me as especially relevant for Staff+ Engineers:

1. Remove barriers to pride in workmanship – Empower engineers to take pride in their work by eliminating obstacles that hinder creativity and innovation. When people enjoy their work, it drives both personal satisfaction and organisational success.
2. Drive out fear – Build a culture where open communication is encouraged, and people feel safe raising issues without fear of blame. This is closely related to the modern concept of psychological safety, which is crucial for fostering innovation and a supportive work environment.
3. Break down silos – Promote collaboration across teams to ensure smoother workflows and better outcomes. When silos are broken down, organisations become more agile and effective.
4. Institute training and self-improvement – Continuous learning is essential, not just for individual growth but for the long-term success of teams. Investing in self-improvement helps teams adapt and thrive in an ever-changing landscape.

Practical Applications in Modern Software Engineering

Deming’s teachings are directly applicable to modern software engineering practices. In my talk, I explored three key areas where these principles can be applied today: Shift Left, Site Reliability Engineering (SRE), and Developer Productivity.

Shift Left

Shift Left involves moving security and quality testing earlier in the software development lifecycle to detect issues sooner and reduce costs. Deming’s appreciation for a system fits perfectly with this approach—by integrating quality into the development process from the start, we reduce the need for inspection and rework later on. As Deming said, “Quality does not just happen: it is built in.” Adopting the new philosophy is also critical for embracing this shift, as it requires a mindset shift across the organisation. Everyone must be on board with the idea that quality is a shared responsibility, not something that can be inspected into a product, especially not at the end of the process. This mindset change helps ensure that quality is built in from the beginning rather than flung over the fence to dedicated test teams.

Site Reliability Engineering (SRE)

SRE applies software engineering principles to IT operations, aiming to create scalable and reliable systems. Deming’s belief that “A bad system will beat a good person every time” is particularly relevant to SRE. When incidents occur, it’s not about individual failures but about systems allowing those failures to happen. Knowledge about variation is key here—understanding the different types of variation (such as common cause vs. special cause) allows us to address system issues effectively and improve reliability. Theory of knowledge also plays a crucial role. In SRE, SLOs are operational definitions that help us to measure system performance through automated monitoring and alerting, allowing for data-driven insights. This approach ensures that we’re continuously learning and improving based on real data rather than assumptions.

Developer Productivity

Improving developer productivity has become a major focus in modern engineering teams. As Deming said, “It isn’t enough to do your best; you have to know what to do, and then do your best.” This aligns with his theory of knowledge, which emphasises the need for data-driven decisions. To improve productivity, we need to measure the right things and understand them within the context of our systems. Additionally, psychology plays a critical role—factors like job satisfaction, motivation, and psychological safety all directly impact productivity. By creating environments where developers feel valued and supported, we can significantly boost both productivity and morale.

Final Thoughts

Speaking at StaffPlus London 2024 was one of the highlights of my year. Travelling from New Zealand to such a renowned event and sharing my insights alongside experienced technical leaders was an incredibly rewarding experience. I am so grateful for the opportunity to explore how Deming’s timeless teachings continue to resonate in our modern world and how they can help us lead more effectively as Staff+ Engineers.

Reflecting on my relatively recent journey with Deming’s work, I’m reminded of his unwavering belief in continuous improvement and the pursuit of knowledge. For those of you in Staff+ roles, I encourage you to embrace systems thinking, foster collaboration, and always keep learning. As Deming famously said, “Learning is not compulsory; it’s voluntary. Improvement is not compulsory; it’s voluntary. But to survive, we must learn.”