The Wisdom of Being Wrong, Cont.: What we can learn from people trying to be ‘More Right’
This article is the second instalment in a series exploring our ability to make complex decisions. In our previous discussion, "The Wisdom of Being Wrong: Rethinking Certainty in a Complex World," we introduced the 'Less Wrong' and 'More Right' mindsets. Here, we examine the 'More Right' mindset in more detail, uncovering the lessons and pitfalls of striving for absolute certainty in a complex world. For those who have not yet had the opportunity, I encourage you to begin with the first part of the series [link], setting the stage for the exploration that follows.
Defining the 'More Right' Mindset
From Cobras to Poppy Fields: The Unintended Consequences of Simplification
How to make a plane?
In the heart of a bustling city, the presence of cobras becomes more than just a whisper of danger—it's a reality that people live with, day by day. As the city's British administrator, stories of citizens bitten by these venomous creatures reach your desk with increasing frequency. Determined to address this threat, you devise a plan simple in its logic yet bold in its ambition: offer a reward for every dead cobra, believing this incentive will motivate the populace to rid the city of its slithering menace. Yet, this plan backfires spectacularly as people begin breeding cobras to claim rewards. When the bounty program is abruptly ended, these snakes are worthless to the breeders and released, leaving the city with more cobras than ever before. This scenario gave name to the 'cobra effect'.
The cobra effect describes how there are often unforeseen consequences when we try to use oversimplified solutions to complex problems. Finding simple solutions to problems is of course one of the great joys of life. I for one loved it when I found out I could use a folded sheet of aluminium foil to sharpen my kids’ scissors (look it up :-) ). However, that is not what we are talking about when we consider the ‘cobra effect’.
Consider this story that unfolded in Afghanistan during the 00s. Opium production was rampant, and the (again) British administrators wanted to discourage farmers from cultivating the source of opiates, poppies. The initiative (again) offered financial incentives to farmers who would destroy their poppy fields. As we could have predicted, this led to an increase in poppy cultivation, as farmers sought to capitalise on the payments, thereby intensifying the very issue it aimed to solve. So, while the ‘cobra effect’ may seem obvious and easy to predict, it seems not that easy to avoid.
So how do these stories connect with the mindsets of being ‘Less Wrong’ and being ‘More Right’ that I have introduced in the previous article of this series? The mindstate of being ‘Less Wrong’ is about continuously making sense of an inherently complex world by trying to connect the dots of evidence we get when analysing the world. In contrast, the ‘More Right’ mindset is about trying to prove yourself right, about being sure we can reduce the complexity of the world into a small set of metrics and only look at those. And in that sense, the ‘cobra effect’ is an outcome of the ‘More Right’ mindset.
As it says in the definition, the ‘More Right’ mindset tries to reduce the complexity of the world into a small set of metrics. This reduction of complexity is an approach of its own called reductionism—the practice of breaking down complex systems into their constituent elements to understand and control them. Later on in this article I will further introduce the concept of reductionism, and how it lies at the basis of the great advances in science and technology that we have seen since the start of the Enlightenment.
So, if reductionism has brought us so many great advances, what’s the problem with the ‘More Right’ mindset? The 'More Right' mindset, while aiming for precision and control, often blinds us to the wider consequences of our actions. We will see the ramifications of overly simplistic solutions using examples like how Boeing can’t build safe aeroplanes using only economic growth metrics, and how social policies at Microsoft that fail to account for human complexity then fail to drive the innovation that the policies set out to achieve. These cautionary tales not only challenge us to question the metrics we choose to value but also remind us of the intricate web of cause and effect that governs our reality.
Not all is lost! As we see more examples of the ‘More Right’ mindset applied, and understand where it comes from historically, we will also start to see the essential balance between reductionism and a holistic view, revealing the path toward engaging more effectively with the multifaceted world around us.
The Historical Roots and Impact of Reductionism
Alchemy to Economics: Tracing the Reductionist Path
In my time at Booking.com, I witnessed first-hand the power of simplifying the world to a set of metrics. Our drive to understand customer behaviour through A/B testing provided clear, actionable insights that often led to significant improvements in user experience and business outcomes. The methodology underlying A/B testing is part of an approach called reductionism — effectively you look at the parts of a problem that you can analyse and understand thoroughly to increase your understanding of the whole.
Going back to the time of ancient Greek philosophers like Aristotle, it was common to try and understand the whole world as one. Aristotle didn’t only try to understand the elements of an object like a chair, he also claimed you couldn’t understand a chair unless you fully understood its purpose. Just before the Enlightenment this philosophy was behind the discipline of alchemy. A discipline that sought not merely to understand the material world but to transform it. Unlike the elemental categorization that dominates modern chemistry, alchemy operated on the premise of interconnectedness and transformation. Practitioners believed in the possibility of transmuting base metals into noble ones, most famously epitomised in the quest for the Philosopher's Stone—a substance thought capable of turning lead into gold and granting eternal life.
This alchemical worldview was deeply holistic, perceiving nature as a complex, interwoven tapestry where substances could change their essence through mystical processes. However, the pursuit of such grand transformations, while rich in symbolic and experimental inquiry that already started in the times before Aristotle tried to identify the ‘grand purposes’ of items, eventually reached its limits. The alchemical framework, with its enigmatic processes and elusive goals, proved too unwieldy for those seeking to understand the natural world in replicable, verifiable terms. This impasse nudged the intellectual tide toward a more dissective method, focusing on the parts rather than the presumed mystical whole.
Isaac Newton, a luminary in the realms of physics and mathematics, embodies this pivotal transition. Though now celebrated for his laws of motion and universal gravitation, Newton was equally immersed in the alchemical tradition. His extensive work in alchemy was at the basis of his scientific pursuit. He sought to uncover the secrets of nature. As Newton's endeavours progressed we observe a significant evolution in thought—from a mystical, interconnected view of the universe to a framework that prioritises understanding its constituent parts. This latter understanding is what led to Newton's scientific breakthroughs that laid the groundwork for classical mechanics.
The transformation from the holistic inquiries of alchemy to the focused lens of reductionism is a big change in the history of science. As we saw in the story of Isaac Newton the shift away from the holistic alchemical traditions led to the scientific breakthrough of writing down the laws that govern the physical world. This shift wasn't an isolated phenomenon. Contemporaries of Newton, including Robert Boyle and Antoine Lavoisier, were making strides in chemistry and physics by adopting reductionist approaches. They eschewed the quest for universal answers in favour of isolating and understanding the rules of specific phenomena. This era laid the foundations for modern scientific inquiry, demonstrating the power of reductionism in unlocking the secrets of the natural world. Yet, as our exploration into the 'More Right' mindset reveals, the legacy of this shift is double-edged, offering profound insights while also sowing the seeds for oversimplification's pitfalls.
The move towards reductionism, a trend not solely confined to the scientific domain, also made a significant impact on the corporate sphere. Let’s take a look at a time when a similar transition as happened in the transition from the times of alchemy to the Enlightenment happened in the business world. Dialogues on corporate social responsibility (CSR) were gaining traction well before the 1970s, with advocates arguing for businesses to play a constructive role in society—a stance purported to be not only ethically sound but also beneficial for business longevity. These discussions highlighted the notion that a business is an integral component of a larger societal ecosystem, advocating against the simplification of business performance to a solitary metric.
The publication of Milton Friedman's 1970 essay, 'The Social Responsibility of Business Is to Increase Its Profits', was the start of a pivotal shift in corporate philosophy. Friedman advanced the argument that the primary indicator of a business's success should be its proficiency in maximising shareholder profits. This perspective served to narrow the conceptual scope of a business's duties, marginalising the wider ramifications of corporate activities on societal and environmental spheres and laying the foundation for what I label as the 'More Right' mindset within the business landscape.
Championing a profit-centric framework, Friedman's stance encouraged a focus on short-term financial gains, often at the expense of considering the sustainability and ethical dimensions of business operations. In the half-century following the essay's publication, this viewpoint has underpinned corporate strategies characterised by aggressive cost-cutting and an overemphasis on short-term earnings, neglecting the complex interplay between a corporation and its broader ecosystem. This singular focus on profit as the metric for analysing a business's success is emblematic of the 'More Right' mindset under discussion.
Today, as businesses face increasingly complex challenges, including climate change and social inequality, there's a palpable shift in the discourse. There's an emerging consensus on the limitations of a narrowly focused, profit-only approach. This shift in corporate thinking is part of a wider intellectual transition from a reductionist to a more holistic understanding of complex systems, whether in scientific inquiry or in the domain of corporate governance.
Having traced the evolution of reductionism from its alchemical origins to the cornerstone of scientific inquiry and its impact on the business world, let’s look at some examples in the next section. Here, the principles of reductionism, championed by influential figures like Milton Friedman, manifest in strategies that prioritise profit maximisation, often at the expense of broader considerations. The experiences of Boeing and Microsoft in the next section, exemplify how a 'More Right' mindset, deeply rooted in reductionist thinking, shapes organisational cultures and decision-making processes, with profound implications for innovation, safety, and ethical governance.
Organisational Cultures Shaped by 'More Right'
From Engineering Excellence to Profit Primacy: The Corporate Shift
Remember Boeing? Maybe you have read reports about Boeing aircraft in recent years (between crashes and doors blowing out of planes) or seen a recent episode of ‘Last Week Tonight’ by John Oliver where he does a deep-dive into how Boeing got in the news so negatively. It wasn’t always so. In the mid-90s Boeing was without a doubt the company that was ruling aviation. Some of the reasons why Boeing was considered such a success is its culture of engineering and excellence. A quote from Bill Allen (CEO from 1945 to 1968 - in the years that Boeing designed the famous 747 airliner) shows what it was like to work at Boeing (taken from the book Built to Last):
“Boeing is always reaching out to tomorrow. This can only be accomplished by people who live, breathe, eat and sleep what they are doing…. [I am] associated with a large group of knowledgeable, dedicated [people] who eat, breath and sleep the world of aeronautics….. Man’s objective should be opportunity for greater accomplishment and greater service. The greatest pleasure life has to offer is the satisfaction that flows from… participating in a difficult and constructive undertaking.”
In the years when Boeing’s 747 was designed, a director asked a manager involved in the project what the expected returns on the investment were he was apparently told “they’d run some studies, but couldn’t recall the results”. (From Legend and Legacy: the story of Boeing and its People 1992). The Boeing 747 ended up completely redefining air travel, something that could never have been put into a simple ROI calculation.
In fact, Boeing had so many orders that in the 90s, they had bought one of their smaller competitors to be able to use their factories to make more orders. This competitor, McDonnell Douglas, not only brought its manufacturing capabilities but also a markedly different corporate culture, one deeply influenced by the economic principles of Milton Friedman. McDonnell Douglas had a culture of focusing on shareholder value, a perspective directly echoing Friedman’s 1970 essay, 'The Social Responsibility of Business Is to Increase Its Profits.' While McDonnell Douglas struggled to build planes that were as reliable — its DC10 model was the first to be fully grounded by the US Federal Aviation Administration after a fatal crash — the doctrine of maximising shareholder value prevailed in its strategic approach.
This philosophy became even more pronounced when, due to a series of leadership changes, the former CEO of McDonnell Douglas ended up leading Boeing. The new leadership, steeped in Friedman's reductionist view that a corporation's ultimate responsibility lies in maximising profits for its shareholders, steered Boeing into a 'value-based environment where unit cost, return on investment, shareholder return are the measures by which you’ll be judged' [from John Kay (www.johnkay.com/2003/12/10/boeing-and-a-dramatic-change-of-direction/ )]. Under this new regime, engineers were suddenly compelled to not only consider the aerodynamics and safety of their planes but also the stock price of the entire company during meetings. A single metric—shareholder value—became the most crucial parameter to optimise for, a direct manifestation of the 'More Right' mindset driven by Friedman's economic theories.
This shift towards prioritising shareholder value above all else culminated disastrously in the design and development of the Boeing 737 MAX aircraft. Under intense competitive pressure from Airbus, Boeing sought a cost-effective way to counter its rival's new offerings for the medium-distance market. Opting to modify an older model with more powerful engines rather than designing a new plane from scratch seemed the most financially prudent decision—a decision that prioritised short-term financial metrics over the long-term implications of safety, training, and regulatory compliance. This decision, made in the spirit of maximising shareholder value as Friedman advocated, led to the tragic crashes of two 737 MAX planes, a horrible reminder of the dangers inherent in a myopic focus on financial optimization at the expense of broader considerations like safety, innovation, and corporate integrity.
As this stark departure from Boeing's foundational ethos of 'reaching out to tomorrow' demonstrates, the embrace of Friedman's principles marked a critical turning point. While aiming to optimise for shareholder return, the company inadvertently compromised the complex, nuanced aspects of aircraft design and safety, underlining the vital need for a balance between financial considerations and the unwavering commitment to the principles of engineering excellence and innovation that once defined Boeing.
Another vivid illustration of the 'More Right' mindset's impact on organisational culture is found in Microsoft’s adoption of the stack-ranking system. This practice required managers to categorise their employees into a fixed hierarchy based on their performance, from top to bottom, effectively forcing a curve of winners and losers among team members.
This system was introduced in the 1990s, aiming to identify and reward the company's top performers while encouraging underperformers to strive for improvement. However, over time, this system began to erode the collaborative spirit that had driven Microsoft’s innovation. As employees were pitted against each other for the top spots, the incentive to work together, share ideas, and take collective risks diminished. Innovation suffered as employees focused more on competing with their peers than on exploring new ideas and solutions.
Critics of the stack-ranking system argue that it exemplified a 'More Right' approach by attempting to quantify and optimise employee performance through a simplistic, linear metric. This not only overlooked the complex, multifaceted nature of individual contribution but also ignored the synergistic potential of collaborative effort. The system's focus on individual rankings failed to account for the variability and interdependence inherent in creative, innovative work.
The backlash and negative press around this practice grew so strong that by 2013, under CEO Satya Nadella's leadership, Microsoft announced the end of stack ranking. Nadella’s move towards a more collaborative and growth-oriented culture was widely seen as an effort to move away from the 'More Right' mindset, recognizing that fostering an environment where employees feel valued and empowered to innovate was critical to the company’s future success.
Both Boeing's shift in focus towards shareholder value under the influence of Friedman's economic theories and Microsoft's stack-ranking system illustrate the 'More Right' mindset in action. These examples underscore how an overriding emphasis on quantifiable outcomes—whether financial metrics or employee performance rankings—can lead to decisions that overlook the complex interdependencies and human factors essential for long-term success and ethical integrity. This mindset, while seeking definitive correctness and control, often simplifies complex realities to their detriment.
Matching Complexity
The Essential Balance for Decision-Making
Imagine hosting a dinner party where you've invited a diverse group of guests, each with their unique dietary preferences and restrictions—vegetarians, meat-lovers, gluten-free, and nut allergy sufferers, to name a few. To ensure the success of your dinner party (as measured by the availability of a certain dietary preference for your guests), you decide to offer a variety of dishes that cater to all these different needs. This way, regardless of a guest's dietary requirements, they'll find something at the table they can enjoy.
This approach to dinner planning mirrors Ashby's Law of Requisite Variety, which states that for a system to effectively handle the diversity of challenges it faces, it must possess an equal or greater variety of responses. In the context of your dinner party, the system (the menu) must be as varied as the dietary needs of your guests to ensure everyone's satisfaction. Just as a more varied menu can successfully accommodate more dietary preferences, a system with more internal complexity and options can better navigate and control the variability of its environment. Thus, the success of your dinner party hinges on the variety of your menu.
In all the real-world examples of the ‘More Right’ mindset in action we have mentioned before, the complexity of the world has been flattened into a single metric or set of metrics. These metrics have been assumed right from the outset, and then executed against. What actually happens when we do that? Let’s take a look at some of the foundational research into decision-making and problem-solving. We have already introduced the concept of Ashby's Law of Requisite Variety. This principle, articulated by cybernetician W. Ross Ashby, posits that for any system to be effective in managing its environment, the variability within the system must be equal to or greater than the variability within the environment itself. In other words, a system's internal complexity must match the complexity it seeks to navigate and control.
Ashby's Law of Requisite Variety offers a foundational insight into the limitations of the 'More Right' mindset, particularly evident in the complex domain of data science. By articulating the necessity for a system's internal complexity to at least match that of its environment, Ashby's Law underscores a critical flaw in only seeking overly simplistic solutions.
Let’s say you are a data scientist who is asked to build a model to predict the behaviour of customers in the future. The goal is to distil vast, chaotic customer behaviours into a neat, comprehensible algorithm. The allure of precision drives us to refine these models to produce the most accurate predictions possible. Yet, as Ashby's Law illuminates, the unpredictable nature of customers demands a level of adaptability and resilience that a narrowly precise model cannot provide. The real challenge lies not in achieving precision but in capturing the essential variability of the customer.
In my own experience, this disconnect between model capability and real-world complexity frequently emerges. As a data scientist I used models and was aware of the models' limitations. These models were tools with specific, constrained applications. However, once these models enter the broader organisational ecosystem, they are sometimes interpreted as definitive guides for decision-making, applied without due consideration of their underlying assumptions or the variability they fail to capture.
This misapplication illustrates a core issue with the 'More Right' mindset: it seeks validation rather than exploration, focusing on data points that confirm a predetermined view rather than challenging it or seeking a fuller understanding. It's a mindset that strives for certainty in a world inherently resistant to it, mistaking the map for the territory.
Yet, recognizing the limitations of reductionism does not diminish its contributions. The reductionist approach has propelled scientific and technological advancements, offering clarity and understanding where there was once only mystery. However, the key lies in reconnecting these insights to the larger whole, in acknowledging that while the parts may be comprehensible, the whole is more than the sum of its parts. In a world defined by its complexity, Ashby’s Law reminds us our approaches must reflect the rich tapestry of variability that characterises real-world systems. Embracing this complexity, rather than seeking to oversimplify it, may lead us to more robust, resilient, and ultimately successful strategies.
Beyond the 'More Right' Mindset
Embracing Complexity: The Path Forward
As we have explored through historical anecdotes, scientific evolution, and modern corporate narratives, the reductionist approach that seeks to simplify complexity into quantifiable parts—has undeniably shaped our world. The ‘More Right’ mindset shows up when we rely only on reductionism, and never connect these outcomes back to the larger picture. The limitations of this approach are evident in its outcomes. The cobra effect and the poppy fields of Afghanistan serve as cautionary tales about the unforeseen consequences of actions driven by overly simplistic solutions to complex problems. The Boeing 737 MAX tragedy underscores the dangers of prioritising financial metrics over the holistic considerations of safety and engineering integrity. These examples highlight a critical oversight: the failure to account for the interconnectedness and inherent unpredictability of natural and social systems.
Ashby's Law of Requisite Variety further illuminates this issue by emphasising the need for internal complexity within systems to match or exceed that of their environments. This principle challenges the 'More Right' mindset by advocating for models, strategies, and organisational structures that embrace adaptability and variability—attributes essential for navigating the complexities of the modern world.
In my own experiences at Booking.com, the focus on quantifiable metrics through A/B testing revealed the double-edged sword of reductionism. While providing clear, actionable insights, this approach obscured the broader context, leading to solutions that lacked sustainability or deeper understanding. It led to teams wanting to prove themselves ‘More Right’, and continue with what they were doing, instead of being curious with what the customer actually needed. In the same vein, businesses following Friedman’s original argument to maximise profit for shareholders, like the example of Boeing, will eventually falter if they ignore how they are part of a wider ecosystem.
There is reason for optimism though. I believe the ‘More Right’ mindset is based on a misunderstanding, one that implies we need to stop once we understand the parts or once we have optimised the right metric.
This realisation underscores the necessity of moving beyond a purely 'More Right' mindset to embrace a more holistic view—one that acknowledges the value of understanding parts but recognizes the imperative to reintegrate these insights into a comprehensive understanding of the whole. And this is exactly what I witnessed in my experience at Booking. Teams that used experimentation as just another data point, that gathered evidence, and used the opportunity of conflicting evidence to start asking new questions, they built products that customers cared about.
The reductionism that led to the 'More Right' approach has indeed propelled us forward, but it is the integration of these parts back into the whole that will guide us through the complexities and uncertainties of the future. As we look ahead, it is crucial to balance the precision and clarity offered by reductionism with a 'Less Wrong' mindset that values exploration, adaptability, and a systemic understanding of our interconnected world. In my next article, I will delve deeper into the 'Less Wrong' mindset, exploring how it diverges from 'More Right' thinking, and how reductionism in the right context can provide a more nuanced framework for problem-solving and decision-making in an increasingly complex world.
Article Series: The Wisdom of Being Wrong
This article is part of a series of articles.
Part 1: The Wisdom of Being Wrong: Rethinking Certainty in a Complex World
Part 2: The Wisdom of Being Wrong, Cont.: What we can learn from people trying to be ‘More Right’ (this article)
