Noise A Flaw in Human Judgment

Imagine a world where the very same case, presented to different experts, yields wildly divergent outcomes. A patient with identical symptoms might receive one diagnosis from Dr. A and a completely different one from Dr. B. Two judges, reviewing the same crime and evidence, could hand down sentences that vary by years. An insurance underwriter might quote one premium, while a colleague, looking at the exact same risk profile, offers another. This pervasive, often invisible, inconsistency in human judgment is what lies at the heart of the matter: noise.

Noise is not the same as bias. Bias is a consistent deviation in judgment, a predictable lean in one direction, like a scale that always reads a few pounds heavy. Noise, however, is the unwanted variability, the scattershot of individual judgments around an average. It's the difference in readings when you step on the same scale multiple times, or the discrepancies between multiple scales. It manifests as both "within-person variability," where the same individual makes different judgments under slightly different circumstances, and "between-person variability," where different people judge the same situation differently. This randomness, often overlooked because it doesn't tell a neat story or isn't easily attributable to a single cause, significantly degrades the quality and fairness of decisions across countless domains.

The impact of this pervasive noise is far-reaching and often severe. In the judicial system, it can lead to gross injustices, with similar crimes receiving vastly different punishments depending on the specific judge assigned. In medicine, it can result in incorrect diagnoses or treatment plans, jeopardizing patient health. Businesses suffer economic costs from inconsistent performance evaluations, hiring decisions, and strategic choices. The unsettling truth is that wherever human judgment is involved, noise is present, and likely in greater measure than one might intuitively believe.

Part of the challenge in addressing noise is its elusive nature. We are often more attuned to detecting bias because it appears as a systematic error. Noise, being statistical and random, is harder to spot in individual instances; it only becomes visible when examining an ensemble of similar judgments. Furthermore, there's a natural human inclination to believe that others reason as we do, leading us to underestimate the extent of variability in judgment. We crave definitive answers and closure, often settling for convenient explanations that feel right, rather than confronting the messy reality of random error.

Recognizing the problem is the first step, but what can be done? The path to reducing noise lies in what is termed "decision hygiene." These are preventative strategies, much like washing hands to prevent the spread of germs, designed to bring more consistency to judgment processes. This can involve breaking down complex decisions into independent parts, using relative rather than absolute judgments, and aggregating multiple independent forecasts to average out individual errors. Structured approaches, such as the Mediating Assessment Protocol (MAP), guide decision-makers through a series of steps to ensure a more disciplined and uniform thought process.

While complete eradication of noise in human judgment may be impossible – as long as judgment is involved, some variability will remain – significant reduction is achievable. The goal isn't necessarily to eliminate human intuition, but to discipline and delay its influence until a thorough and diverse body of evidence has been carefully considered. Organizations, in particular, have a strong incentive to design decision-making processes that speak with "one voice," enhancing fairness, credibility, and accuracy. By making a conscious effort to identify and mitigate noise, we can foster a world where decisions are more consistent, equitable, and ultimately, better.