This heading is a popular quote from Deming. Like Deming, early quality leaders were excellent data-driven mathematicians. However, Berger and Mandelbrot studied large data sets of error clustering in telephone circuits in 1963. They discovered that the mean and variance of defect data do not converge as the sample size increases—the exact opposite of what statistics claim to predict. Although predictable convergence forms the theoretical basis of SPC, actual testing demonstrates that this is not true.
So how come SPC (and its latest incarnation, Six Sigma) evolved as a method that was never validated? Well, consider the challenge of doing that, even for a single process. Millions of repetitions of that single process would be required, plus every single part made by that process would have to be individually and flawlessly inspected. No one was ever able to do this and yet SPC was adopted as though they did. Despite his abiding commitment to SPC, Juran himself recognized that there were special cause Red-X events that cannot be predicted by a variation model—we call them mistakes. Hence a correlation between SPC theory and outcome can never be validated. Because of these problems, the need for a radical new approach was recognized.
While doing my seminal research in product design at Stanford, one question kept surfacing: What can be done during the design phase that will make a significant difference in product quality? Recognizing the shortcomings in validating statistical methods just described, and the serious error in using small data sets, I collected large samples of defect data from stable production lines that made millions of products a year. I discovered that most of the things that we think are important to quality are not. For example, most of us think that automation ensures fewer defects. I specifically tested that assumption, but discovered that automation did not make much of a difference in eliminating defects. Defects in an automated process are low only when that process is set up correctly. But almost all automation is set up incorrectly from time to time, generating a lot of scrap. On balance, automation produces a level of scrap and rework similar to manual operations. I tested more than 75 methods, and product and process attributes, to find which approaches improved quality the most. There were a lot of other surprises: Most quality improvement efforts don’t make much difference—except for two: the first of which is–simplify the task. We will address the second after simplification.
The impact of complexity on quality is demonstrated in Figure 1, where data points closely hug the best fit red and the black sloping lines (from a Disk Drive Company and Motorola, respectively), demonstrating a high correlation between defects per unit and assembly complexity (as measured by the total assembly time in seconds). The vertical data separation between companies reflects differences in the effectiveness of their quality methods, with the Motorola approach being the best.
Utah State University Research
Working with a team of MBA students, a large aerospace manufacturer shared the data for all defects recorded across hundreds of suppliers—the blue squares in Figure 2. This data covered over 320,000 defects addressed in roughly 10,000 quality reports. As in Figure 1, the sloping line fit to the data in Figure 2 shows that defect rates increase with increasing product complexity measured, in this case, by the average part price. The reports which linked every defect with its associated root cause analysis revealed even more: The terms “out-of-tolerance” and “distribution” never appeared in any report—and the term “variation” appeared only five times.
By contrast, the following terms repeatedly appeared: “wrong part,” “missing part,” “misrouted,” “misconnected,” “drawing error,” “design error,” “non-functional,” and so forth. Mistakes! Special causes (Red-Xs) were the major problem and no SPC distribution can predict those, let alone eliminate them. Virtually every single one of the 320,000 defects cited across 10,000 quality reports was caused by a mistake. The message is clear: You cannot make your company’s quality methods more effective without making mistake-proofing a top priority. Indeed, the two fastest ways to improve your quality are: 1) Simplify (product design/product tasks); and 2) Focus on preventing and controlling mistakes—through mistake-proofing.