Control Charts – What Are You Controlling?

I’m an enormous fan of W. Edwards Deming and his classic quality-management book, Out of the Crisis. In addition to referring to it several times a year, I also often present this book to young friends and family members graduating from college along with a handwritten note that says, “I know this appears to be a snooze of a gift, but you’ll thank me later — I promise!”

Deming was one of the great leaders in U.S. American thinking about how to change management practices to improve the quality of products and services across all industries. He was also a very enthusiastic proponent of Shewhart’s Statistical Process Control (SPC) techniques. In recent years, the healthcare industry has embraced many SPC concepts and methods, and that is a truly terrific trend — with one big exception. As much as I admire and subscribe to Deming’s and SPC theories and practices (and I do), I am deeply alarmed at what I see as a fundamental and widespread lack of knowledge on the part of healthcare professionals about how to use control charts to improve the quality of healthcare.

When I see a control chart for mortality or pneumonia or urinary tract infection rates (for example) my first question is always, “Can you describe to me the exact process you are controlling for — that is, monitoring?” To date, without exception, I have not encountered anyone who can articulate a clear, compelling, and comprehensible answer to this key question.

You see, the primary purpose of displaying data according to SPC techniques is to identify when a process is exhibiting strange or unusual behavior, so that you can make the crucial distinction in evaluating it between common-cause variation or trouble (faults in the system) and special-cause variation or trouble (faults from fleeting events). Identifying the type of variation (common v. special) is central to deciding what processes to correct to achieve the desired improvements.

Here is an example of a control chart for mortality that is often presented to me:

mortality-control-chart

This chart shows that the average mortality rate for this hospital is around 2.2%, with a lower limit of about 1.1% and an upper limit of about 3.1% (note— there is no information on this chart about what type of rate this is — perhaps per 1,000 discharges?). It also shows that, over time, the rate has both increased and decreased within these control limits. Unfortunately, this is where the investigation almost always ends. Groups see an average, and corresponding control limits and think, “we’re done.” Either they figure there is not much more to discuss (the results are within the control limits, right?), or they have no clue about what the next set of questions and actions should be. Result? Nothing happens — and there is no improvement.

If people really understood how to use the information presented here, they’d ask some useful questions to investigate further, including but not limited to:

  • Is this data risk-adjusted for patient characteristics and co-morbidities? (If not, then it’s of no real use — full stop.)
  • Assuming this is risk-adjusted data, and because the rates over time are within the upper (3.1%) and lower (1.1%) control limits, it would appear that this hospital may have some degree of common-cause (faults in the system) variation. Shall we explore that?
  • Do we have a process by which we identify the patients with the highest risk of dying? What do we then do with that information?
  • Do we have processes in place for how these high-risk patients are managed throughout the continuum of care?
  • Does everyone know and understand these processes? Are they consistently followed?

Unfortunately, I almost never hear these types of questions about process control.

No matter how useful and effective SPC techniques are, they cannot solve all problems; and they must be applied wisely, because there is no shortage of ways that using them can go wrong. Statistical process control charts are designed (it seems over-obvious to state, but is ignored by many) to tell us something about our processes — many of which don’t as yet exist. (You know it’s true.) Until there is actual process to analyze, we may be just as well served by a simple line chart that displays performance over time, or “observed” versus “expected” mortality ratios (for example), while we put in the hard effort necessary to understand our patient populations, establish and define our processes, and become more skilled and confident in understanding and using process control charts for quality improvement.

Oh, and a copy of Deming’s book is an absolute must-have. You’ll thank me later — I promise.

Share
This entry was posted in Data Analysis, Decision Support, Newsletters, Statistics (is not a dirty word). Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *


× one = 4