By: DMKashmer MD MBA MBB (@DavidKashmer)
In healthcare, some process improvement systems have developed to a level where they use run charts or control charts to determine how well a system is performing. I have seen these charts outside operating rooms, administrator’s offices, and even hospital cafeterias across the country. Each location uses a control chart with the idea that the tool will show when the data (and the associated system) are out of control. That, however, is an over-simplified message which often leaves us to apply control charts improperly. Do you use control charts to follow performance? If so, beware and read on. Here are the major mistakes commonly made with control charts, and what you can do to avoid them:
(1) Choosing the wrong type of control chart for your data…
First, realize that the type of control chart you should use with your data varies with the type of data you have. For example, in healthcare, one of the most useful charts is the ImR chart. ImR stands for “Individuals Moving Range”. ImR charts are particularly useful when an individual patient or item moves through a system–just one item at a time. One of the first problems I see with the use of control charts is that staff often pick the wrong control chart and apply it to their data. Beneath, please find a simplified decision tree for how to pick the correct control chart for your data:
(2) Making the control chart before you have the context–especially the voice of the customer.
This next issue requires you to think a little bit about what the point of a control chart is. Remember, control charts are utilized so that you recognize when process values are out of control (beyond expectations for what system should be doing) or are tending to become out of control. The chart may highlight cases / values / outcomes that are out of control much more quickly than other techniques. Sometimes control charts can even tell us things like whether the central tendency of the process shifting.
Useful rules for the interpretation of control charts include the Westinghouse rules. For those rules, click Western Electric Rules.
With that said, you may think that a control chart tells you whether a process is being effective. Guess what? Control charts in no way tell you about whether a process is adequate, useful, or performing well. Control charts only tell you whether a process is in statistical control. A control chart will answer the question “Are these data within expectations for the process I have?”…and that’s all they do!
So, here is the headline: do not use control charts until you have improved your process as much as possible. Make sure your process satisfies the voice of the customer (VOC).
Beneath is a sample control chart. It shows that no individual case was out of expectation for the process. Sounds great, right?
Unfortunately, this control chart looks great…and demonstrates a process that is in no way adequate. Why? Now look at the figure below. This figure demonstrates that this process does not satisfy the VOC, or the requirements for the process output, at all. This brings up the interesting situation where a process is completely in control and yet wholly inadequate. We see this all the time in healthcare when run charts and control charts are misapplied. If you do it wrong, the data will look in control, the process will remain completely inadequate, and both you and your audience will be fooled!
The black line (and red VOC) in the ImR above makes the point: the control chart shows all data are in control (as if to say “nothing special here so go about your business”) and yet the process hums along on its merry way to making outcomes that are completely unacceptable. That’s the danger of mis-applying a control chart: doing so makes us miss the whole point.
So remember, before you apply a control chart, make sure you’ve improved your data and tried the best you can to become compliant with the voice of the customer or true system specification limits.
(3) Apply a control chart that’s based on the wrong data distribution.
The next issue deals with whether your data are normally distributed. Most control charts that I see commonly assume data are normally distributed. This is a classic foul. In fact, much data for health systems, including things like patient time in the emergency department, length of stay for patients, and many other examples are often non-normal data. Applying a control chart, with its assumption that the data are normally distributed, is a non-starter. We need to use control charts based on the distribution that we know the data follow. If the data are not normally distributed, and if, in fact, the data distribution is supposed to be a non-normal distribution, it makes no sense to apply a control chart that requires the assumption that the data follow the normal distribution. As a matter of fact, the misapplication of a control chart will mislead us. Garbage in, garbage out after all.
I hope you find your five minute review on classic problems with control charts useful. These ideas have kept me out of trouble as I’ve gone to apply control charts for statistical process control projects in healthcare. Remember, if you haven’t improved the process and placed in context already, it makes no sense to apply a control chart. The control chart will only tell you whether the process is performing within some typical zone and not whether it’s “good enough”. A control chart simply can’t tell you, when used improperly, that things are just fine.