05-2-Tips.Rmd

# Tips and tricks for successful control chart use

## READ ME (or else)

- The definition of your control limits depends on the trade-off between sensitivity and specificity for the question at hand. Typical control charts are built on 3$\sigma$ limits, which provides a balanced trade-off between sensitivity and specificity, that is, between under- and over-alerting to an indication of special cause variation. When you need to err on the side of caution---for example, in patient safety applications---2$\sigma$ limits may be more appropriate, while understanding that false positives will be higher. If you need to err on the side of certainty, 4-6$\sigma$ limits may be more useful.   

- With fewer than 20 observations, there is an increased chance of missing special cause variation. With more than 30 observations, there's an increased chance of detecting special cause variation that is really just chance. Knowing these outcomes are possible is useful to help facilitate careful thinking when control charts indicate special cause variation.       

- Ensure your data values and control limits make sense. For example, if you have proportion data and your control limits fall above 1 (or above 100%) or below 0, there's clearly an error somewhere. Ditto with negative counts.    

- For raw ordinal data (such as likert scores), do not use means or control limits. Just. Don't. If you must plot a single value, convert to a proportion (e.g., "top box scores") first. However, stacked bar or mosaic charts help visualize this kind of data much better, and can be done in the same amount of space.      

- Control charts don't measure "statistical significance"---they are meant to reduce the chances of incorrectly deciding whether a process is in (statistical) control or not. Control limits are *not* confidence limits.       

- YTD comparisons don't work because they encourage naive, point-to-point comparisons and ignore natural variation---and can encourage inappropriate knee-jerk reactions. There is never useful information about a process in only one or two data points.    

- A control chart should measure one defined process, so you may need to create multiple charts stratified by patient population, unit, medical service, time of day, etc. to avoid mixtures of processes.       

- With very large sample or subgroup sizes, control limits will be too small, and the false positive rate will skyrocket. Use [prime charts](#prime) THIS LINK DOESN'T GO ANYWHERE instead.      


## When to revise control limits

If you need to determine whether an intervention might have worked soon after or even during the improvement process, you shouldn't be using a standard control chart at all. Use a run chart or an EWMA or CUSUM chart to try to detect early shifts.

When you have enough data points after the intervention (about 12-20), with no other changes to the process, you can "freeze" the median and/or mean+control limits at the intervention point and recalculate the median and/or mean+limits on the subsequent data. However, by doing so you are *already assuming* that the intervention changed the process. If there is no evidence of special cause variation after the intervention, you shouldn't recalculate the SPC chart values.  

Say that an intervention happened at the start of year 3, but there was a lag between the intervention and when it actually showed up in the data. 

```{r cc_trend, fig.height=3.5}
# Create fake data with change in process at 28 months
set.seed(3)
intervention = data.frame(Date = seq(as.Date("2006-01-01"), by = 'month', 
  length.out = 48), y = c(rpois(28, 6), rpois(20, 3)), 
  n = round(rnorm(48, 450, 50)))

# Plot control chart with break at intervention
qicharts::qic(y = y, n = n, data = intervention, multiply = 1000, x = Date, chart = "u", 
    runvals = TRUE, ylab = "Value per 1,000", main="", breaks = 24)
```

Of course, the change point can be placed arbitrarily in a `qic` graph---with corresponding changes in control limits. For example, using the same data as above, compare those results with those when the change point is moved forward by 2, 4, or 6 time steps (pretending we don't actually know when the process truly changed):

```{r cc_trend_2, fig.height=3.5}
# Plot control chart with break 2 months after intevention
qicharts::qic(y = y, n = n, data = intervention, multiply = 1000, x = Date, chart = "u", 
    runvals = TRUE, ylab = "Value per 1,000", main="", breaks = 26)

# Plot control chart with break 4 months after intevention
qicharts::qic(y = y, n = n, data = intervention, multiply = 1000, x = Date, chart = "u", 
    runvals = TRUE, ylab = "Value per 1,000", main="", breaks = 28)

# Plot control chart with break 6 months after intevention
qicharts::qic(y = y, n = n, data = intervention, multiply = 1000, x = Date, chart = "u", 
    runvals = TRUE, ylab = "Value per 1,000", main="", breaks = 30)
```

As you can see, the conclusions you could draw from a single control chart might be different depending on when the breakpoint is set.  

Use common sense and avoid the urge to change medians or means and control limits for every intervention unless evidence is clear that it worked.   DO WE WANT TO TALK ABOUT USING THE LOG2(n) + 3 RULE OR THE CROSSINGS RULE TO CHOOSE THE CUT-POINT?

SPC charts are blunt instruments, and are meant to try to detect changes in a process as simply as possible. When there is no clear evidence in SPC charts for a change, more advanced techniques---such as ARIMA models or intervention/changepoint analysis---can be used to assess whether there was a change in the statistical process at or near the intervention point.