By: DMKashmer, MD MBA MBB FACS
You may have learned about concepts like sensitivity and specificity regarding diagnostic tests. These useful concepts can be applied to many ways we make diagnoses in Medicine. In this write up, let’s briefly review the concepts of sensitivity and specificity as they relate to the identification of injured trauma patients.
Sensitivity is “PID”
First consider sensitivity. Sensitivity is the chance that a test is positive in disease. An easy way to remember sensitivity is as “PID”. PID, here, does not stand for Pelvic Inflammatory Disease. It stands for the chance that a test is “positive in disease”. A triage system can be looked at in terms of the probability that is positive in disease. By this, I mean consider that you can evaluate a triage system based on how sensitive it is to identify significantly injured (ISS greater than or equal to 16) patients. After all, isn’t that the point of triage?
If we consider a critically injured trauma patient to have an injury severity score of 16 or greater, the sensitivity of the triage system can be described as the probability that a full trauma team activation occurs for a trauma patient who has an ISS of 16 or greater. This would indicate the sensitivity of the trauma triage system as a whole for significantly injured patients. Of course, as with any methodology, this one suffers from the fact that we use injury severity score to determine whether the trauma patient is critically injured. Injury severity score is, of course, retrospective. This is always challenging because triage decisions are made in anterograde fashion with limited information and there’s always a certain signal to noise ratio. But hey–you gotta start somewhere. At least the ISS measure gives us that starting point.
Specificity Is “NIH”
Next, consider the specificity of a trauma system. Specificity can be remembered as NIH. NIH, here, does not stand for National Institutes of Health (or, worse yet, “Not Invented Here”). Instead, it stands for “negative in health”. The specificity of a trauma triage system may be regarded as the probability that it does NOT give a full team activation in patients who are NOT significantly injured. Again, if we consider that patients with an ISS of less than 16 are not critically injured, we would care about the probability that the trauma activation is not called in patients who have an ISS less than 16.
And Now To The Interesting Stuff…
The concepts of sensitivity and specificity can allow us to do some interesting things. We can create both the odds ratio positive and odds ratio negative for patients coming in the triage system. This would allow us to determine the probability that a patient entering the system with a certain pre-test probability of injury is classified appropriately as a trauma activation. We could also determine the probability that they will not be classified as a trauma activation. Remember, the odds ratio positive is defined as the sensitivity of a test over the quantity one minus its specificity (end quantity). That looks like:
OR+ = SN / (1-SP) where OR+ is the odds ratio positive, SN is sensitivity, and SP is specificity.
The odds ratio negative is defined as quantity one minus the sensitivity (end quantity) divided by the specificity. Here’s that one:
OR – = (1 – SN) / SP, where OR – = odds ratio negative, SN = sensitivity, and SP = specificity
Therefore, the more specific the test, the smaller the odds ratio negative and smaller the resultant odds of a patient being critically injured given system activation. This is one useful conceptual way to make anterograde decisions or to conceptualize trauma.
Using Our Numbers To Make Decisions
Let’s pretend, for example, a trauma triage system has a 25% sensitivity for labeling severely injured trauma patients, and has a 98% specificity for labeling patients who do NOT have significant traumatic injuries as people who should NOT have activations. Therefore, the odds ratios:
OR+ = 0.25 / (1 – 0.98), or 12.5
OR – = (1 – 0.25) / (0.98), or 0.76
Now let’s pretend EMS brings a patient, and the person determining whether to activate the system hears a lot of story and is unsure about whether to activate. “Eh,” they say “it’s a 50 / 50 chance. I could go either way about whether to activate.” Let’s use our triage-as-a-test to see what happens in the scenario as we make anterograde decisions:
First, we assign a 50% probability to the likelihood that the patient has significant injures. After all, in this example, the person deciding whether to activate thinks it could “go either way.” This 50% is sometimes called the prettest probability.
To use the odds ratios, we need to convert the probability into odds. Probability is converted to odds by the formula:
p / (1-p) = odds, where p = probability.
Here, that’s 0.50 / (1-0.5), or 1. So that’s the “prettest odds”. (The odds something exists before we run the test.)
Next, let’s pretend that the triage system / info / criteria / whatever-we-use says “Don’t activate!” So we use the OR- to modify the prettest odds:
(1)(OR-) = posttest odds; here, that’s (1)(0.76) or post-test odds of 0.76.
Last, we convert the posttest odds back to probability. Probability is converted to odds by the formula:
o / (o+1) = probability
So, here, that’s (0.76 / 1.76) or 43%!
So, wait a minute! The patient had a 50% chance (in the triage person’s mind) of being significantly injured before the triage test / criteria were used. Now, after the triage system said “Don’t activate!” that patient still has a 43% chance of being significantly injured. Is that a useful triage system? Probably not, because it doesn’t have the ability to change our minds…especially about patients who may be significantly injured!
What lets us do that math? It’s called Bayes’ Theorem, and it introduces how to use test information to modify probabilities. Conditional probability is an interesting topic–especially when it comes to triage.
Next, let’s discuss a few more interesting ways to measure our triage system.
One interesting way to measure triage system is how often critically injured patients are not activated as full team traumas. This is one measure of “undertriage”. To use this measure, we would review the total number of trauma patients who did not have a full trauma team activation and who were significantly injured with an ISS over 16 and we would divide by the total number of patients who came through the system with an ISS over 16. This would demonstrate what probability, given critical illness, we had of identifying the patient improperly. Said differently, this answers the question, “What proportion of significantly injured patients were not identified by our trauma system?”
Another interesting method is the matrix method, or Cribari grid. The grid answers the question “What percentage of patients, among those who were not full trauma team activations, were severely injured?”
Both methods point to the same ultimate issue: a type of error called undercontrolling. This type two statistical error indicates the risk of NOT adjusting or under-recognizing something even when, in fact, that thing exists. The concept of under controlling can be challenging to completely understand. Consider the picture below. It always helps me.
Therefore, when we talk about undertriage in systems, it is important to determine which measure we are going to adopt and how we are going to measure ourselves going forward. In new trauma systems, or minimally developed trauma systems, there are sometimes challenges with application of the Cribari grid. Trauma registries and other trauma data sources may not fully and adequately capture all trauma patients as some may be shunted to medical services or there may be other difficulties. It’s important to work with physician colleagues and the entire team to learn what the triage measures mean and how different ones apply.
For more information regarding concepts in triage including type two errors and thoughts on trauma triage, look here.