By: DMKashmer, MD MBA MBB FACS (@DavidKashmer)
Do you remember Mark Twain’s three categories of falsehood? Mr. Twain described these as “Lies, damn lies, and statistics.” (I’ve also seen the quote attributed to Benjamin Disraeli.) Well, no matter who said it, the bottom line is clear: we need to be very careful with statistics. So, if you’re performing a quality improvement project for your system, what are the pitfalls of data analysis.
Just Having Data Is A Good Start, But Isn’t Enough
Up front, let me take a moment to compliment you, again, on even getting data for your quality project. Deciding to make decisions based on your team’s data rather than your gut or your own feelings will get you a lot farther down the path to success. Yes, your colleagues may be worried, initially, until you show them that the data in your project are not assignable to any one person. (It’s team and system performance–not individual based.) However, let me share with you that I’ve been in organizations which try to use their gut or feelings or some other whimsy to make decisions. Over time, you’ll come out way ahead with data…you’ll make constant improvement and you’ll be able to show those improvements over time. You’ll know if you’re doing better or worse. Not so with organizations that practice by whim or feelings. (Feelings have a real value, don’t get me wrong, but feelings without data are like lost children.)
So, congrats on even having data. But, my colleague, you need to go further to have a successful, high quality program: you need to analyze those data effectively (and correctly) to avoid basing your decisions on damn lies (!) So this brings us to the next step of a sound quality improvement project: analysis.
Pitfalls of Opening Pandora’s Box
You see, one of the pitfalls of making data-driven decisions is that you need to be able to correctly analyze the data…and that’s no easy task. Six Sigma practitioners are trained to use standard statistical tools to demonstrate the valid, meaningful conclusions you can make based on your data–and let me share with you that, prior to my training, I had no idea of what needed to be done to understand and demonstrate meaning from data. To my medical colleagues: yes, we take biostatistics classes and these make us conversant in techniques; however, going from sample design to data collection to meaningful conclusions is NOT what I’d seen in medical school or elsewhere.
A Few Tricks of The Trade
In reality, there are more than a few tricks to the trade. You’ve seen, in the links above, how important it is to decide whether you data are normally distributed (and what to do if they’re not). You’ve seen, again above, some of the relevant ideas about how to collect data (and what type of data) to make later analysis much more straightforward.
We’ve discussed, in earlier entries, the idea of what to do when data aren’t normally distributed. Take a look here.
Get Professional Help
With all that in mind, it’s no wonder people seek professional help! Allow me to recommend, here, that you either develop the in-house expertise necessary to analyze and obtain data effectively or you find someone who can. (Just email the team at firstname.lastname@example.org for our recommendations and ideas about where to go for more info.)
Some Parting Thoughts
If you’ve made it through the Define & Measure phase of your quality project, and you have data you’re looking to analyze, allow me to compliment you again. You’re miles ahead of what I’ve seen in some organizations, and are on your way to looking at yourself squarely and both characterizing your system’s current performance as well as improving it over time. Nice work–you’re miles ahead of others and miles further on the journey to excellent performance.
Now it’s time to focus on specifics of data analysis and some examples of how these tools come into play. Stay tuned for the next entry on data analysis with examples from projects gone by.
Questions, comments, thoughts? Let me know beneath.