Scary Movies and Scary Graphs

I am not a big fan of scary movies. The way I see it, life is frightening enough without handing over my hard-earned cash to have the bejesus scared out of me. What I particularly hate about horror flicks is the feeling they give me of impotence — being unable to save someone from the doom that I know with complete certainty is about to arrive. I want to scream at the movie screen, “Stop! Don’t go in there! The lunatic hiding under the bed does not have your best interests at heart!” It never ends well — it is all too stressful.

I feel the exact same way when I see a report or graph that is clearly wrong due to some miscalculation or misalignment in the data. Just as in the scary movie, I know a bad scene is just waiting to happen and I am screaming, “Stop! Don’t go into your important meeting with that poorly-designed graph: this is not going to end well for you…”

Here is a recent, real-life example of what I mean.

I was looking over the graph below for a client. It’s meant to display the rate of appeals by patients to their insurance companies for a specific type of claim — one denied because the services were deemed “not medically necessary.” As I scanned the graph, my eyes immediately jumped to the blue bar for Q3 ’12, and I wondered how, with an “n” of four (4), the medical- necessity appeal rate could possibly be 133%.

medical-necessity-appeal-rate

Interestingly enough, the client had an answer at the ready: because of the time-lag caused by administrative processing, the insurance claims in the numerator are not the same as the claims in the denominator. For example, if nine (9) claims were denied in June, and the appeals on four (4) of them weren’t processed and finalized until October, and there were only three (3) claims denied in October, this report uses the denied claims in October (3) as the denominator and the number of appealed claims from June (4) as the numerator, resulting in a Medical Necessity Appeal Rate of 133% — which isn’t really possible. You can’t appeal more denied claims than you have.

Is your head spinning? Mine, too. The client seconded my confusion, and added that in fact she’d had a hell of a time explaining this to the audience at her presentation. I also got the impression that she felt the same way I did about the looming doom this graph presaged. After all, the group needed and expected reliable information from her-and she couldn’t provide it.

The lessons here are pretty straightforward:

  • Insurance claim data can be tricky because of the time lags we’ve noted. It may take a little more programming work to align the denied claims all the way through to adjudication, but there’s not much choice if you want to report the true appeal rate. And yes, you may as a result see a bit of a delay before you can graph all aspects of the data — that’s the price of getting it right.
  • If something looks funky on a graph, you should ask why. The person who most needs to understand the elements that go into the graphs and reports you present is you. It is always better to sit on them until you are sure they are correct and complete-and until you understand exactly what they are showing, and why.

Remember: the last thing you want to do is to walk into a meeting with a bad report. As for the victims in a scary movie, opening that door is not going to end well for you.

Share
This entry was posted in Communicating Data to the Public, Data Analysis, Graphs, Newsletters. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *


× 9 = eighty one