The use of EHR data to reliably detect uncommon (but strong, discrete) signals from a single drug or treatment is itself a Medical Informatics “Grand Challenge.” An example would be finding out about VIOXX’s association with myocardial infarction earlier than we did, via an EHR-based automated postmarket surveillance process.

Doing this is a “grand challenge” due to the nature of EHR data, which is as far from “clinical trials clean” as possible. The statistical methods needed to reliably pull signals out of the muck for even a single drug are still exploratory, the problems formidable if one wants to stay scientifically sound.

To detect relatively more nebulous “outcomes differences” between TWO OR MORE drugs or treatments via EHR data – did treatment A lower blood pressure more than drug B, did drug C lessen depression more than drug D – rises to the level of a “Grand Fraud.” It is near impossible to do with reasonable certainty from reams of EHR data, from different vendor systems, input by myriad people of different backgrounds with differing interpretations of terminologies (students/MD’s/RN’s etc) under different pressures (time, reimbursement maximization), and so forth.

Ominously, there is a lot of potential advantage to be had with terabytes of uncontrolled data and a political agenda. I fear that what will come from “comparative effectiveness research” that draws upon uncontrolled EHR data will be politics masquerading as comparative effectiveness research.

Ironically, the gold standard in medical science is the controlled clinical trial, yet EHR-based comparative effectiveness research itself as a research methodology, now touted by our government, seems to have gotten a pass.

Where are the meta-clinical trials that compare EHR data mining-based comparative effectiveness research as a methodology, vs. the “other” methodology of controlled clinical trials to compare drugs or treatments?

Good luck to private practitioners and medical innovators. Good luck, pharma. Good luck, patients.

This movement towards EHR uncontrolled data alchemy represents a further deviation from medical science towards the Syndrome of Inappropriate Overconfidence in Computing writ large.