Measuring data reliability for preventive services in electronic medical records (BMC Health Services Research)

Background: Improvements in the quality of health care services are often measured using data present inmedical records. Electronic Medical Records (EMRs) contain potentially valuable newsources of health data. However, data quality in EMRs may not be optimal and should beassessed. Data reliability (are the same data elements being measured over time?) is aprerequisite for data validity (are the data accurate?). Our objective was to measure thereliability of data for preventive services in primary care EMRs during the transition to EMR. Methods: Our data sources were randomly selected eligible patients` medical records and data obtainedfrom provincial administrative datasets. Eighteen community-based family physicians inToronto, Ontario that implemented EMRs starting in 2006 participated in this study. Wemeasured the proportion of patients eligible for a service (Pap smear, screening mammogramor influenza vaccination) that received the service. We compared the change in rates ofselected preventive services calculated from the medical record audits with the change inadministrative datasets. Results: In the first year of EMR use (2006) services decreased by 8.7% more (95% CI 11.0%-6.4%, p < 0.0001) when measured through medical record audits as compared withadministrative datasets. Services increased by 2.4% more (95% CI 0%-4.9%, p = 0.05) in themedical record audits during the second year of EMR use (2007). Conclusion: There were differences between the change measured through medical record audits andadministrative datasets. Problems could include difficulties with organizing new data itemprocesses as well as continued use of both paper and EMRs. Data extracted from EMRs hadlimited reliability during the initial phase of EMR implementation. Unreliable data interfereswith the ability to measure and improve health care quality.