IN BRIEF: Proper data validation is essential to successful implementation and rollout of healthcare reporting and analytics platforms. While the task of collecting and maintaining quality data can consume time and resources, the payoff is worth the investment. Poor data quality often results in missed care opportunities, confusion and frustration and even diminished care quality.
Health Care for the Homeless (HCH), a Baltimore-based “safety net” care provider, gets excited about data. Center staff – both non-clinical and clinical – went to extra lengths to ensure its Azara DRVS implementation included more than the usual amount of data validation, an examination of its data collection processes, a spirit of collaboration among a diverse staff and a willingness to accept an overarching truth: the effort to validate data properly will expose mistakes within the data, process deficiencies and other unforeseen challenges. But the reward for tackling these issues head-on is a health center with powerful, truly actionable data.
THE STORY: A medical student volunteering at HCH in 2014 made a surprising observation regarding the center’s tracking method for cervical cancer screenings: there were roughly thirty different ways to order a test through the EHR system and a half-dozen places to record the results. In addition, the center’s clinical staff insisted it was conducting more screenings than database records showed. A deeper look into the center’s data revealed the problem went beyond preventative screenings.
“There were some pretty large discrepancies,” said Chuck Amos, director of performance improvement at HCH. “Having done that analysis, seeing that depth of the problem and the amount of time it consumed, we got a critical mass within the organization to commit to implementing Azara DRVS.”
The center, which provides an array of medical and social services to more than 9,000 people in Baltimore and sites elsewhere in Maryland, has employed a practice management system since 2006 and added an EHR in 2010. The latter was hosted offsite by a Health Center Control Network (HCCN). HCH brought the system in-house in 2013, marking the first time the center completely controlled its patient data. It also ignited a realization that the center needed a more efficient method of managing and mobilizing its data.
In the fall of 2014, HCH chose Azara DRVS as its reporting and analytics software and assembled an implementation team. Amos led the day-to-day effort. His professional experience includes EHR development. Database support manager Irina Gayevsky was pulled in from existing HCH staff, as was Monica Orr, the center’s EMR system administrator. HCH also invested in new staff, hiring Rikki Ward, a clinical data analyst with a Master’s in Public Health (MPH) pedigree. Neither Gayevsky nor Orr report directly to Amos, but HCH leadership agreed to allow them to dedicate several hours per week to the DRVS project.
“This has been a challenge in timing and flexibility,” said Orr. “It required the buy-in of my supervisor to be flexible and supportive in making sure that all of the plates stayed in the air and nothing came crashing down.”
Having the process improvement and information technology teams on board is essential to the project, but not the only factor in its success.
“Another critical piece of the process is that the chief medical officer (Nilesh Kalyanaraman, MD) has been on board with the need for (DRVS) from day one,” said Amos. “He made it very clear to the provider staff that he expects them to be involved in the validation.”
Want Good Data? Test, Test and Test Some More
Amos jokes that he thought Azara was kidding when it requested data validation for only five patients for each of the key measures during the DRVS implementation.
“Five patients is not a statistically significant sample,” he said. “We have so many providers that enter data into the EMR in so different many ways, and we have extremely complicated patient cases because of the population we serve. There was no way five patients was going to give us a proper sample.”
Amos boosted the sample size to five patients for each of the staff members performing validation work. It still wasn’t where he wanted it, but he felt it would provide a reasonably accurate picture. Azara doesn’t think five patients is an ideal validation sample size either, but it is wary of asking time-strapped staff at health centers to work on a larger one.
“The reason we are so impressed with Chuck and his team is that they were hungry for more validation, as opposed to pushing back on the need to do it at all,” said Heather Budd, Azara’s vice president of clinical transformation.
Providers regularly challenge Azara on the need to validate every report; Budd said they must understand that they own the quality of their data, and it is only as accurate as the effort put into collecting and validating it.
“We’re laying the foundation for them with DRVS, but clients have to make sure the data actually reflects the care they deliver and how they document it,” she said. “DRVS can make access to data much more efficient, but it doesn’t mean you can take your hands off the wheel.”
Data that suggests insufficient care, such as low screening rates for chronic diseases, can cause clinical staff to question the data’s validity. Providers may feel the data indicates they aren’t providing adequate care, and they may be defensive or dismissive. Understandably so, but often the reality is not poor care, but rather inconsistency in the EHR data capture practice or because providers may deliver the care but fail to document it properly. Unfortunately, when it comes to data, if it wasn’t recorded, it’s as if it never happened.
“Either providers and staff are not recording the data in the places the practice told us are their standard, or there is data in places they don’t know about, so it’s simply not been mapped. Either way, the data needs to be discovered,” said Budd. “Our team works very hard to make sure that providers are given proper credit for the work they are doing.”
Amos and his DRVS implementation team at HCH pride themselves on being thorough and detail-oriented, so scrutinizing the data’s validity came naturally, as did weeding out problem areas during the implementation.
“Everyone on the team is very committed to testing,” he said, adding that they strive to present the clinical staff with a system that is as efficient and accurate as possible. Otherwise, providers are unlikely to embrace it. Azara agrees.
The data validation process revealed discrepancies in how HCH staff defined items that, on the surface, might seem straightforward. Amos said the process improvement team and many providers used different criteria for designating whether a patient should be considered a smoker, and entered into the EHR as such.
“Part of validating is defining,” said Orr. “When Azara would come back to us to say ‘we’ve been counting smoking cessation advice as patient education,’ we had to decide if smoking cessation advice really is a piece of patient education. So, part of that validation process is deciding what you consider to be your data, how do you define it, and then, what you include in that data set.”
With Staff, Mixing Things Up is a Good Thing
Amos credits his team’s varied professional backgrounds and skillsets as a vital part of HCH’s early success with the data validation and DRVS implementation. In addition to her deep experience as an EHR administrator, Orr worked at the HCCN that serves the Baltimore region, granting her experience in working with multiple health centers and problem solving for disparate groups; Gayevsky is credited as an ultra detail-oriented database manager who frets over even miniscule data anomalies. Ward’s background includes a MPH, which Amos said was one of his requirements when filling the clinical analyst post because he wanted someone schooled in rigorous data analytics methods.
“Having a multidisciplinary approach before you even bring clinical people to the table is something that other centers can replicate” said Amos, adding that tapping the right resources for the right phase of the project is the key. “We were able to cut our clinical people out of the first five weeks of validation because we didn’t need an MD to tell us, ‘Oh, this person’s race is wrong.’”
Removing the clinicians from the early phases demonstrated respect for their time and allowed Amos’ team to maximize their buy-in when they were really needed in the validation process.
The DRVS implementation has presented HCH with numerous opportunities for growth and exposed weak points within the center’s data infrastructure. The investment in time and resources dedicated to the validation process has helped to set the center on a path toward making its data truly actionable. The process hasn’t been without mistakes, but discovering issues hasn’t deterred the center’s validation effort.
When the implementation was over, Gayevsky, the detail-oriented database manager, hugged Lori Lynes, HCH’s DRVS implementation specialist. She thanked Lori for helping give her a tool that will save her so much time. She saw that the reporting platform would allow her to focus on the other demands of her role, while using the reports to help keep the data in sync with evolving workflows. Azara views HCH’s approach as a good, replicable model for maximizing the use of a reporting and analytics platform.
“We are very pleased to have collaborated with Health Care for the Homeless on their data validation process,” said Budd. “Chuck, his team, and the rest of the center put in a lot of extra effort to ensure their data is as accurate as possible. We are confident their commitment to validation will allow them to use DRVS to its full potential and improve their care delivery and efficiency.”
Even more importantly, HCH is happy with the quality of its data now, having made the upfront investment in validation, and can confidently present the data to the rest of the clinical staff. HCH also benefited from Azara’s flexibility during the implementation and validation process.
“They were accommodating of us changing our minds about how to handle visits, about how to handle providers, “said Ward. “Azara understands that this is a process that is going to take a while.”