By Jennifer Bianca Browning, Senior Analyst, ODCA
A year ago, after spending months glued to the District’s COVID-19 case numbers and other data, wondering about where the pandemic was headed and if we were getting the information we needed, ODCA got an invitation from the Delaware State Auditor to join a national COVID-19 data quality audit. The initiative came about because federal guidance was lacking on what data states should collect and report. The idea: To create a framework states could use to assess the data being released. We enthusiastically joined the participating states to use a COVID-19 data quality audit template as a guide that covered testing, contact tracing, hospitalization, and death data. The result is our report, National COVID-19 Data Quality Audit: District of Columbia, which covers the period from March 2020 to April 5, 2021
Limiting the audit scope to COVID-19 data rather than a full audit of the District’s COVID-19 response met the objective of the multistate audit and also our goal to not overburden agencies during a pandemic. Despite the stressful conditions, we were impressed by our interactions with DC Health and the Office of the Chief Medical Examiner (OCME), both of which approached this audit with a spirit of openness and a focus on improvement.
This Auditude post digs dig a little deeper to highlight findings that may have been overlooked. As we’ve noted in prior posts to this blog, headlines over bad news get more attention than headlines over good news. Our report shares the good news that DC Health has done a lot right in collecting and reporting COVID-19 data and has demonstrated a continued interest in improving their current procedures in comparison with other jurisdictions. As we consider improvements, we would be remiss if we didn’t repeat the recommendation that when the pandemic abates, the District should undertake a full review of the COVID-19 response to confirm what’s been learned so we can improve for the future. In the meantime, here are some areas we hope are not overlooked and want to explore.
An Improved Data Dashboard
DC Health has made significant progress in reporting data to the public. As we detailed in our November 2020 report on the District’s data dashboard, DC Health revamped and expanded the way they report COVID-19 data, making it more understandable to residents by launching a new Reopening Metrics page in September 2020.
DC Health has since added other pages, including a Vaccinations page to track how many D.C. residents are vaccinated. The Vaccinations page now includes information on breakthrough infections. In terms of the national COVID-19 data quality effort, the District was answering most questions and reported 74% of the information the audit template asked about.
That said, key data still isn’t being presented. Nine recommendations from our November 2020 report still need to be implemented. In our most recent report, we make the following additional recommendations on reporting data:
Data to Determine Health Disparities
Complete data are essential for both assessing equity and working toward improvement. Importantly, the audit found that most demographic data was complete in DC Health’s positive case data; under 1.2% of positive cases had missing race, sex, and/or age data. In DC, although Black residents make up less than half of the population, they comprise 77% of COVID-19 deaths. This sobering figure highlights how Black communities continue to be hit disproportionately hard by the pandemic. Again, a comprehensive review of the District’s response should yield lessons learned and pathways for needed improvement.
DC Health did use and monitor data to improve the completeness of ethnicity data. As we detail in our report, nearly 20% of ethnicity data (Hispanic/Latino, non-Hispanic/Latino in this case) was initially missing. DC Health updated data that they published with new information, and as a result, they have been able to reduce the amount of missing ethnicity data. For March 2021, a year into the pandemic, only 6.8% of ethnicity data was missing in downloadable data that DC Health publishes at https://coronavirus.dc.gov/data.
New Technology Systems in a Pandemic
DC Health provided a grant to the D.C. Hospital Association to replace an outdated system with EMResource, a newer IT system, for reporting COVID-19 data required by DC Health and the U.S. Department of Health and Human Services (HHS). All District hospitals with the exception of Sibley Memorial Hospital now report key COVID-19 data into the centralized EMResource system.
DC Health also built out its technology system for managing contact tracing information, adding new features and making changes to improve data quality. For example, the contact tracing system logs individuals’ location and time of activities, allowing DC Health to run an algorithm to identify possible clusters of cases.
The DC COVID Alert Notice (CAN) app was developed by Google and Apple as a way to help contact tracing by anonymously notifying people who were in close contact with someone who tested positive. Since Mayor Bowser launched the app on October 20, 2020, there have been more than 1 million opt-ins, however the same phone can opt-in more than once. As with any innovation, we don’t know yet whether the DC CAN app technology is a new frontier in contact tracing and exposure notification or is just another shiny, new technology that does not live up to expectations.
Regular Reporting and an After-Action Review
As cases surge due to the Delta variant, we hope DC Health continues to improve its COVID-19 dashboard, including addressing our recommendations and providing regular, up-to-date reporting. Importantly, the reporting pace slowed during the summer. With the Delta variant and children back in school, accurate, timely COVID-19 data remains as crucial as ever.
Our most recent report covered data reporting from March 2020 through April 5, 2021. Since the end of May, DC Health has stopped reporting case numbers daily, combining Friday and weekend days together. Now in mid-September case numbers are again very high. On September 21, 2021, the daily case rate was at the same level as January 15, 2021. The weekend case counts are lumped together, which can cause confusion if the user doesn’t realize it. We also have been seeing backlogged cases reported, often with the weekend cases, and without any explanation. Consistent with our published reports, we encourage DC Health to report daily and to maintain high data quality including clear explanations, especially given the current high case rate.
It bears repeating that while our COVID-19 data audit highlighted both positives and areas for improvement in DC Health and OCME’s data processes and reporting. However, the audit wasn’t an evaluation of the District’s overall response; we didn’t evaluate whether testing, contact tracing, hospitalizations, and death certifications worked as they should.
We do think establishing what worked and what didn’t is incredibly important once the pandemic abates. Here is our recommendation with an eye to the future to leave the District better prepared to respond to the next emergency, improve inequities, and save lives:
