Secret Testers Used as an Evaluation Tool

Aug 3, 2016 | A Blog Posts by Checked & Balanced

To view the full report visit http://www.dcauditor.org/reports/customer-service-tests-seven-large-agencies-show-mixed-results

Last month, the Office of the District of Columbia Auditor released a report depicting mixed results in customer service tests conducted at seven major D.C. government agencies (311, the Office of the Chief Financial Officer, Department of Consumer and Regulatory Affairs, Metropolitan Police Department, D.C. Public Schools, Department of Human Services, and Department of Public Works).   

As shown in the chart below, average scores for knowledge conveyed by telephone, telephone courtesy, and knowledge conveyed by e-mail ranged from 3.7 to 4.2 on a 1-to-5 scale, reflecting good performance in a majority of cases but inconsistency in others.

The customer service study reflects ODCA’s effort to use a broader range of evaluation tools that capture government programs and services in action.  In this study, ODCA staff served as “secret testers” who contacted the agencies by phone or e-mail to seek information or assistance, using scripts written to reflect common questions or problems, and then rated agency performance on the 1 to 5 scale.  Using the secret tester method, ODCA was able to gauge the strengths and weaknesses of customer service in the D.C. government and offer ideas for improvement.  Given the potential of the secret tester approach, ODCA hopes to use this method to illuminate the quality of other government services.

The secret or covert tester is being used in a variety of audits and evaluations of government programs, by internal and external evaluators alike.  The following are three major examples:

  • The U.S. Government Accountability Office has used testers to evaluate the government’s ability to detect and stop individuals trying to bring hazardous materials or weapons across borders or onto government facilities.
  • Groups such as the Urban Institute and the Equal Rights Center have tried to measure the extent of unlawful discrimination by sending individuals who are virtually identical except for their race, gender, disability, or other protected characteristic, to apply for jobs or housing and record the results.
  • The D.C. Alcoholic Beverage Regulation Administration has used underage testers to verify whether its licensees check the IDs of young people seeking to buy alcohol.

Post Your Comments Here

Comment Policy

We welcome your comments and expect that our conversation will follow the general rules of respectful civil discourse. This is a moderated blog, and we will only post comments from bloggers over 13 years of age that relate to topics the posted topics. We do not discriminate against any views, but we reserve the right not to post comments. To read more about the comment policy click here.