From Cheating 2.0 to Security 3.0
BackBy Dave Meissner — September 2009
Certification exams are a measure of qualification increasingly relied upon to gauge an individual’s knowledge, skills and abilities. The results of these exams are used by managers for employment decisions and by government agencies and the general public when seeking qualified professionals.
And, particularly in tough economic times — with unemployment rates soaring — picking up added skills can mean the difference between landing a job and being first runner-up. Unfortunately, however, lying about those skills by cheating or fudging resumes becomes increasingly tempting.
A desperate candidate might try anything to get ahead, including securing advance copies of an exam, computing answers using hand-held computers or high-feature cell phones, or hiring someone else to impersonate him or her at the test center. These advances in cheating technology — otherwise known as cheating 2.0 — may seem insurmountable, but test sponsors and service providers are fighting back. As the technology that can be used to cheat evolves, so too does the technology used to prevent it.
Security 101: The Basics of Prevention
Any test worth taking occurs in a proctored environment: either a brick-and-mortar test center or another type of a facility where proctors are brought in specifically for the testing event.
First and foremost is maintaining a safe and cheat-free testing environment. Test center operators have a waiting space along with a separate “testing room” to delineate the secure testing environment from the registration and intake area. Generally, nothing outside of the test candidate’s physical body is allowed in the secure testing room. If any materials are allowed by the test sponsor — such as scratch paper — the test center will provide its own to the candidate at the time of check-in to ensure that test takers are not able to “smuggle in” notes.
Best practices often include the provision of color-coded whiteboards or scratch paper, distributed at the beginning of a testing session and collected and counted at the end. This ensures that candidates not only don’t smuggle notes into the exam, but that they don’t copy exam questions onto the notepaper and smuggle them out, which would give those who have not yet taken the exam an unfair advantage.
The identity of the candidate also must be verified. When a candidate arrives at a test center, he or she is asked to present either one or two valid, government-issued IDs with a photo and signature. The identification is then swiped through a machine that reads the information stored in the magnetic strip or bar code on the back of the identification. This information is then compared with the “visible” information on the front of the ID to ensure a match.
Once inside the testing area, candidates are placed in workspaces that are partitioned off from others so that they cannot view their neighbors’ exams. Test center administrators and proctors patrol the testing area at specific intervals to look for abnormalities. For added security, many test centers use a closed-circuit television (CCTV) system to focus on candidates’ faces and hands. If something out of the ordinary comes to light during the testing process, the test center administrator can take a closer look by zooming in. The footage also can be reviewed at a later time, should an abnormality present itself during the scoring process, so scorers can determine exactly what happened during the course of the exam.
And who, exactly, monitors what goes on in the exam room? Test center administrators (TCAs) are the watchful professionals responsible for ensuring candidates are who they say they are; walking through the testing room at regular intervals to physically proctor the exams; and guaranteeing that candidates are not able to sneak any materials into the secure testing room. Many large test services providers require that test center administrators be certified in the practice — ensuring they are qualified, skilled and knowledgeable about what to look for in the center, as well as what to do in certain situations.
The Next Level
Physical security aspects aside, perhaps the most secure facet of modern testing and the best defensible method against cheating 2.0 is computerization. Test items for computer-based testing (CBT) are typically stored electronically and transported to the test center in an encrypted state via a secure pipeline directly from the test services provider. When using secure output lines, the possibility of an information breach during transit is extremely low. This process is completely different from the transit of standardized paper-and-pencil exams, which require physical shipment through the mail, creating somewhat easier access for cheaters.
With computer-based testing, exams can have multiple forms, thousands more questions to choose from and a random nature that cannot be achieved through paper-based testing. Computer-based exams can be leveraged to eliminate the predictability and static nature of paper-based testing, allowing for randomized item presentation, dynamic testing and secure transit of information to and from the testing center — all facets that make it substantially more difficult for cheaters.
Using a computer to test also allows sponsors to consider the incorporation of performance-based items into their exams. Most exams assess an individual’s knowledge through multiple-choice questions. Multiple-choice items are extremely valuable and will be a critical element of tests for many years, but they may sometimes also be susceptible to cheating — through the sharing of potential test questions with another candidate — and “item harvesting” — the coordinated attempt to collect a large number of test questions and then distribute them for a profit.
Supplementing multiple-choice items with performance-based ones — tasks that are representative of the activities a candidate might be expected to perform on the job — can improve the overall value of the exam while making it virtually impossible to pass the test without a thorough understanding of the material.
Another type of testing that is increasingly being used is called “linear on the fly,” or LOFT. LOFT is a dynamic forms generation testing model that utilizes item response theory statistics to produce an individually assembled exam for each candidate. The success of LOFT exams is highly dependent on having enough items in the item bank to support the model — ideally eight to 10 times the number required for a psychometrically sound computer-based test. Concurrently, the method adjusts the item selection routine to account for item exposure, making the memorization of significant portions of the overall exam extremely difficult. The LOFT process ensures that each candidate receives a completely unique and individualized exam, making cheating of any type close to impossible.
Certain test services providers also use technology to provide analysis of items and exams to detect abnormalities in the test process. Abnormalities include anything from unusual response patterns or unexpected candidate behavior — such as ending a test early, not completing a test or requesting frequent breaks — to sudden performance improvements. All of these can be indicators of a potential security concern that can be investigated by a thorough review of the computer files captured during a test event.
Finally, some test centers use biometrics as an added security measure. Biometrics includes practices such as fingerprinting. The fingerprint reader, which is the most widely used and accepted practice, captures an image of a fingerprint that is used to monitor the movement of the candidate in and out of the test room. The fingerprint also can be compared electronically to a central database to ensure that the candidate did not test previously under a different name. Should a candidate come back to take another test years later, the information can be pulled up and compared. Additionally, should someone who is not the valid candidate appear at a testing center years later and claim to be the candidate, the center would be able to tell — just from referencing both the fingerprints in the database and the saved identification information.
Migrating from Cheating 2.0 to Security 3.0
A Boston Globe article last year reported that among 200,000 test attempts, there were 1,000 confirmed incidences of cheating. The article made quite a big deal about the number, blowing the reality way out of proportion. In fact, this is a rate of only 0.5 percent. This same story could have reported the findings differently by saying that 99.5 percent of tests are valid and reliable measures of individual skills and abilities.
Despite IT advances that could help candidates cheat on standardized certification exams, testing security has only grown stronger as technology progresses, due in part to the rise of large-scale test administrators and the CBT model. Digital video recording systems, biometrics and dynamic exams all work together to ensure that while cheaters may be using cheating 2.0, test centers are already using security 3.0. In doing so, the very certifications that employees pursue to maintain marketability challenge them to demonstrate ethical behavior as well as knowledge and skill sets.
You may think: Who cares if people cheat on exams? Aren’t they only hurting themselves? Well, consider this: Would you want someone who cheated on his or her nursing exam standing over your child in an operating room? How about someone who didn’t really understand accounting fudging your taxes? Or someone who smuggled cheat sheets into a test about construction safety codes building the house your life savings is going into? Would you be willing to risk it? Didn’t think so.
David Meissner is vice president of solution services at Prometric, a leading provider of comprehensive testing and assessment services. He can be reached at email@example.com.