This feature first appeared in the Winter 2021 issue of Certification Magazine. Click here to get your own print or digital copy.

I often hear the question, 'Why do our learners need an evaluation?' My response is always the same: 'Why wouldn't they need an evaluation?' Then a thought occurs, 'Do you check to make sure that you cook your chicken all the way through?'

This may appear to be a bizarre link in logic — but getting things right is important. In the case of cooking chicken, it is a safety concern. Imagine how you'd feel about biting into an undercooked drumstick? With learner evaluation, especially at the certification level, it is a quality concern.

I have heard people say, 'We do not need an assessment if there is no need to measure or verify learner knowledge.' When you design a complete learning experience with intentional learning outcomes and goals, it is imperative that you assess each learner's knowledge and skills.

Thinking through your evaluation plan will provide valuable perspective that helps you to prioritize projects. In other words, you need to cook the chicken all the way through.

Here's another way to think about evaluation differently, using my first grade daughter as an example. She is attending school virtually. She recently had a week where she was testing her independence by not turning in assignments. To make things worse, she skipped a spelling test.

School assignments are there to reinforce key ideas and are a valuable part of the learning process. A spelling test is an assessment. It tells the teacher what she knows. My daughter assures me that she knows how to spell all the words in the test, but without the results of the spelling test, how can the teacher validate her skills?

Evaluation is the tool to measure success. You can be competent and credible, but if you leave your expertise unvalidated, then you miss an opportunity to objectively demonstrate your skills. There are many types of evaluation. I will now discuss different forms of how to assess learner progress and then make the case for certification as an industry standard.

Forms of evaluation

Many companies, such as NetApp where I work, provide training for employees, customers, and partners. Evaluation techniques vary based on the goal of the program. Evaluations can range from simple knowledge checks to drive learner engagement on through to formalized certification programs that enable learners to establish themselves as specialists.

Knowledge checks are the most basic form of evaluation. Back when we could still meet each other in person, knowledge checks could be as simple as a speaker using rhetorical questions to gauge audience reaction. For learning purposes, they provided feedback to the instructor to either move forward with the lesson or to go back and revisit an important point.

In an online course, knowledge checks are often a set of questions that reinforce key points. Rarely is this data collected. Knowledge checks are a teaching tool that provide an opportunity for remediation.

Assessments are the next level of measurement. This form of evaluation is more structured and closely aligned with learning objectives. An assessment generally does not include more than 20 questions. Assessments can be used to allow learners to test out of a course, or they can be a standalone module that allows a developer to capture knowledge gained.

This rich data provides a developer with performance measures to make course improvements. An assessment typically has a pass gate of 80 percent and is sometimes limited to a reduced number of attempts. Assessments check for learning outcomes.

Accreditations are a third form of learner evaluation. At NetApp, we have an accreditation program that is geared toward a specific partner audience. Compliance requirements often drive partners' participation. There is often intrinsic reasoning, however, behind achieving this type of credential.

Accreditation is aligned to a learning path versus a specific course, typically. Accreditations have lower stakes than certifications but enable trainers to see if a learner can perform a series of skills in their role. Accreditations check for skills learned.

In some instances, an accreditation exam is created prior to investing in the creation of a certification exam. This allows the teams to see how much demand there is for an exam at a lower stakes level and to understand how the exam performs. If you see someone with an accreditation badge, which accredited partners frequently share on social media, it provides evidence of hard work and competency.

Because accreditation exams are typically not proctored, however, and often include fewer questions than certification exams, an accreditation is not always as rigorous or as psychometrically sound as a certification.

Certification as an industry standard

In the tech industry, creating a certification exam is no easy feat. In fact, there must be a strong business case to move forward. Creation involves input from all stakeholders. There is a significant financial and time investment to do the work correctly and there must be ROI.

An evaluation manager needs to look critically at the business case to see if the certification meets both the goals of the business and the needs of the candidates. Two questions must be asked: What does someone certified in this area do for the company (or other certifying organization)? Alternatively, what does this new certification exam do for the candidate?

First, what can a certification do for a company or professional association? As someone passionate about starting with the end in mind, I favor the ROI Methodology that was developed by the ROI Institute. As part of this methodology, data is collected at five different levels. Most relevant to the topic of this article, the ROI Methodology emphasizes application of knowledge and impact of application.

In an ROI training that I attended, the instructor asked, 'If someone left your classroom with knowledge and then did nothing with it, would that be success?' This question (and its answer) drives my desire to measure beyond an assessment-level.

Next, what can a certification do for a candidate? If you ask 10 candidates, 'Why did you choose to take a certification exam,' then you might uncover 10 different motivations.

According to a yearly survey conducted by Pearson VUE, however, the top personal reasons for certification include improving professional standing, increasing knowledge of a certain technology area, and improving chances of receiving a salary increase. In essence, candidates gain greater self-confidence in their abilities and transferrable knowledge as a result.

It can take more than a year to research, create, publish, and market a new certification exam. At NetApp, the team works with a psychometrician to take a group of subject matter experts (SMEs) through a verified process to create sound items (questions) for an exam.

Exam items are not designed to 'trick' a candidate but are closely aligned to a carefully crafted blueprint to validate both knowledge and skills. Not all companies have performance-based certifications today, but a well-written scenario-based question can test if a candidate is able to take their knowledge of a concept and apply it correctly.

Experience is a requirement to be successful on a certification exam, especially when a candidate moves past the entry-level exams and aims to become a specialist. Certifications validate job role skills and knowledge.

It is common knowledge that certifications are not easy to achieve. Difficulty is built in by design. I would like to say that certifications are the 'end' of a learner's journey, but they often expire and require continuous learning and application to maintain. Based on how fast technologies evolve, some certifications require more frequent updates than others.

Cloud technologies are a good example. It is not easy to update certification exams quickly and uphold the integrity of the process. Since certifications are the ultimate test of a learner's expertise, they continue to have a place in how companies evaluate learner success and maintain their station as an industry standard.

Walk away knowing this

Knowledge checks, assessments, accreditations, and certifications are all forms of evaluation to measure learner progress and success. Companies should strive to design all these forms of evaluation — with intent. Rather than questioning evaluation, they should determine the right level of evaluation for the desired outcome.

Certifications provide multiple benefits, from preparing for the exam to achieving an industry validation from a reputable company or professional association. With the shift to virtual learning, it is even more critical to provide opportunities for learners to know where they stand in their learning journey.

Certifications are a way for a candidate to tell the industry, 'I am competent and credible.' Think of what else a certification tells the industry? You cook your chicken all the way through. You value quality. You care about your career. You are ready for advancement. You have specialized knowledge your employer can tap to improve business outcomes.

When people understand that there is value in an e-value-ation, they will ask to be evaluated!

NetApp Learning Services has an industry recognized certification program that has been available to NetApp employees, partners, and customers for more than 10 years. Additional information on the program is available online: https://www.netapp.com/support-and-training/netapp-university-training-and-certification/certification/