This feature first appeared in the Fall 2017 issue of Certification Magazine. Click here to get your own print or digital copy.

The recent escalation of information security incidents around the world, from the Equifax hack and the publishing of NSA-derived exploits and global ransomware attacks, to the barrage of hacking continuing to plunge the Ukraine into a constant state of unease, has highlighted the increased need for better risk assessment and management in all areas of business.

As a former information security professional turned exam sponsor, I view risk assessment and management as being innately imbedded into the management of our credentialing program. It is increasingly evident, however, that my path to the certification realm differs significantly from the path taken by most professionals who also end up there.

As technology continues to transform our industry rapidly, everyone needs to understand risk assessment and management basics to make informed decisions affecting the validity, integrity, and credibility of our assessment and credentialing programs.

In the credentialing world, the concept of "legal defensibility" is a consistent theme. We spend significant time, money, and effort ensuring our programs are legally defensible, and applying rigorous psychometric standards and processes. Yet this term rarely extends beyond supporting the basic validity of the assessment score interpretations. Exam security rarely extends beyond maintaining the confidentiality of the test items and delivery.

In the information security world, legal defensibility is enshrined in two specific concepts: due diligence and due care; bridging these two is the risk assessment process. A basic understanding of these concepts and the process will allow certification sponsors to make better decisions across their credentialing programs.

Due Diligence

Due diligence is a legal standard assessing whether an entity applied reasonable effort to ascertain, identify, and document possible issues. Since "reasonable" is a moving target, best practices of due diligence generally comprise two components.

The first is to encourage the development of an exhaustive, detailed list of possible scenarios and issues, no matter how unlikely. The second is to make this exercise an ongoing, systematic part of the organization's activities.

The first step in program risk assessment is simply to define the risks associated with your activities. This should be an open, freeform, brainstorming process where no risk, no matter how small or unlikely, is disregarded.

The more detailed and exhaustive this list of potential risks is, the easier it is to prove due diligence — one missing risk that should have been listed is much more damaging than one thousand irrelevant risks.

I have been known to list things like meteor strikes and armed insurgency. While they may be unlikely, the key is to make sure you've thought of everything that could happen to pose a risk to the organization. Whether that risk is probable or significant is evaluated later in the process.

The second component of due diligence is making sure the organization updates and maintains the list of potential threats, adding new threats and updating existing ones based on current understandings. The business landscape is dynamic and quickly changing, and an organization needs to make the effort to ensure these changes are adequately assessed.

How often this needs to be done depends entirely on the organization and its activities. Many organizations may find that a yearly evaluation is sufficient, while others in highly volatile business environments may need to do it more frequently. From a due diligence standpoint, the important part is to demonstrate an ongoing concern for understanding the risks associated with the organization's operations.

Risk Evaluation and Assessment

The risk evaluation and assessment process takes the results of due diligence as its input, and generates the documented evidence of due care as its output. The process takes each potential risk, evaluates the effect and likelihood of that risk occurring, compares the cost of experiencing the risk with the cost of addressing the risk, and makes a final determination of how the organization will deal with that risk.

The end result is a prioritized list of the risks most important to address, and a plan of action to address them if, or when, they occur.

There is no real standard for determining the effects and likelihood of risk, but most organizations use some combination of similar ideas. For example, it may be sufficient to organize your risks based on estimates of the potential cost of the risk and the likelihood of happening (Figure 1) as a means to prioritize your efforts. This allows for quick identification of high risk issues that likely demand immediate attention.

A more detailed risk assessment and disposition document (Figure 2), listing each risk, cost, probability, exposure, and disposition is also very common. The methods of calculating exposure are myriad and diverse depending on the needs and capabilities of the organization. The intent is to create a weighted measure of the potential cost of a risk and the probability of it becoming a reality to create a general sense of priority.

Although much more detailed, calculating exposure allows the prioritization of potential risks similar to the risk assessment grid in Figure 1. In addition, the more formal entry of disposition provides a place to document an organization's due care process as well as action plans.

Due Care

Due care is the second standard applied to legally defensible risk management practices. Due care assesses whether the entity took reasonable effort to mitigate and avoid identified risks, or the reasons for accepting a risk.

Similar to due diligence, "reasonable" can be a moving target and open to interpretation: It is fairly easy to say that it is unreasonable to spend $1 million to protect a one dollar note, while spending one dollar to protect $1 million is a no-brainer. Anything in between requires the organization to demonstrate the fact that each identified risk was evaluated and considered, and document how each risk was addressed (disposed), including the rationale for that decision.

There are many possible ways to address risks. You can mitigate a risk by taking some direct or indirect action, often (but not always) incurring some cost to do so. You can also assign a risk to a third-party, which is the case when you take out insurance to protect yourself from the risk of loss. Finally, you may simply decide to accept the risk either because it is too costly to mitigate, cannot be assigned to a third party, is either too small or improbable, or because there is no way to address the risk currently.

The important part of due care is demonstrating not only how you made your decision, and what that decision was, but also that a decision was made. From a due care perspective, there is a big difference between a risk you didn't address at all and a risk you chose to accept or couldn't address. The former shows a lack of due care, while the latter suggests you applied effort.

It is important to make the evaluation of each risk an ongoing, systematic part of the organization. Just like the risks themselves, what may be considered reasonable due care can change over time. What once may have been financially prohibitive to protect against might now be reasonable. Or the risk may have become more pronounced, making the cost to mitigate it more appropriate.

In some instances, a new technology may now exist allowing you to address a risk previously not addressable. Just like due diligence, due care requires an ongoing evaluation and isn't something done just once and then forgotten.

Putting Risk Assessment into everyday action

The preceding process may seem daunting, cumbersome, and unlikely to aid in the everyday management of a credentialing organization, but becoming familiar with these basics can be extremely valuable. Making risk assessment a part of your standard operating procedures supports strategic decision making, increases your ability to quickly adapt to rapidly changing environments, and fosters innovative thinking. These are the quintessential characteristics of a successful organization.

Getting into the habit of listing the potential risks for different courses of action can help drive strategic decision making. For instance, many of us deliver our high-stakes proctored exams via a single organization worldwide. This presents many risks should that vendor's operations become compromised or otherwise unavailable.

As a result of these risks, some organizations are looking for alternate means of delivering proctored exams, like online proctored exams or other solutions. These alternatives, of course, have their own set of risks. Utilizing risk assessment approaches can help identify the most strategically effective decisions, balancing the needs of the organization with the risks of various options.

At the very least, engaging in the risk assessment process provides an objective framework for the decisions that are made and their rationale.

Risk assessment as an ongoing process also helps organizations nimbly respond to dynamic business environments. The proliferation of hidden video cameras affecting the probability of exam exposure in traditional proctored exams is an increasing risk that was not substantial even five years ago.

Understanding how this changes the calculus between the comparative risks of proctored and online-proctored exams helps an organization respond to new technology. Recent advances in augmented-reality (AR) may not only change the way we assess individuals, but may also change what constitutes the minimally qualified candidate. We cannot control the world around us, but proper risk assessment provides objective means to analyze and adapt.

Lastly, the very act of listing potential risks and comparing the risks with various alternatives fosters an environment ripe for innovative solutions. Critical factors driving many organizations are the perceived risks and resultant effects on validity from assessment exposure. This single risk drives many other decisions and activities.

If assessment exposure could be mitigated directly, innumerable credentialing risks would disappear, opening a myriad of new opportunities. By putting all of the risks inside a box, it becomes easier to discern connections between various risks and see what falls outside the box. It provides a different perspective that sparks innovative approaches.

In parting

The world is changing at an ever-increasing pace driving credentialing and learning organizations' need to understand how these changes may affect them. Basic risk assessment is a simple process that can help. By fostering the tools and the mindset for evaluating potential risks, we can make better decisions, quickly adapt to potentially disruptive technological innovation, and foster innovative thinking.

Further, by engaging these tools, we set a new standard for a legally defensible program, one that also includes due diligence and due care to our operations beyond the validity of interpretations from our assessments.