LO 41.4: Describe and assess the use of scenario analysis in managing operational

LO 41.4: Describe and assess the use of scenario analysis in managing operational risk, and identify biases and challenges that can arise when using scenario analysis.
Scenario analysis is defined as the process of evaluating a portfolio, project, or asset by changing a number of economic, market, industry, or company specific factors. Scenario analysis models are especially useful tools for estimating losses when loss experiences related to emerging risks are not available to the financial institution. Inputs to scenario analysis models are collected from external data, expert opinions, internal loss trends, or key risk indicators (KRIs). Expert opinions are typically drawn from structured workshops for large financial institutions. However, surveys and individual meetings can also be used to gather expert advice. Studies suggest that most financial firms analyze between 50 and 100 scenarios on an annual basis.
One of the challenges in scenario analysis is taking expert advice and quantifying this advice to reflect possible internal losses for the firm. The following example illustrates how a firm may create a frequency distribution of loss events that can be used in scenario analysis.
Figure 8 illustrates data constructed for a financial institution based on expert inputs. Information is gathered on loss frequencies for pre-determined loss brackets. Thus, a frequency distribution is created to model the probability of losses based on the amount of loss on an annual basis. This frequency distribution is then used in the OpRisk framework for the firm.
Figure 8: Scenario Analysis Model for Loss Frequencies
Loss Bracket
N umber o f Losses
Over $5,000,000 $1,000,000 to $5,000,000 $500,000 to $1,000,000 $250,000 to $500,000 $100,000 to $250,000 $50,000 to $100,000 Total
3 9 18 25 41 12, 168
Frequency 1.8% 5.4% 10.7% 14.9% 24.4% 42.9% 100.0%
2018 Kaplan, Inc.
Page 51
Topic 41 Cross Reference to GARP Assigned Reading – Cruz, Chapter 2
Biases and Challenges of Scenario Analysis
One of the biggest challenges of scenario analysis is the fact that expert opinions are always subject to numerous possible biases. There is often disparity of opinions and knowledge regarding the amount and frequency of losses. Expert biases are difficult to avoid when conducting scenario analysis. Examples of possible biases are related to presentation, context, availability, anchoring, confidence, huddle, gaming, and inexpert opinion.
Presentation bias occurs when the order that information is presented impacts the experts opinion or advice. Another similar type of bias is context bias. Context bias occurs when questions are framed in a way that influences the responses of those being questioned. In the case of scenario analysis, the context or framing of questions may influence the response of the experts.
Another set of biases are related to the lack of available information regarding loss data for a particular expert or for all experts. Availability bias is related to the experts experience in dealing with a specific event or loss risk. For example, some experts may have a long career in a particular field and never actually experience a loss over $ 1 billion. The availability bias can result in over or under estimating the frequency and amount of loss events. A similar bias is referred to as anchoring bias. Anchoring bias can occur if an expert limits the range of a loss estimate based on personal experiences or knowledge of prior loss events. The availability an expert has to information can also result in a confidence bias. The expert may over or under estimate the amount of risk for a particular loss event if there is limited information or knowledge available for the risk or the probability of occurrence.
Expert opinions are often obtained in structured workshops that have a group setting. This group setting environment can lead to a number of biases. Huddle bias (also known as anxiety bias) refers to a situation described by behavioral scientists where individuals in a group setting tend to avoid conflicts and not express information that is unique because it results from different viewpoints or opinions. An example of a huddle bias would be a situation where junior experts do not voice their opinions in a structured workshop because they do not want to disagree in public with senior experts. Another concern for group environments is the possibility of gaming. Some experts may have ulterior motives for not participating or providing useful information in workshops. Another problem with workshop settings is the fact that top experts in the field may not be willing to join the workshop and prefer to work independently. The lack of top experts then attracts less experienced or junior experts who may have an inexpert opinion. These inexpert opinions can then lead to inaccurate estimates and poor scenario analysis models.
One technique that can help in scenario analysis is the Delphi technique. This technique originated from the U.S. Air Force in the 1950s and was designed to obtain the most reliable consensus of opinions from a group of experts. This technique is useful for many applications for analyzing cases where there is limited historical data available. More specifically, the Delphi technique is often applied in situations that exhibit some of the following issues: Precise mathematical models are not available but subjective opinions can be gathered
from experts.
Experts have a diverse background of experience and expertise, but little experience in
communicating within expert groups.
Page 52
2018 Kaplan, Inc.
Topic 41 Cross Reference to GARP Assigned Reading – Cruz, Chapter 2
Group meetings are too costly due to time and travel expenses. A large number of opinions is required and a single face-to-face meeting is not feasible. Under the Delphi technique, information is gathered from a large number of participants across various business units, areas of expertise, or geographical regions. The information is then presented in a workshop with representatives from each area. Recommendations are determined by this workshop group and quantified based on a pre-determined confidence level. A basic Delphi technique commonly goes through the following four steps: 1. Discussion and feedback is gathered from a large number of participants who may have
diverse exposure and experience with particular risks.
2.
Information gathered in step 1 is summarized and presented to a workshop group with representatives from various locations or business units surveyed.
3. Differences in feedback are evaluated from step 2.
4. Final evaluation and recommendations are made based on analysis of data and feedback
from participants and/or respondents.
O p e r a t i o n a l R i s k P r o f i l e s