LO 3.1: Define backtesting and exceptions and explain the importance of

LO 3.1: Define backtesting and exceptions and explain the importance of backtesting VaR models.
Backtesting is the process of comparing losses predicted by a value at risk (VaR) model to those actually experienced over the testing period. It is an important tool for providing model validation, which is a process for determining whether a VaR model is adequate. The main goal of backtesting is to ensure that actual losses do not exceed expected losses at a given confidence level. The number of actual observations that fall outside a given confidence level are called exceptions. The number of exceptions falling outside of the VaR confidence level should not exceed one minus the confidence level. For example, exceptions should occur less than 3% of the time if the confidence level is 93%.
Backtesting is extremely important for risk managers and regulators to validate whether VaR models are properly calibrated or accurate. If the level of exceptions is too high, models should be recalibrated and risk managers should re-evaluate assumptions, parameters, and/ or modeling processes. The Basel Committee allows banks to use internal VaR models to measure their risk levels, and backtesting provides a critical evaluation technique to test the adequacy of those internal VaR models. Bank regulators rely on backtesting to verify risk models and identify banks that are designing models that underestimate their risk. Banks with excessive exceptions (more than four exceptions in a sample size of 250) are penalized with higher capital requirements.
2018 Kaplan, Inc.
Page 25
Topic 3 Cross Reference to GARP Assigned Reading – Jorion, Chapter 6