LO 3.3: Explain the need to consider conditional coverage in the backtesting

LO 3.3: Explain the need to consider conditional coverage in the backtesting framework.
So far in the examples and discussion, we have been backtesting models based on unconditional coverage, in which the timing of our exceptions was not considered. Conditioning considers the time variation of the data. In addition to having a predictable number of exceptions, we also anticipate the exceptions to be fairly equally distributed across time. A bunching of exceptions may indicate that market correlations have changed or that our trading positions have been altered. In the event that exceptions are not independent, the risk manager should incorporate models that consider time variation in risk.
We need some guide to determine if the bunching is random or caused by one of these changes. By including a measure of the independence of exceptions, we can measure conditional coverage of the model. Christofferson2 proposed extending the unconditional coverage test statistic (ZAuc) to allow for potential time variation of the data. He developed a statistic to determine the serial independence of deviations using a log-likelihood ratio test (ZAind). The overall log-likelihood test statistic for conditional coverage (ZJ?cc) is then computed as:
LR = LR + LR- j ina
uc
cc
Each individual component is independently distributed as chi-squared, and the sum is also distributed as chi-squared. At the 93% confidence level, we would reject the model if LRcc > 5.99 and we would reject the independence term alone if LRind > 3.84. If exceptions
2. P.F. Christofferson, Evaluating Interval Forecasts, International Economic Review, 39 (1998),
841-862.
Page 32
2018 Kaplan, Inc.
Topic 3 Cross Reference to GARP Assigned Reading – Jorion, Chapter 6
are determined to be serially dependent, then the VaR model needs to be revised to incorporate the correlations that are evident in the current conditions.
Professors Note: For the exam, you do not need to know how to calculate the log-likelihood test statistic for conditional coverage. Therefore, the focus here is to understand that the test for conditional coverage should be performed when exceptions are clustered together.
B a s e l C o m m i t t e e R u l e s f o r B a c k t e s t i n g