LO 47.2: Explain how model risk can arise in the implementation of a model.

LO 47.2: Explain how model risk can arise in the implementation of a model.
In the previous section, we looked at the most common model errors. However, even correct models can be incorrectly implemented. This section looks at the most common implementation issues. Models may be affected by programming bugs or approximation errors, and models that seemed to work under normal conditions may have errors when tested under stressed market conditions.
Common Model Implementation Errors
Implementation error could occur, for example, when models that require Monte Carlo simulations are not allowed to run a sufficient number of simulations. In such a case, even if all the model inputs and assumptions are correct, the results may still be incorrect if insufficient time is given for the computations.
For the implementation of models, important considerations should include how frequently the model parameters need to be refreshed, including volatilities and correlations. Analysts responsible for maintaining models must consider whether adjustments should occur periodically at scheduled dates, or only when material economic events occur. Similarly, the treatment of outliers should also be considered. For example, should outliers be considered extreme outcomes only (that is, not part of the true distribution), or should they be considered part of the true distribution? Correctly answering these questions became especially important in the post-financial crisis period.
Page 118
2018 Kaplan, Inc.
Topic 47 Cross Reference to GARP Assigned Reading – Crouhy, Galai, and Mark, Chapter 15
Correctly estimating parameters like durations, volatilities, and correlations is very difficult, and implementing a model with input errors will result in inaccurate results. For example, in the 1970s the investment banking firm Merrill Lynch used incorrect hedge durations for government bonds, which resulted in a considerable loss to the firm. In another example, during the stressed conditions of the financial crisis, default correlations within structured products moved toward the binary extremes of +1 or 1. In other words, the cumulative default rates of collateralized debt obligations (CDOs) either all remained below a threshold with no defaults in any tranches, or all moved above a threshold, leading to defaults of even the AAA-rated tranches.
Common Valuation and Estimation Errors
Models also rely on the accuracy of inputs and values fed into the model, and are therefore subject to human error. Human error is particularly of concern in new or developing markets where adequate controls have not been fully defined and implemented.
Common valuation and estimation errors include: 1.
Inaccurate data. Models may use both internal and external data sources, where the responsibility for data accuracy is not clearly assigned. This could lead to errors from using inaccurate data.
2.
Incorrect sampling period length. Increasing the number of observations is expected to improve data accuracy and reduce estimation errors. Flowever, including old (and therefore obsolete) statistics could put too much weight on stale data.
3. Liquidity and valuation problems. Accurate pricing and valuation may not be possible in all markets. Prices for a particular asset may not exist in certain markets, or the bid-ask spread may be too high to offer accurate valuation.
M i t i g a t i n g M o d e l R i s k