Ch5 Backtesting VaR

1. Backtesting
VaR models are only useful insofar as they predict risk reasonalby well. -> Modeling validation
*Model validation: general process of checking whether a model is adequate; backtesting, stress testing, & independent review & oversight.

1-1. Definition
A formal statistical framework that consist of verifying actual losses are in line with projected losses.

1-2. Backtesting VaR
- systematically comparing the history of VaR forecasts with associated portfolio returns
- reality checks -> reexamining for faulty assumptions, wrong parameters, or inaccurate modeling
- central to the Basel Committee's ground-breaking decision to allow internal VaR models for capital requirements.

2. Setup for Backtesting
i. number of exception: number of exceedences; too may excpetions -> the model underestimates risk
ii. the fit between absolue value of the daily profit & loss against 99% VaR, daily price volatility
iii. VaR measures assume that the current portfolio is frozen over the horizon (static).
**In practice, the actual portfolio is contaminated by changes in its composition: intraday trades, fees, commissions, spreads, & net interest income. => backtesting usually is conducted on daily returns to minimize the contamination.
iv. Returns selection
- actual portfolio return
- hypothetical return: obtained from fixed positions applied to the actiual returns
- cleaned return: actual return minus all non-mark-to-market items.
* choice to use either hypothetical or cleaned returns
v. both actual & hypothetical returns should be used for backtesting because both sets of numbers yield informative comparisons.
vi. passing backtesting with hypothetical but not actual => the problem lies with intraday trading
not passing backtesting with hypothetical => modeling methodology should be reexamined

3. Model Backtesting with Exceptions
Confidence level & statistical decision problem, accept or reject decision
The choice of the level for the test is not related to the quantitative level p selected for VaR. -> The decision rule may involve a 95% confidence level for backtesting VaR numbers, which are themselves constructed at some confidence level, say, 99% for the Basel rules.

3-1. Failure Rates
- faulure rate: N/T where N = the number of exception, T = sample of size (day)
- nonparametric: simply counting the number of exceptions
- classic testing framework for a sequence of success & fauilures (Bernoulli trials)
- binomial probability distributions:

Expected Value: E(x) = pT & Variance: V(x) = p(1 - p)T
**central limit theorem & approximation of the binomial distribution by the normal distribution

3-2. Type I & Type II Error
Type I Error
- rejecting an accurate model

Type II Error
- accepting an inaccurate model

**tradeoff between Type I & Type II Error => test is powerful if it creates a low type 1 error rate and a very low type 2 error
- the choice of the confidence level, the decition rule to reject the model, for the decision rule is not related to the quantitative level p selected for VaR.

***Kupiec's approximate 95% confidence regions, defined by the tail points of the log-likelihood ratio -asymptotically distributed chi-square with one degree of freedom under the null hypothesis that p is the true probability. LRUC is the test statistic for unconditional coverage.
It is difficult to backtest VaR models constructed with higher levels of confidence because detection of systematic biases becomes increasingly difficult for low values of p

e.g.) probability level p = 5%, if T = 252 then 6 < N < 20, if T = 1000 then 37 < N < 65
for T = 252 [6/252 = 0.024, 20/252 = 0.079]
for T = 1000 [37/1000 = 0.037, 65/1000 = 0.065]
=> the interval shrinks as the sample size extends

3-3. Holding period for VaR
two theores about choosing a holding period:
i. the holding period should correspond to the amount of time required to either liquidate or hedge the portfolio -> VaR calculate possible losses before corrective action could take effect
ii. the holding period should be chosen to match the period over which the portfolio is not expected to change due to nonrisk-related activity
**holding period is more significant than the confidence level

4. Basel Rules
The Basel rules for backtesting the internal-models approach are derived directly from failure rate test. The current verification procedure consists of recording daily exceptions of the 99% VaR over the last year (= 2.5 instances).

**Basel Penalty Zones
- Green: 0 to 4
- Yellow: 5 to 9
- Red: 10+

The penalty for banks is subject to their supervisors' discretions. Four categories of causes for exceptions are:
- basic integrity of the model: positions reported incorrectly, an error in the program code => penalty applied
- model accuracy could be improved: model not measured risk with enough precision => penalty applied
- intraday trading: positions changed during the day => penalty considered
- bad luck: volatile market, correlations changed

4-1. High VaR confidence level
- Type I error: 10.8%, Type II error: 12.8% => not powerful

**increasing the power of the test
- lowering the required VaR confidence level to 95% -> sharply reduces the probability of not catching an erroreous model
- increasing the number of observations. e.g.) T= 100

4-2. Conditional Coverage Model
So far the framework focuses on unconditional coverage (ignoring time variation or conditioning in the data).
- with a 95% VaR confidence level: E(x) = 13, if we observed 10 of these exceptions occurred over the last 2 weeks -> verification system should be designed to measure proper conditional coverage.
- Christofferson's LRCC
LRCC = LRUC + LRind
where:
LRind = the serial independence of deviations using a log-likelihood ratio test

-> reject the model if LRCC > 5.99
If exceptions are determined to be serially dependent, then the model needs to be revised to incorporate the correlations that are evident in the current conditions.