Bryan Kelly Slides

Systemic Risk and the Macroeconomy: An Empirical Evaluation

Stefano Giglio1 Bryan Kelly2 Seth Pruitt3

1 Chicago Booth and NBER


• Many systemic risk measures proposed in aftermath of 2007 financial crisis

• Individual measures explored separately, little/no empirical analysis as a group

• Preliminary step: provide a basic quantitative description of existing SR measures

  • Construct 17 previously proposed measures of systemic risk in the US and 10 measures for the UK and EU

  • Extend series as far as possible, at least past 1990s (contrast with last decade)

• Each measure shown to work to capture some particular aspects of financial distress

• Not a clear picture of which aspects of systemic risk they are capturing or what is the criterion to judge them


•Three fundamental questions about systemic risk

  •Definition – What is SR?

  •Measurement – How do we measure and detect SR?

  •Welfare – How does SR affect us?

•We start from Welfare to evaluate Measurement, and aim to provide stylized facts that say something about Definition

Three Main Objectives

1.Propose a criterion for evaluating SR measures

  •To be relevant for policy,interested in SR to the extent that it has consequences for economic welfare

  •Can we detect ways that SR affects the real economy?

  •Quantify this by looking at conditional quantiles of real activity variables and their relation with SR measures

Three Main Objectives

•We apply the criterion and evaluate how many proposed measures of SR actually associate with greater risk for the real economy

  •Only a few SR measures are associated with higher risks to the real economy

•Perhaps each measure contains some information that is useful to capture SR

  •Do not always move together

  •May capture different aspects of distress in financial markets

Three Main Objectives

2.Propose dimension reduction techniques to detect relationship between the real economy and SR measures jointly

  •Suppose a latent SR factor drives the SR measures and the quantiles of future real macro variables

  •We propose two dimension reduction estimators to estimate this factor; prove they are asymptotically unbiased

  •Empirical finding: Information gain from aggregation → SR measures as group informative about macro variable

Three Main Objectives

3.Produce stylized facts on the relation between SR and the macroeconomy

  •We can study which measures work and which don’t work, by our metric

  •These stylized facts can aid future work to dig deeper into the underlying mechanisms

Three Main Objectives

Three stylized facts

  1.SR affects the lower tail of the future macroeconomic distribution, not the center

  2.Financial volatility is informative

  3.Government responds to distress but not enough

Summary: Systemic Risk and the Real Economy

  •Results reach positive conclusion regarding the empirical systemic risk literature

  •Increases in SR index associated with a large widening in the left tail of economic activity

  •One s.d. increase in SR shifts the 20th percentile of the IP growth distribution down by more than 50%, from around -1.4% per quarter unconditionally, to -2.2%

  •During crises 20th percentile drops below -3% per quarter, twice as large as in normal times

  •SR index also predicts reactions of policymakers: the 20th percentile of changes in the federal funds rate drops by 60%, from -50bps to -80bps.

Landscape of Systemic Risk Measures

Which Measures?

1.Aggregated versions of institution-specific measures:CoVar,ΔCoVar, MES, SES, MES-BE.

2.Comovement and contagion:Absorption Ratio, DCI,International Spillover Index.

3.Liquidity and credit:AIM, TED Spread, Default spread, Term spread

4.Instability and volatility:Volatility, Turbulence, Book Leverage,Market Leverage, Size Concentration

•We do not cover direct linkages and CDS-based measures

•We construct measures for the US, plus UK and Euro Area (France,Germany, Italy, Spain

Systemic Risk Measures


Summary of Comovement Among Measures

•All spiked during financial crisis, not surprising given a posteriori origins

•Correlations among measures are low on average⇠0.2.

  •Some measures lead others (CoVaR, credit spreads, volatility), some only “coincident” indicators

•In long sample, many reached similar levels as in recent crisis. Interpretations include

  •Measures are simply noisy

  •Sometimes capture stress in financial system that does not result in economic crises either because either policy response diffused instability or system stabilized itself

Macroeconomic Criterion and Accompanying Tests


•Financial distress impacts real outcomes through capital / credit /liquidity contraction

•Bernanke & Gertler (1989), Kiyotaki and Moore (1997), Bernanke,Gertler & Gilchrist (1999), Brunnermeier & Sannikov (2010), Gertler & Kiyotaki (2010), Mendoza (2010), He & Krishnamurthy (2012)

•Emphasis on non-linearity. Distribution of real outcomes changes when degree of SR changes. Particularly interested in effects at low quantiles of real outcomes

•”… what measurements will be the most fruitful to support our understanding of linkages between financial markets and the macroeconomy is an open issue.” Hansen (2012)

A Criterion for Evaluating Measures

•SR indicator should demonstrably associated with future macroeconomic outcomes

  •Important from regulatory point of view

  •Summarizes vast amount of economic activity and decision-making

•This criterion addresses the field’s need of an empirical description of link between proposed financial markets and the real economy

Operationalizing Criterion: Quantile Regression

•Operationalize: Quantile regression test for the SR measure’s ability to predict distribution of future macro shocks

  •OLS models the conditional mean relationship b/w X and y

  •QR models the conditional quantile relationship b/w X and y

•Main advantages

  •Reduced-form implementation of theoretical SR/macro association (e.g. He and Krishnamurthy)

  •Broader view of the conditional distribution of y given X

  •Can flexibly evaluate the relationship at different parts of y’s distribution
    •eg. lower tail, central tendency, upper tail

Quantile Regression in Thirty Seconds


Loss Function


Example of No Quantile Relation: Absorption Ratio


Example of Strong Quantile Relation: Turbulence


Dependent Variable: Macroeconomic Shocks

Macroeconomic target yt is a shock derived from:

  •Industrial Production

  •CFNAI, or CFNAI subcomponent

    •Aggregate index (Total)

    •Production and Income (PI)

    •Employment, Unemployment and Hours (EUH)

    •Personal Consumption and Housing (PH)

    •Sales, Orders and Inventory (SOI)

yt is a one quarter-ahead “shock,” or innovation to AR(p)

  •US: IP 1946-2011, CFNAI 1967-2011; less for UK, EU Robust: Recursive; AIC; ARX(p)

Benchmark Quantile Forecast Analysis


Out-of-Sample 20th Percentile IP Shock Forecasts


Out-of-Sample 20th Percentile CFNAI Shock Forecasts


Results for Individual Measures

  •Individual predictors: Mixed, somewhat weak out-of-sample performance

    •Notable exception: Financial volatility variables, leverage

  •Can we fruitfully put them together?

Dimension Reduction Technique

Quantile Regression with Many Predictors

  •How do we forecast quantiles when number of predictor variables is large?

  •Multiple QR works poorly due to standard “many-predictor” issues

  •Two new methodologies:

    •Principal Component Quantile Regression

    •Partial Quantile Regression: Analogue of PLS for quantiles



Principal Component Quantile Regression (PCQR)


Partial Quantile Regression (PQR)


Comparison of PCQR and PQR



•Latent variables system: error-in-variables (EIV) bias

•Both PCQR and PQR have a mismeasured factor in the last step

•Measurement error in QR is hard to deal with — no expression for the quantile solution

•Angrist, Chernozhukov, Fernandez-Val (2006) misspecified QR: We adapt to express the EIV bias and analyze in our asymptotic experiment under our assumptions

•Since we look at N,T limit, factors will become precisely estimated and the bias goes to zero

•PQR has EIV even in the first stage


  •PCQR: asymptotically unbiased as N,T!1,assuming all factors extracted in first stage

  •PQR: asymptotically unbiased as N,T!1,assuming a bit more structure on problem

Contribution: A methodology for constructing systemic risk indices that are asymptotically unbiased in large SR panels

Real test is how they work in finite samples

  •Simulation evidence supports the asymptotics

  •Empirically successful, as we now see ..

Out-of-Sample 20th Percentile Shock Forecasts


Robustness: Other Quantiles

Out-of-Sample 10th Percentile IP Shock Forecasts


Robustness: Other Quantiles

Out-of-Sample 10th Percentile CFNAI Shock Forecasts


What do we learn from these results?

•There is a SR factor strongly related to future downside macroeconomic risk

  •One s.d. increase in SR shifts the 20th percentile of the IP growth distribution down by more than 50%, from around -1.4% per quarter unconditionally, to -2.2%

  •During crises 20th percentile drops below -3% per quarter, twice as large as in normal times

•Financial-sector volatility may be most “fruitful” for understanding the linkages between financial markets and the macroeconomy

•Next: What can we say about the nature of SR?

Stylized Facts about Systemic Risk and the Macroeconomy

) Downside Macroeconomic Risk

  •So far we have shown the ability of SR measures to predict the lower tail of the distribution of future macroeconomic outcomes

  •What happens if we look at medians?

(1) Downside Macroeconomic Risk

Median Forecasts: Weaker predictability for central tendency


(1) Downside Macroeconomic Risk

Median Coefficient vs. 20th Percentile Coefficient t Tests


(1) Downside Macroeconomic Risk

  •All of these measures, including volatility, are much more successful for the lower tail of macro outcomes

  •Financial market distress creates or makes the economy susceptible to downside macroeconomic risk

(2) What is the Role of Policy?

  •How did policy behave in these episodes?

  •Does monetary policy respond to SR distress?

2) What is the Role of Policy?

Out-of-Sample Fed Funds Forecasts


(3) The Special Role of Financial Volatility

IP Shock Forecasts: Financial Volatility and Nonfinancial Volatility


(3) The Special Role of Financial Volatility

  •Is this about Aggregate Volatility …

    •”Uncertainty shock” literature like Bloom (2009) traces a response of average macroeconomic outcomes to increases in Aggregate Volatility (return volatility on aggregate portfolio)

  •… or about Financial Volatility in particular?

(3) The Special Role of Financial Volatility

VAR Perspective


(3) The Special Role of Financial Volatility

VAR Perspective



•Propose macro/welfare relevance criterion for evaluating systemic risk measures

  •Only a few provide this information

•Propose (factor estimation) approach to aggregating systemic risk measures and overcoming measurement difficulties faced by individual measures

  •Systemic risk factor strongly related to future macroeconomic outcomes

•Stylized facts can guide model-building
  1.Systemic risk is strongly related to downside macro risk
  2.Policy responds to SR but has limited effectiveness
  3.Financial firms’ equity volatility is most informative for future macro outcomes