Ian Cooper Presentation

The behaviour of
sentiment-induced share returns:
Measurement when
fundamentals are observable

Richard Brealey
Ian Cooper
Evi Kaplanis
London Business School



Share prices and sentiment

  • Many theories about the effect of sentiment on share prices are expressed in terms of deviations from fundamentals
    — Daniel, Hirshleifer, and Subrahmanyam (1998),
    Barberis, Shleifer, and Vishny (1998)
  • Barberis et al (2012):
    — Sentiment pushes prices away from fundamentals in the
    short term
    — In the medium-term they mean-revert
  • The effect should be concentrated in “hard-to-arbitrage” stocks
  • On the other hand, there is evidence of market-wide sentiment effects (Arif and Lee (2014)) which may affect all stocks



The importance of controlling for fundamentals

  • Results about sentiment and returns can be different if controls for fundamentals are not included (Barberis, Shleifer, Wurgler (2005))
  • Various methods have been tried:
    — Use a sample where fundamentals are presumed to stay the same (Barberis et al)
    — Use control variables: Derrien and Kecskés (2009) show the effect of sentiment on equity issuance disappears when controls are included
    — Use a DCF estimate of fundamental value: Brown and Cliff (2005) show that DDM values are positively related to sentiment. But DDM values are hard to estimate and depend on a discount rate model.
    — Use long-short portfolios: BW use long hard-to-arbitrage, short easy-to-arbitrage portfolios, depends on long-short portfolio being neutral with respect to fundamental factors
  • Our study: Use a sample where we believe we have a good proxy for fundamental value

Our study

  • Use a sample of firms where we should have an excellent proxy for fundamental value: Upstream oil stocks
    — Empirically, oil price explains cross-section of values (Miller and Upton (1985a), (1985b))
    – Theory says discount rate uncertainty is not so important for these stocks once the oil price is controlled for (Hotelling)
  • Split the stock return into the part caused by changes in fundamentals and the part that represents the deviation from fundamentals
  • Test whether sentiment predicts the behavior of the deviation from fundamentals in the way that theory says it should
  • Two measures of sentiment:
    — Baker-Wurgler index should predict reversion to fundamentals
    — Retail investor sentiment index should predict momentum trading

Main results

  • Sentiment predicts returns in the expected way
    a. Baker-Wurgler (“BW”) sentiment predicts mean-reversion
    b. Retail “Bullish” sentiment predicts momentum
  • The effect appears only after the major increase in interest in commodities, particularly oil, in 2000
  • Only then do fundamentals explain a large part of returns
  • The effect appears to work via the fundamentals themselves rather than through deviations from fundamentals
  • Economic size is about 0.3% per month from bullish sentiment and about 0.3% per month from BW sentiment
  • The effect appears in a Hi-Lo portfolio, but because it has a net exposure to fundamentals


  • 121 US oil exploration and production stocks (SIC 1311)
  • Monthly data 1983-2011 (CRSP)
  • Fundamentals measured by spot oil price, spot gas price, oil contango
  • Equally weighted portfolio of all stocks
  • Sub-portfolios sorted by variance over prior 60 months
  • Highest tercile variance portfolio minus lowest tercile variance portfolio (“Hi-Lo”)
  • BW sentiment
  • American Association of Individual Investors “bullishness” survey

Serial properties of our data

  • Our data show standard properties of returns data:
    — Short-term momentum
  • Lehman (1990), Jegadeesh (1990), Jagadeesh and Titman (1993)
    — Longer term mean-reversion
  • Poterba and Summers (1988), Lo and MacKinlay (1998), Cutler, Poterba, and Summers (1991)
    — Consistent with excess volatility relative to fundamentals
  • Shiller (1981), LeRoy and Porter (1981)

Variance ratios of our sample

Variance ratios of the All-stocks portfolio
relative to the 1-month variance rate
Equally-weighted all US oil production and exploration
stocks listed on NYSE (121 stocks)


Sentiment measures

  • Baker-Wurgler composite index of sentiment (index of IPO volume and returns, market volume, relative pricing of high and low volatility stocks)
  • Survey data (Qiu and Welch (2004), Brown and Cliff (2004, 20025))
    — We use American Association of Individual Investors survey: Proportion who report that they are bullish (“Bullish” sentiment)
  • Hypothesis is that BW sentiment is an index of the level of mispricing, so returns following high BW sentiment will be low, reflecting reversion to fundamental value, and that this will be more pronounced for hard-to-arbitrage stocks
  • Hypothesis is that Bullish sentiment picks up momentum trading and high bullish sentiment will be followed by high returns


Baker Wurgler

  • BW sentiment
    monthly serial
  • Bullish
    monthly serial
  • Monthly
    between them


Sub-period analysis: Motivation

  • For our sub-period analysis, we split the period at the end of 2000
  • Open interest in oil futures (CFTC) and flows into commodity hedge
    funds (Managed Futures Funds, TASS data) increased significantly


Preliminary results

  • Across the sample period the fundamentals explain 41% of the
    variance in the All-stocks portfolio returns
  • Hi-Lo portfolio returns are not well hedged against fundamentals, oil
    and gas price are highly significant factors for Hi-Lo returns
  • Correlation between returns and lagged sentiment are larger for high-
    variance portfolio and the Hi-Lo portfolio: appears at first sight to be
    consistent with hard-to arbitrage hypothesis
  • Including controls reduces considerably the significance of sentiment
    variables are reduced considerably for both portfolios for entire
    period and for both sub-periods

Splitting returns

  • To avoid overfitting the coefficients, each month we split
    the returns into fundamentals and non-fundamentals using
    lagged parameter values
  • Estimate fundamentals regression for prior 60 months:
    Rit = ai + bi Δ WTIt + ci Δ GASt + di Δ Conti + uit
  • Fundamentals: Returns to WTI spot price, Gas spot price,
    change in Contango of 6th-1st futures price
  • Use estimated coefficients and actual realizations of{ΔWTIt, ΔGASt, ΔContt}to estimate fundamental return for that month
  • Remainder of return is non-fundamental
  • Roll the windows

Regression structure

  • Fundamental return from t-1 to t is: Ft – Ft-1
  • Non-fundamental return from t-1 to t is: NFt – NFt-1
  • Regress Ft – Ft-1 and NFt – NFt-1 on lagged returns and lagged sentiment measures
  • Include BW sentiment, St, and Bullish sentiment, Bt, in a VAR

VAR Structure and Hypotheses



VAR for entire period: Hi-Lo portfolio


VAR Results: Summary

  • BW predicts reversion to mean
  • Bullish predicts momentum
  • Both effects operate through fundamentals (i.e. oil and gas prices)
  • Hi-Lo portfolio is exposed to fundamentals
  • No effect of sentiment through deviations from fundamentals
  • BW sentiment responds to fundamental return, not non-fundamental return

Sub-period results: Hi-Lo portfolio


Different measurement intervals


Sentiment transmission in the second period




  • Including lagged market return does not change results
  • Looking only at high variance portfolio alone does not change results
  • Including 274 NASDAQ exploration and production stocks:- High vol portfolio is almost exclusively NASDAQ
    – Hi-Lo is better hedged against fundamentals
    – Effect of sentiment still appears only in second period
    – Predictability of Hi-Lo return is lower
    – Effect of sentiment appears only through fundamentals
    when return is split (not in current paper)

What if the oil price is not “fundamental”?

  • Evidence that oil and gas prices are affected by sentiment (Pindyck (1993))
  • Regress oil price change on sentiment and “deep” fundamentals
  • Deep fundamentals are changes in:
    – Oil production, oil consumption, gdp, proven oil
    reserves (annual data), oil inventories (quarterly)
  • Low power test, but coefficient on BW is mildly significant and negative, coefficient on bullish is mildly significant and positive
  • Consistent with channel of influence of sentiment being
    through oil price

Regression of changes in the oil price on deep
fundamentals and sentiment


Results: Summary

  • Sentiment predicts returns in the expected way
    a. Baker-Wurgler sentiment predicts mean-reversion
    b. Retail bullish sentiment predicts momentum
  • Economic size is about 0.3% per month from bullish sentiment and about 0.3% per month from BW sentiment
  • The sentiment effect for these stocks operates through fundamentals(i.e. oil and gas prices)
  • Significant effects appear only after 2000 when:
    – interest in oil as an asset increased considerably
    – fundamentals became more important in determining upstream oil stock returns
  • Inconsistent with theories where sentiment causes deviations from
    fundamental value
  • Inconsistent with with hard-to-arbitrage theories about sentiment

Caveats and issues

  1. Maybe our results are sector-specific because, in this
    instance, what we use as fundamentals are themselves
    tradeable (Basak and Pavlova (2014))
  2. The highly time-varying effect of sentiment here has a
    clear cause that may not be present for other stocks
  3. Our result that that the Hi-Lo portfolio loads on fundamental factors may not apply in other cases
  • Raises the question of what “fundamental value” means in
    this instance
  • Raises question of why sentiment should predict returns
    in “easy-to-arbitrage” assets like oil