Inefficient Forecast Narratives: A BERT-Based Approach

Abstract

This paper contributes to previous research on the efficient integration of forecasters’ narratives into business cycle forecasts. Using a Bidirectional Encoder Representations from Transformers (BERT) model, I quantify 19,300 paragraphs from German business cycle reports (1998–2021) and classify the direction of consumption forecast errors. By testing the model on an evaluation sample, I find a highly significant correlation of modest strength between predicted and actual sign of the forecast error. The correlation coefficient is substantially higher for 12.8 % of paragraphs with a predicted class probability of 85 % or higher. By qualitatively reviewing 150 of such high-probability paragraphs, I find recurring narratives correlated with consumption forecast errors. Underestimations of consumption growth often mention rising employment, increasing wages and transfer payments, low inflation, decreasing taxes, crisis-related fiscal support, and reduced relevance of marginal employment. Conversely, overestimated consumption forecasts present opposing narratives. Forecasters appear to particularly underestimate these factors when they disproportionately affect low-income households.