Essays on Forecast Evaluation in Macroeconomics and International Finance Open Access
Downloadable ContentDownload PDF
This dissertation shares three common themes: (i) forecasting rare macroeconomic events, i.e. GDP declines and currency crises; (ii) the use of non-parametric methods to evaluate binary indicators, in particular, the advantages of the analysis of the Receiver Operating Characteristic (ROC) curves; and (iii) value of qualitative information from expert surveys and textual analysis in macroeconomic forecasting. Chapter 1 contributes to the literature on evaluation of the qualitative survey directional forecasts using the World Economic Survey (WES) for the U.S. economy in 1989q1-2015q4. I offer a methodology which combines the ROC curves analysis with the traditional analysis of the contingency tables. I propose criteria to assess in-sample and out-of-sample directional predictive value of the binary indicators, including directional forecasts from the qualitative surveys. I find that the WES has high out-of-sample value in forecasting movements in GDP and consumption, and moderate for imports, trade balance, inflation, and short-term interest rate. It has no value in predicting changes in investment and exports. I also motivate and confirm that the WES Economic Climate (EC) indicator is as a more accurate predictor of future movements in the real GDP than future expectations alone. Additionally, I show that the ROC-optimal thresholds yield more accurate predictions than their alternatives proposed by Hutson et al. (2014).Chapter 2 re-examines indicators of currency crises suggested by Kaminsky and Reinhart (1999) and subsequent studies using the ROC curves analysis. I utilize a training set (1975-1995) to confirm a list of indicators with the in-sample predictive value, and test their out-of-sample using data for 1996-2002. Four variables have both in-sample and out-of-sample predictive value: the deviation of the real exchange rate (RER) from a trend, the foreign reserves, the ratio of broad money M2 to reserves, and the decline in exports. I show that the ROC-optimal thresholds issue more accurate signals than the minimum noise-to-signal ratio previously used in the literature. I also employ modified ROC curves to display the relationship between the precision of sent signals and recall of crisis episodes. Finally, I propose forecast combinations using several ad-hoc rules which help to improve forecast accuracy.Chapter 3 contributes to the discussion of asymmetric information about the U.S. economy between the Federal Reserve System (FRS) and the Survey of Professional Forecasters (SPF) via textual analysis of the Federal Open Market Committee (FOMC) minutes. It builds on Stekler and Symington (2016), who scored the texts of the FOMC minutes in 2006-2010 to produce the indexes for the current and future outlooks and their calibrations to the U.S. real GDP. I extend their time-series adding 26 years of observations to cover 1986Q1-2016Q4. Following Ericsson (2016), I interpret the derived calibrations (FMIs) as elicitcasts of the Greenbook (GB) forecasts. Results indicate that the FMIs are unbiased, efficient, rational, and contain the same informational advantage as the GB forecasts. The forecast encompassing tests suggest that both the FMIs and the SPF forecasts contain their own unique knowledge and can learn from each other. I find that the SPF forecasters already pay close attention to the FOMC minutes available to them at the time of forecast deadline and efficiently use its information in their set. Yet, they could improve their forecasts should the FOMC minutes from the first quarterly meetings become available without a three-week publication lag. During their second quarterly meetings, the FOMC policy-makers accounted only for their own earlier assessment of the U.S. macroeconomy – they did not put weight on the SPF forecasts released a few weeks earlier in the same quarter. The results are robust to the use of alternative scale.Overall, I find that directional forecasts are informative. The qualitative WES survey can produce accurate directional macroeconomic forecasts. The ROC curves analysis helps to set an association between the consensus scores and the growth rates as well as to find accurate indicators of currency crisis. The qualitative statements from monetary policy deliberations can be converted in to the GDP growth forecasts with unique information about the US economy.