The comparison of macroeconomic forecasts from various official institutions calls for prudence. Projections relating to the same variables and the same period often exhibit numerical discrepancies. Different factors need to be taken into account when analysing the results: the type of model used, the time of publication, the form and type of policies taken into account and the incorporation of elements of uncertainty.
One of the most important factors that explain the diversity of results is the existence of different forecasting methods. These are set on economic models that are generally based on different assumptions. Among the wide variety of models, some have a more theoretical basis and richer microfoundations. Others are of a hybrid nature and use, on the one hand, theoretical foundations to define economic agents’ choices and, on the other hand, more empirical features, by econometrically estimating behaviour equations (for example, the ECB’s Area-Wide model, the NiGEM used by OECD and the CFP’s model). Various methodologies/models are often employed in the same exercise to achieve robust results.
The time of publication of the exercise should also be taken into account. The most up-to-date information is not uniform over time: quarterly figures are published by the statistical authorities only during the following period. This means that the various institutions may base their forecasts on different information. The databases may even be different between exercises run by the same institution, so any comparison should be made with caution. The diagram below shows the normal forecasting calendar for the Portuguese economy delivered by the main international (European Commission, IMF and OECD) and national institutions (MF, Bank of Portugal and CFP) and the releasing of National Accounts.
Normal calendar for forecasts and the releasing of National Accounts
Notes: the calendar shows the usual month of publication of each exercise; the position within each month is arbitrary as the day of the month is not predefined; the international institutions generally publish other forecasts in documents about Portugal but at irregular intervals.
Another aspect is related to the no policy change scenarios adopted by each institution. These assume that the economic policy measures in place will remain unchanged over the time horizon. No policy change scenarios do not set out to achieve the best forecast possible, but rather to act as a benchmark against which the impact of new measures may be measured. In the short run most institutions contemplate measures for which legislation is in place or have been officially announced. Unless indicated otherwise, it is assumed that they remain unchanged over time. However, for the medium run, policy measures that could be implemented but whose degree of certainty varies, are assessed. In such cases the general rule is to exclude those measures for which sufficient information regarding their meaning or to the authorities’ intention to implement them is not available. As each institution makes its own assessment this can lead to different conclusions.
Furthermore, the lack of a precise definition as to what is “sufficient information” also gives rise to distinct no-policy-change scenarios. Some institutions such as the Bank of Portugal, the IMF and the CFP (Subsection 3.1.1) provide details of the fiscal measures taken into consideration. The IMF also assumes that when there is no information, the primary structural balance remains unchanged (except where stated otherwise). Other institutions such as the European Commission, the ECB, and the OECD do not specify in detail which measures have been included. These different criteria difficult the comparison between different projections.
Finally, it is important to consider the different institutions’ opinions in relation to the qualitative factors that affect the forecasts. All forecasting exercises involve uncertainty that is hard to quantify but should be accounted for. Examples include the economic agents’ reaction to budget stimuli or external demand developments. The weighting of these factors is influenced by the institutions’ different visions or by their access to information on unpublished variables. At the end of the day, under scenarios involving greater uncertainty, the decision on certain projection assumptions is constrained by each institution’s context and criteria.
Despite these limitations the comparison of different forecasts is a valid exercise. Nonetheless the influence of these factors suggests they should be interpreted with prudence, even when comparing forecasts from the same institution. The assumptions underpinning each projection exercise should be analysed and disclosed along with comparing exercises.
“Box 3 – Comparability of different institutions’ macroeconomic forecasts”, initially published in the Report 3/2016, “Public Finance: Position and Constraints 2016-2020”, from March 2016.