Is there a German word for the pleasure you experience when someone calls out economists who are consistently wrong? Because that's what I'm feeling. RNZ's piece on the problem with economists' forecasts reveals an issue that is not all that new, but people must have short memories. It's an issue that is dear to my heart. It's time to start calling economists out on their bad forecasts.
Full disclosure: I'm currently slowly making my way through Nate Silver's The Signal and the Noise: Why so many predictions fail -- but some don't. It's all about how to deal with uncertainty in forecasting. I'm enjoying it.
The problem with economists' forecasts is that they're often wrong. Embarrassingly wrong.
The article gives economists a bit of a ribbing, and fair enough. But the fault doesn't wholly lie with economists -- they are after all dealing with multiple variables and layers of uncertainty attached to each. And economists presumably provide these forecasts because they're something the public and policymakers want to know. Forecasts are useful because policy responses can be slow: you really want to get your underground bunker built before the hurricane hits. Economic forecasts fill a necessary gap.
I could almost live with that, if the forecasts were delivered with appropriate humility and caveats. But often they're not -- or at least, the soundbites that make it to print do not reflect that. It's the over-confidence that bugs me. And while some of this may come down to naivety about the way media works, it's a little hard to swallow when many of these economists are seasoned media commentators.
There are also incentives at play. Economists are rewarded by the media for their grandiose predictions: they're the ones who get headlines, airtime and name recognition. By the time future events eventuate, the public memory of the exact forecast is long gone, all that remains is the memory that 'this economist said something smart on TV this one time'. The media too have an incentive to publish clear and confident forecasts: they make good soundbites.
The RNZ article suggests the media include more caveats and disclaimers when broadcasting economists giving their forecasts. That would help. But we can go further.
Here are 3 things that I think would help incentivise better reporting of forecasts:
1) Have a public database of economists' past forecasts compared with the actual outcome. We already see a bit of this with political polling. Keeping a track record of economists' past claims has to be a good start to incentivising better commentary. Those with low confidence in their forecasts might be more reluctant to comment to media if they know they'll be held accountable. And it incentivises those who do talk to media to take more care with their claims. It's also a useful tool for the public to compare different takes and understand how forecasts stack up to reality.
2) Improve the supply-side of the equation. This is cribbed from the aforementioned Nate Silver book, who in turn references George Mason University economist Robin Hanson. Robin Hanson's recommendation is to use prediction markets, where the public can place bets on economic or policy outcomes. Hanson's insight is that there are simply very few incentives to produce good forecasts. By adding a real financial stake to forecasting, the incentives increase and shift away from just looking good to peers (e.g. being on TV and making headlines). An advantage of prediction markets is that they're dynamic: they can provide real-time information by being constantly updated when new information comes to light. Anyway, all of this sounds good and reasonable. But then I remember we don't have iPredict anymore in NZ so...there goes that idea.
3) Improve the demand-side of the equation: publish the margins of error, or confidence intervals, in forecasts. This is also cribbed from the Nate Silver book. The idea is to encourage the media and public to be better consumers of forecasts. Again, the public are already familiar with this format with political polling (whether they understand margins of error, I cannot comment). Good economists know and will admit that their forecasts carry a range of uncertainty. Being able to communicate that uncertainty is not only useful for the public, but is also fairer on the economists who should know and want to highlight the uncertainty of their forecasts.
Not all economists who make forecasts are bad, and not all forecasts are bad. What we need are better mechanisms for discerning which ones to take more seriously.