I'm going to pass on weighing his arguments, as those are a little outside my wheelhouse. There's an instructive figure borrowed from the Institute for Health Metrics and Evaluation contagion model that bothers me.
Here's something I wrote last week.
The best you can do, dear reader, is improve the precision of the mean fit, which involves reducing the margin of error. Too much of the reporting and the briefing focuses on those solid lines: actual and projected cases. Those clouds that suggest the upper and lower bounds on the projection matter. When you see the clouds becoming less tall, then perhaps the policy makers will start talking more seriously about reopening life as we used to know it.That's what bothers me about the modelling. Note how the margin of error goes to zero as the forecasts go to zero, and there appears to be no margin of error estimate once past the expected end of new cases, somewhere around Independence Day. Now, I'm way outside my area of expertise, and I remember just enough econometrics to be dangerous, but I seem to recall that one of the hazards of macroeconometric forecasting was that the forecast error got larger, the farther out of sample (which is to say, the greater the time between the mean of the actual data you were using to calibrate and the out-year forecasts you were making) your projection ran.
In the above figure, that mean occurs somewhere around the end of March, and the projections are being updated on the basis of the (more favorable than projected) returns coming in. But the out-of-sample forecast error is also getting smaller, with the arrival date of the last case being the point estimate and there being no uncertainty about that last case coming after that day. Call me puzzled.
SECOND SECTION: Rick Moran at Pajamas Media, also puzzled. "The researchers argue that policymakers may believe that policies well short of the draconian measures taken by China in Wuhan prevented the horrific loss of life when, in fact, the model giving those numbers in the first place was fatally flawed."


1 comment:
Part of the problem is the media discussing this model ... don't know what they're talking about.
There really is not much sense in which you can forecast this (or any other epidemic). The reason is that bugbear in time series that the sample you draw is typically unrepresentative of the population you're trying to model. So, a forecast of this epidemic, since any sample we draw will be from the upswing, will trend forever.
More properly, models like the IMHE one are doing calibrated simulations. You get some math that looks right, you plug some point estimates ideally drawn from solid literature, and you see what comes out.
Forecasts from statistical models show widening confidence bands because we know something about the distribution of forecasts.
Simulations show something that looks like confidence bands (but really are not) derived from inputs with random variation. What they're really saying in those charts of the IMHE is that eventually the variables of interest evolve to the point where the sensitivity analysis on the inputs no longer matters much.
If that doesn't sound comforting ... it's because it isn't.
Post a Comment