Last year I posted a series of long-term predictions about covid. The first batch I’d made around March 15th, but failed to publish at the time. The second batch was made on April 24th. Making and evaluating public predictions is a habit I’m trying to pick up from folks like Scott Alexander. Scott has recently graded his predictions from last April, so I’ve been inspired to do the same.
I’m going to discuss both sets of predictions, and my reasoning, and try to tease out where I was right, where I was wrong, and what I’ve learned.
From March 15th:
1. a. The coronavirus peak will not arrive before May 1st – 99%
b. The Georgian government will maintain social distancing policies, including school closures and the recommendation to work from home, until May 1st – 80%
c. School closures will continue until June 1st – 60%
d. Schools will remain closed through the end of the school year – 50%
1.a. I’ve already called myself on vague language like “the peak”, which could be interpreted to mean many different things. However, I think it’s reasonably obvious in retrospect that nothing resembling a covid peak happened between March 15th and May 1st 2020, so I’m giving myself this one.
1. b, c, d – Well, these all happened, so I give myself some points for that. I’ll note a couple of other things, though. Phase 1 reopenings started here on April 27th, and the curfew was lifted by May 23rd. In retrospect, we could have opened schools safely in May or June, and so I think predicting that school had a 50/50 chance of reopening by the end of the year was reasonable. Indeed, despite my extreme risk aversion on covid, I felt comfortable attending small, in-person gatherings in Tbilisi by the end of May. I understand that at the time we were still operating under an environment of uncertainty. However, consider this – we could have opened schools in May and kept them open through the summer to make up for lost time! There’s no iron law that school has to be from September to June. Instead, the government decided to open restaurants and tourist accommodations and keep schools closed. My takeaway is that predicting something like school closures is less about the actual path of the pandemic and more about some aspects of human behavior and psychology that I don’t actually have a great handle on. But also, to claim on March 15th that there was a 50% chance schools would stay closed all year was radical. When I expressed this opinion to other teachers, they gave me That Look – like, “okay, you’re talking about nerd stuff, but I’m talking about reality.”
2. a. The coronavirus peak will arrive between May 1st and June 1st – 30%
b. The coronavirus peak will arrive between June 1st and July 1st – 30%
c. No coronavirus peak (it is not seasonal, or southern hemisphere cases spike in a way that requires continued countermeasures in northern countries) – 39%
Georgia had a peak in active cases on May 2nd. Globally, the peak in active cases seems to have been January 31st 2021 – which I’m scoring as option c), because indeed what happened was that the situation required continued countermeasures in northern countries. What I was trying to express with this prediction is that I thought coronavirus would be seasonal (it was) but I also thought globally the seasonal-ness would balance out (it did). What I’ve learned is I need to be a lot more clear about what I’m actually trying to predict, and that sometimes phrasing the question the way a betting market would phrase it (e.g. using precise dates rather than saying “covid will be seasonal”) sometimes detracts from that clarity. Looking back to one year ago, it seems we did have an accurate picture of the course the pandemic would take, and the Spanish flu of 1918 would turn out to be a very good predictive baseline.
3. If coronavirus peaks by summer (60%), then :
a. Coronavirus will have a second peak in fall – 90%
b. Starting in September – 50%
c. Starting in October – 30%
d. Starting in November – 10%
There’s a clear consensus that we did indeed get a second wave in fall. In many countries this took the form of a graph with two very obvious peaks. In other countries there was more of a rise to a plateau, a leveling-off in summer, and then a huge fall peak. The global numbers follow this second pattern – not a summer peak, but a summer plateau, followed by a peak in fall. I’ll give myself credit for being highly confident that there would be a fall peak, although thinking back honestly, I thought the first peak would be higher and the second peak lower, but in reality it was the reverse – the first peak was so low that it was a plateau in some places, whereas the second peak absolutely dominated.
I think the reason for this peak/wave structure is that we, locally and globally, did a phenomenal job of crushing the curve in the first wave – yes, even the countries that did this badly seem to have done it well enough to completely deform the graph and, indeed, flatten the curve – that is, until fall, when the combination of lockdown fatigue and cold weather hit us all with a double-whammy. But you can see how having a global strategy – even one which was poorly-implemented in places – made a difference when compared with previous pandemics.
4. Therefore:
a. Social distancing/school closures by October 1st – 45%
b. by November 1st – 70%
c. by December 1st – 80%
My belief was that the fall peak would cause school closures and other restrictions to be imposed by December 1st, and they were. Georgia postponed school openings in September until October 1st, and then during October, the government closed some schools intermittently including 42 kindergartens on October 12th, and then they closed all schools nationwide by by November 3rd. Georgia’s “partial lockdown” began on November 28th.
Now, on to my predictions from April 24th, 2020:
Given that Georgia is loosening restrictions starting next week, and that we’re expecting at least some kind of Easter surge, I predict 307 won’t be the peak number of infections (confidence: 90%).
Obviously this turned out to be correct. The Easter surge materialized and about a week after this prediction, on May 2nd, we hit our spring peak with 365 active cases.
1. a. A global coronavirus peak (in active cases) will not arrive before May 1st – 99.99%
b. The Georgian government will maintain social distancing policies, including school closures and the recommendation to work from home, until May 1st – 99.9%
c. School closures will continue until June 1st – 99.9%
d. Schools will remain closed through the end of the school year – 99.8%
These were easy predictions to make at this point since the government had already basically said as much, but it’s important to note I was very confident they wouldn’t reverse course during May or June. Perhaps too confident – in retrospect, I’ve seen the government become a bit wishy-washy in terms of living up to their commitments, and this year we’ve seen them promise to open schools in Tbilisi on March 1st and then move that timeline back to February 15th. Although these predictions came true, in the future I’ll remember to add extra uncertainty for anything involving any government policy.
2. a. A global coronavirus peak will arrive between May 1st and June 1st – 5%
b. A global coronavirus peak will arrive between June 1st and July 1st – 10%
c. Global coronavirus active infections will not have any peak before July 1st – 85%
I clearly got this one right – technically the official active case count dipped on May 31st and then went back to the rising trend, but I’m going to call that a blip in the data rather than a true peak; looking at the shape of this graph clearly shows no peak before July 1st.
3. If there is a global coronavirus peak by August 31st, then:
a. Coronavirus will have at least one more peak between September 1st and November 30th – 90%
b. Starting in September – 50%
c. Starting in October – 30%
d. Starting in November – 10%
The “If” statement never happened, so I’d have to render this one moot. However, in countries where there was a spring/summer peak – like Georgia – there was indeed also a fall peak. This is my attempt to predict the timing of the second wave, and I’ll note, again, that I was just so fundamentally wrong about the scale of the second wave that my whole predictive framework here is undermined. Because what in fact happened in Georgia is that the second wave began in July and didn’t peak until December 12th. So there was no “peak” between September 1st and November 30th – but this was because we were on the upward slope for that whole three-month period. But although the second wave both started earlier and peaked later than I would have predicted, I take some solace in having predicted that it would exist and that it would start earlier rather than later. I think I could do better by trying to avoid getting trapped in the “seasonal” mindset – I was taking the idea of predicting a “Fall Wave” too seriously, when in fact the world doesn’t operate in discrete three-month-long buckets. If I were making this prediction today, I might have said something like: “countries with a clear ‘first wave’ will also have a clear ‘second wave’ after lifting restrictions”. But then, that seems tautological – of course if countries lift restrictions they’ll get another wave! Anyone could predict that! On the other hand, this is clearly not obvious to everyone, as we can see from the massive ongoing fights over effectiveness of lockdowns, mask mandates, and other public policy interventions. And maybe I’d have to separate out seasonality predictions from lockdown effect predictions.
Finally, I’d learn from this to have some degree of confidence in seasonality predictions. That’s why I found The Atlantic’s timeline for return to normal quite convincing. Regardless of the exact details, it does seem credible to imagine that there will be some kind of dip this upcoming summer and some kind of uptick next fall/winter – given what we’ve been able to predict and observe so far.
4. z. Schools in Georgia will operate through distance learning from the beginning of the next school year – 45%
a. If schools open, they will switch back to distance learning by October 1st – 55%
b. by November 1st – 75%
c. by December 1st – 85%
Again, reality thwarted my attempt to make precise predictions. What actually happened was that there was a plan to open schools in person, and some schools did open in person, but other schools postponed their in-person opening to October 1st. So technically, *some* schools did operate through distance learning *at* the beginning of this school year. But generally, I’d grade 4.z. as false – we did have some in-person schooling at the beginning of this school year. As mentioned above, the government switched many schools to distance learning in October, and switched all schools to distance learning nationwide by November 3rd.
So, again, the details are a bit hard to reconcile with a very messy reality, but the gist – that we wouldn’t have a normal school year, and that even if we did manage to open we’d almost certainly have to close down again – was correct. Again, it was considered radical and very much outside the Overton Window, at least among my teacher friends, to predict last April that we wouldn’t have a normal school year this year. People really thought the pandemic would go away, rather than have a fall wave. But I don’t think I had any particularly great foresight here – I think starting with the Spanish flu as a baseline (or prior, as they say) and then adjusting based on new studies about coronavirus in particular led to this conclusion being almost inescapable.
5. a. Georgian borders are still closed by July 1st – 90%
b. by October 1st – 70%
c. by January 1st – 55%
All borders were closed by July 1st, except to certain international shipping workers and for limited repatriation of Georgian citizens. Shortly thereafter, flights to a select few countries began. So I’m calling outcome a. true, and b and c false. It’s important that I was wrong about b. because I still think that opening borders was a very surprising move on the part of the Georgian government. In fact I think if they’d managed to actually keep borders closed for real, and make sure truck drivers didn’t have any contact with anyone outside shipping routes, we might have been able to achieve elimination here and dodge the second wave.
I keep hearing people say that in the modern world it’s impossible to close borders unless you are an island country like New Zealand. I don’t know why this should be true, though. Is it people sneaking across land borders? Are there international shipping treaties that governments cannot unilaterally interfere with? I admit this is something I just don’t know very much about. Still, I feel like better border enforcement, everywhere around the world, could have done a lot to slow the pandemic, and yet only a tiny few countries really tried to do it in a serious way.
Anyway, opening the borders was really the beginning of a set of policies which surprised me, culminating in the announcement that lockdowns would absolutely not occur until after hospitals had overflowed – which is sort of a surprising thing for a government official to announce, if you ask me. I think this last prediction, where I thought they’d keep borders closed but they didn’t, marks the point where my vision of a Georgia that fought the pandemic effectively diverged from the reality of a Georgia that gave up fighting and decided to invite a massive second wave with open arms. I think this accounts for the second wave being so much bigger and worse than I thought it would be that it lasted for five months, thus moving the start and end of the wave outside my three-month autumn prediction window.
Overall, I think I did quite well at predicting that we’d still be in pandemic mode going into fall 2020 and winter 2021, but I was wrong in thinking that the first wave would be worse than the second; rather it was the second wave which was much worse than the first, both in Georgia and globally. I did okay at predicting government response, but I think that’s fine because politics is much more difficult to understand and predict than pandemics. I didn’t do a very good job of laying out predictions in a way that would produce clear and transparent results and so I ended up having to evaluate most of my predictions based on some interpretation of how they mapped to reality rather than based on very clear and unambiguous benchmarks, and my lesson there is that next time I should make predictions, wait a few days until I forget what I mean by them, and then take a second look to see if it was still immediately clear what each one meant, what indicators and measures I would look at to evaluate them, and how I would interpret different outcomes, including unexpected or ambiguous ones.
I hope to have a new set of medium to long-term predictions up within the next month or so… sorry I can’t be more specific, but apparently my blogging habits are outside the realm of even my predictive abilities.
Georgia On My Mind 