Exploiting Change

By Richard Martin

There’s a lot of screeching and prancing about going on right now about COVID-19 numbers, mainly in the US, of course, but also in Canada and elsewhere.

I’m not saying any of the numbers are necessarily wrong. On the other hand, we don’t necessarily have the information to make a judgment about their validity. Here are some questions I’ve been asking myself, to which I don’t have the answers. If anyone does, I’d appreciate any references.

Question 1. Are they being counted in the same way in different jurisdictions? There are over 30 countries in Europe, 50 states in the US, 10 provinces and 3 territories in Canada. Are the accounting methods sufficiently similar to make useful and valid comparisons?

Question 2. What is the error margin? We’re familiar with this concept from announcements of public survey results. For instance, Gallup will say that their survey results for a specific survey are within a certain margin of error 19 times out of 20. So, they’re saying that there is a possibility of inaccuracy and that close results shouldn’t be taken as gospel truth. This kind of transparency about results is par for the course in quantitative science. Astronomers report their results with a margin of error. So do particle physicists, biologists, and chemists. Even psychologists and cognitive scientists give a margin of error. This is because error is inherent in any measurement process. It can be minimized and estimated, but never eliminated. This is the a fortiori the case for COVID-19 accounting.

Question 3. What are the relevant rates? Even assuming we had reasonably accurate and comparable numbers of infections, hospitalizations, and deaths, and they were comparable between jurisdictions, that doesn’t necessarily mean the numbers are inherently meaningful, especially as a guide to action or policy. The best example of this is that we lack the denominator in many cases to establish reasonably accurate rates: of contagion, of infection, of morbidity, of mortality.

Question 4. How “granular” are our numbers? In other words, who is spreading, who is getting infected, who is getting sufficiently ill, and who is dying? In Canada, 80 % of deaths are in nursing homes, and these mostly in Ontario and Quebec, by far the two most populous provinces. Anecdotal evidence tends to suggest that this percentage is accurate (but I could be wrong of course), but is it reasonable to compare this figure with those in other countries? Maybe, maybe not, but this is something for public health scientists and epidemiologists to evaluate. It’s probably too early to get definite answers, while generating interesting paths and hypotheses to investigate. Time will tell.

Question 5. How valid are international and interjurisdictional comparisons? I was reading a FB post this morning showing that the death rate in Canada is much lower recently than in Florida, Texas, and other US states. The graph also showed that Canada’s population is about 50% greater than either of those two states. However, Texas and Florida are also a lot more densely populated than Canada. You’re comparing two completely different geographic and demographic profiles. That’s not even taking into consideration cultural, political, economic, and social differences. We can’t simply assume that different countries and regions have similar bases for comparison, no matter how similar they may appear on the surface. A fortiori for intercontinental comparisons. They’re not devoid of interest. It’s just that they usually don’t mean what rhetorical flourishes claim for them.

These are just some questions I’ve been asking myself. I suspect others have too, although they haven’t articulated them as such. Feel free to add your own, but please refrain from conspiracy theories. If we’re going to have rational policy and action, these must be based on reason and prudence, not gut feel and wild swings of passion and emotion.

© 2020 Richard Martin