Metrics and CoVID

[rslade]

Another security lesson from CoVID is in regard to metrics. Those who have tried to create security metrics will know, all too well, how difficult it is to choose those that are actually useful, rather than just being collections of numbers. (Brotby and Hinson's PRAGMATIC acronym is very helpful in providing guidance.)

Among the various statistics that CoVID has generated, such as case rates, new cases, doubling time of cases, hospitalization rates, et cetera, one single number that has been consistently useful is the positivity rate. This is the number of cases confirmed, divided by the total tests done. While there are a number of additional factors to consider, it seems to be generally felt that a positivity rate of about two percent is probably reasonable. Any lower, and it is likely that you are testing too many people too indiscriminately, and wasting money and resources. Any higher, and it is likely that you aren't testing enough, and that cases are, or shortly will be, increasing. Positivity has proven itself "Relevant" from the PRAGMATIC list.

Recently, in British Columbia, we have seen how difficult it may be to keep such metrics "Meaningful" and "Accurate."

BC, often known as "Hollywood North," is home to a thriving and active film industry. If you are a fan of Hallmark romances and mysteries, and other such "made for TV" fare, chances are very good that they were shot here. (When Gloria and I watch them, it is often as much to play "spot the location" as to follow the plots.) This is especially true now during the pandemic, when BC has been a relatively safe place to do film shoots. There are, of course, a number of restrictions to keep filmmaking safe, some imposed by local health authorities, and some required by unions, particularly from the US and places where the case rates have been much higher, demanding fairly stringent precautions. CoVID testing, in particular, is done regularly, and often very frequently, regardless of how many cases turn up.

Testing for the movie industry is done at private labs, so as not to affect lab capacity for the public health system. However, even so, the testing is "reportable," and thus the numbers make their way into public figures. The demands of the movie industry are such that 4-5,000 tests may be done daily, at a time when the public testing capacity is about 16,000 tests per day. Since the movie industry definitely "overtests," the movie numbers artificially depress the overall positivity rate. Our positivity rate in BC may actually be twice what the published figures show.