Sampling Bias in COVID 19

Published in El Pais, 26 January 2021.

Countries around the world are trying various ways to “flatten the curve” with COVID 19 and it has certainly been an educational experience as we know a lot more now than we did at the beginning of the pandemic. We still have a lot to learn however and some governments have different theories on how to continue the fight from travel restrictions to total lockdowns. There are different ways to collect data to help them come to this decision. One way is to look at measures of other countries to measure success rates, for example mask wearing was more common in Asian countries which saw a lower transmission rate than countries that did not have a policy on masks. This led to mandatory mask laws around the world. Others are looking more locally and comparing the spread during lockdowns in their region to when they were more open. Whatever the method we must make sure the data is collected properly as a sampling bias can skew politicians to making poor decisions which can have disastrous results.

For the past month in Medellin Colombia for example, the city has been under lockdown for holidays and weekends, no leaving your house (not even for groceries) unless for medical emergencies. One cannot even walk your dog. This weekend however there will be no lockdown, why? Because they want to compare the transmission rate between weekends under lockdown and weekends with basic precautions. At first look it does not seem like a bad idea, lockdowns have severe economic consequences and if they are not effective than they should not happen. The main problem is such a test has severe sampling bias and can cause a distorted conclusion.

Sampling bias is when data collected has a higher probability of leaning one way than if it is collected properly. With Medellin’s experiment, many people have been locked in their home every weekend for over a month, they have not seen their friends or family, they cannot go out to dinner or drinks. So, with the news that this weekend will be open we will likely see a purge of social distance breaking. Many will go to parties or massive gatherings whereas if weekends were open all this time those same people would likely be contempt with doing something low key like go to a dinner with a couple of people are hang out safely in a park. In the end the infection rate will skyrocket and those who are pro lockdown will point to the biased data as justification.

Those with power need to take better care when collecting data to see what measures are successful and what are not, maybe the city of Medellin needs a lockdown, maybe it does not, one thing we know for sure is whatever data collected from this upcoming weekend will be useless.