16th October 2021
16th Oct 2021
5 Minutes
In this blog, I am collating some of the previous thoughts I shared on my Twitter page. This is far from a comprehensive scientific article, but I think it is worth sharing as there has been much talk about ‘controls’ in the context of Long Covid studies in the public domain. I may expand on or update the ideas below at a later date, but here they are for now, largely as they previously appeared on my Twitter page with minor edits and additions for the purpose of clarity (for updates refer back to original site)
We have been hearing a lot recently about the importance of comparing to “controls” in studies looking at Long Covid as the ultimate solution to answering prevalence questions, i.e., how common is Long Covid. I have a few thoughts about this that I touch on below. The main idea behind comparison to controls is the assumption that there is a baseline level of symptoms in the population that some people experience if (a) they never had covid or (b) if they had covid, recovered, and then experienced symptoms caused by something else. So in order to measure the ‘true’ prevalence of long covid, the proportion of those with symptoms is compared between the post-covid group and the non-covid group. Obviously, that still leaves (b) but let’s talk about (a) here. I will try to cover issues related (b) in another post.
For example, if we have a cohort of people who tested positive for covid and the proportion of those with fatigue at 12 weeks from onset of infection is 20%, the control would be a cohort with no covid in which proportion with fatigue let’s say is 5%. We look at the difference and conclude if the prevalence is significantly higher in the post-covid group compared to the control group (15%).
There are many issues to consider for this to be a meaningful comparison:
How we ascertain covid: we know that test accuracy is not 100% (both for the presence of the virus or the presence of immune markers against it- I will come to that later in this blog) so controls could have had covid (denominator issues)
Some symptoms are generally more common and less distinct than others, so it matters which ones we assess and how many (numerator issues)
It matters if the symptom is persisting e.g., distinct chest tightness feeling from the start of covid and continues the same for months, compared to someone who reports chest pain on the day of survey. Without ascertaining the pattern & intensity over time, it is pointless. Therefore studies that follow people over time (longitudinal) are preferable to those which assess symptoms at one point in time only (cross-sectional)
Functional ability may be a better measure to compare i.e., how do the symptoms/illness affect daily activities. In this case, better to compare with before the infection rather than with other people. So ideally the study cohort needs to collect that information and follow up prospectively for incidence of infection.
Now considering the above, take the hypothetical example that chest pain happens in 5% in both groups, the post covid, i.e., case group, and the ‘non-covid’ groups. i.e., control group. We still cannot conclude that none of this 5% is caused by covid. Therefore, it would not be legitimate to say Long Covid does not exist based on that.
It is vital to talk to people with lived experience of Long Covid when designing such studies. The type of questions asked and how they are phrased matter a lot. There is a clear link between the chronic and acute symptoms. Some continue as experienced in the acute phase, so it is unreasonable to assume they are due to another random cause weeks later.
I want to talk a bit more about (1). There is an issue with research classifying those who tested negative or not tested for SARSCoV2 as controls to compare against illness experienced by those who tested positive. Let me explain.
To compare if infected people get long term illness or symptoms or functional disability more than people who were not infected with SARSCoV2, some studies compare ‘cases’ to ‘controls’ as a way of indicating causality. They simply define ‘case’ by test result. There are limitations with this approach.
In the case of assessing Long Covid, one specific limitation is we do not have an accurate way to establish past infection. So researchers use PCR or antibody test results but these are not accurate enough to establish case status. PCR misses a proportion of true infections and the timing of testing in relation to stage of the infection is crucial. We also know that antibodies wane and may be undetectable in those with true past infection. There is also evidence that having Long Covid is in itself associated with weak antibody response to infection. This potentially means more misclassification of controls.
So including people with negative tests in the control arm of the study could dilute the difference between groups because people who tested negative could have actually had covid. Even more problematic is including non-tested people as controls and assuming they have not had covid, particularly in countries where there was widespread community infection.
Do you see the problem here? For example, a parent reporting that they have been invited to a Long Covid study in kids and they think their child has Long Covid but they never had a positive test due to lack of community testing early on in the pandemic. The child will likely be classified as a control in the study if they are only asked in the study questions about past test result and symptoms. That means the participant would count under non-infected having symptoms potentially diluting the difference between the ‘case’ and ‘control’ groups.
Controls are seldom ‘pure’. They need proper and strict definitions to minimise ‘contamination’ with cases. Selection of controls is a challenge in terms of how they represent the ‘background’ prevalence. If assessing more than one thing (condition), it can become messy.
Another point to make is that counting cases of a certain condition as a routine part of a local, regional or national public health surveillance system does not classically determine prevalence of conditions by having control groups. Comparison to individual controls is usually made in research studies rather than surveillance systems.
Usually, prevalence estimates are compared to population-level averages. So, as a hypothetical example, if the prevalence of back pain in my town is 10% it would be compared to the prevalence of back pain across the whole country, say for example 3%. I would then want to know more about why, in my town, there is more back pain than the national average. We call this ‘hypothesis-generating’ descriptive statistics, which often lead to generating hypotheses about causation (in this case, potential causes of the higher prevalence of back pain in my town). These can further be tested in research studies.
If I start looking into causes in this hypothetical example, I could start to do that by asking individuals in my town to take part in my research. I can ask those with and without back pain (cases and controls) to take part and ask them questions about potential causes.
To summarise, these are some considerations when talking about control selection in Long Covid prevalence estimation (some I have not touched on in detail but may expand later):
Population level prevalence as control vs individual controls
Longitudinal assessment vs cross-sectional
Denominator issues (all tested positive, random or symptomatic-based, only symptomatic denominator)
Numerator issues (definition of Long Covid, method of assessment, subjective (symptoms, functional ability) vs objectives (signs and investigations)
Statistical modelling strategies (survival vs not)
I hope this blog begins to illustrate why having ‘controls’ on the study tin does not necessarily mean much without understanding the details of the comparisons. It is hard to get a perfect study design, but it is important to think carefully about how we ascertain case and control status when comparing and interpreting data for any condition. Long Covid is no exception, though it is one of the more complicated examples given the current scientific uncertainty around it.
LongCovidKids.org is a patient-led advocacy and support organisation led by Sammie Mcfarland for families of children with Long Covid. Our story started with a short film on the long-lasting symptoms of Covid in children.
We are supporting research with The Long Covid Kids Study with PeopleWith
Each Sunday we send out a newsletter providing a weekly round-up of statistics, the latest reports in the media, current research, and LCK news. Read previous issues and SIGN UP
If your child experiences any symptoms that indicate they are unwell, it COULD be COVID-19, and you should get them a test to help identify if it is a current Covid infection. Please see our Frequently Asked Questions page for more information.