EVIDENCE & HOW TO INTERPRET IT

The fact that an opinion has been widely held is no evidence whatever that it is not utterly absurd. Bertrand Russell

Why evidence?

For the next few months I will be discussing the effects of exercise on a number of conditions and diseases – mainly those which become more common as we age. These are the conditions which undermine our health and well-being in middle age and later and which may be prevented or reduced in severity by regular exercise.

I need to convince you that what I tell you is as accurate as possible so I will be including the evidence which supports what I say. Today’s homily is a brief description of how that evidence is collected and evaluated. In the case of exercise/physical fitness and health we will be talking about cause and effect. Here are some examples of evidence types.

1. Epidemiology

“Epidemiology is the study and analysis of the patterns, causes, and effects of health and disease conditions in defined populations. It is the cornerstone of public health, and shapes policy decisions and evidence-based practice by identifying risk factors for disease and targets for preventive healthcare”. OK, that’s pretty indigestible. In simple terms it is the observation of associations between certain behaviours, environmental conditions and physical states with disease. When Professors Doll and Peto from Oxford identified cigarettes as the main cause of lung cancer they used epidemiological evidence – the observation that people who smoked cigarettes had a far higher incidence of lung cancer than those who did not.

A lot of the evidence related to the effects of exercise are epidemiologically based, for instance the finding that people who take a lot of exercise are less likely to develop Type 2 diabetes.

Both of these examples only demonstrate associations, not proof of cause. Association is not the same as causation – though it may be! Take this example: during the period in the 20^th century when the incidence of coronary disease was increasing rapidly, so was the use of the radio – but few people believe that coronary disease is caused by radio waves.

In the case of smoking and lung cancer there is no other reasonable explanation and there are good biological reasons to believe that smoking might cause lung disease. In the case of exercise and diabetes we cannot be quite so sure. There may be “confounders” – that is to say, other factors with which both exercise and diabetes are associated – say sugar consumption. If exercisers consume less sugar than non-exercisers, that might be the explanation for the association. There are many differences between exercisers and non-exercisers and it would be impossible to find groups of each whose other characteristics were identical, so many of the associations which I will describe below are just that – with a very strong presumption of a causal link.

 2. Clinical trials

Clinical trials provide stronger evidence than epidemiology. The purest form of trial is the randomised, double blind, controlled trial (RCT). This is best understood in the example of a drug trial. A group of individuals with a particular condition is selected to clarify the effect of the drug and they are divided randomly into two groups. One group receives the treatment and the other (the control group) receives a placebo – that is an inert pill looking the same as the real treatment. Neither the person giving the treatment and assessing its effect nor the patient receiving it knows who is getting the real treatment and who is getting the placebo. At the end of a predefined period the code is broken and the difference between the two groups analysed. If a trial of a cancer treatment is being tested, this might be the cure rate and if the cure rate is significantly higher in the treatment group than in the controls it can be inferred that the drug is effective.

There are lots of possible pitfalls even in this very straightforward scenario. The patients might not have complied properly with the treatment – this is most likely for the treatment group if the drug has unpleasant side effects. The randomisation process might not be perfect. Despite the randomisation there may be unexpected differences between the two groups. The statistical analysis has its own problems. In clinical trials statistical significance is reached if the chance of the difference found between the two groups is less than one in twenty (in statistical shorthand,  p=<0.05). With a clearly effective treatment this can be derived from a trial with small numbers of patients. The less the effect, the more patients will be needed in the trial to show it. The more patients needed in the trial to show an effective outcome, the less effective the drug must be. If the number needed is very great, a statistically significant effect might be clinically insignificant – not worth the candle. The counter side to this is that small trials are more likely to produce erroneous results – the bigger the trial the more reliable a test it is.

Even apparently well conducted RCTs can give results with problems of interpretation. When such trials are carried out by pharmaceutical firms on their own products they are far more likely to have a positive outcome than independently funded and conducted trials.

If you think that all that is bad enough, just consider the difficulties of carrying out randomised, controlled trials of exercise! For a start they cannot be double blind – treatment group members know they are in the treatment group – and it is quite hard to prevent the observers knowing. Compliance can be a nightmare and there may be cross contamination – that is members of the control group may start exercising, even if they have been asked not to. When I carried out a randomised, controlled, but not blinded, trial of exercise for patients following heart attacks, one of the non-exercise controls bought himself an exercise bike on his way home from his initial exercise test!

3. Meta-analysis

There is another level of evidence which in some instances can provide the strongest evidence – the systematic review and meta-analysis. These are most useful when the numbers required to show an effect in an RCT is larger than can be reasonably recruited in a single trial. It is also an essential technique to unravel the facts when different studies of the same topic come up with different answers.

A systematic review answers a defined research question by collecting and summarising all empirical evidence that fits pre-specified eligibility criteria.

A meta-analysis is the use of statistical methods to summarise the results of these studies In other words, a meta-analysis is a summation of the results of all the trials carried out using the particular intervention under investigation. The total number of subjects is much larger than is seen in single trials and the results should therefore be that much more convincing. Again there are problems. Different trials are different in many aspects and combining them sometimes involves mixing oranges and pears. The choice of trials for inclusion is crucial if a particular question is to be answered – there must be as many similarities as possible. And there is the ever present bias of the tendency to publish trials with a positive outcome but not those with a negative outcome.

Next week I will talk a bit more about evidence – then we will get to the meat of my subject which is the effect of exercise and physical fitness on preventing and treating disease.

PS

 Another word about Corona virus and infections in general. Professor Richard Moxon, emeritus Professor of paediatrics at Oxford University, has started a Blog – https://moxforum.co.uk – which I strongly recommend. Richard is a pre-eminent expert on infections, immunity and immunisation. His blog explains the science of infection, the human body’s reactions and how immunity acts to protect us. This week’s blog explains the “R number”, more correctly called R₀  – the measure of the number of people to whom, on average, one infected person spreads a germ. The politicians bang on about taking decisions based on the “science” – I suspect that this means the evidence which supports their policies. Read Prof Moxon’s blog for clarification.