Evidence Part 2

A big apology

The last blog which you will have received was “Evidence Part 1” on 1st July – with a promise of “Part 2” to follow on 15th July. It never happened! We had a technical problem and the fortnightly blog has failed since then.

The good news is that the problem has been resolved and hereafter your two weekly flow of information is henceforth reinstated! This midweek post will test the system and normal service will be resumed in ten days.

Evidence Part 2

Medicine is very far from being a perfect science.  It is said that half of what doctors believe to be true today will, in time, be shown to be wrong. The problem is that no one knows which half.  Interpreting evidence is a huge problem, but it is at the core of ‘evidence-based medicine’.

Trials of Exercise

Using clinical trials to evaluate the effects of exercise presents one particular problem. The trial can be randomised but it is impossible to make it double blind –  treatment group members know they are in the treatment group and it is quite hard to prevent the observers from knowing too. Compliance can be a nightmare and there may be cross-contamination. This means that members of the control group may start exercising, even if they have been asked not to. The Alton randomised trial of exercise after a heart attack had this experienced. One of the non-exercise controls bought himself a static cycle on his way home from his initial exercise test!

Epidemiology of Exercise

Most epidemiological studies and many trials of exercise use the subject’s own estimate as the measure of their exercise volume and intensity. Unfortunately this is highly unreliable, since most people greatly overestimate how much exercise they take. An alternative measure is the physical fitness of the person involved. This is independent of the exaggerations told by the subject, both to the researcher and to themselves, so should be a much more reliable measure. In the majority of cases, it most certainly is. However, fitness level is not wholly determined by the amount of exercise undertaken. There is also a genetic component, and this reduces the accuracy of fitness level as a reflection of exercise-taking.

Systematic Review & Meta-analysis

There is one more level of evidence, which in some instances can provide the strongest information. This is the systematic review and meta-analysis. These are most useful when the numbers required to show an effect from a randomised controlled trial (RCT) are larger than can be reasonably recruited in a single trial. It is also an essential technique for unravelling the facts when different studies of the same topic come up with different answers. A systematic review answers a defined research question by collecting and summarising all the evidence collected through observation that fits pre-specified eligibility criteria. A meta-analysis uses statistical methods to summarise the results of these studies. In other words, a meta-analysis is a summation of the results of all the trials carried out using the particular intervention under investigation. The total number of subjects is much larger than in single trials and the results should therefore be that much more convincing. Again, there are problems. Different trials are different in many aspects and combining them sometimes involves mixing ‘oranges and pears’. The choice of trials for inclusion is crucial if a particular question is to be answered – there must be as many similarities as possible. Unfortunately, there is the ever-present bias of the tendency to publish trials with a positive outcome but not those with a negative outcome.

Presentation of evidence

It is very easy to be misled by the way in which evidence is presented. Those with a predetermined view, including those funding the study, tend to present their results in the way that is most likely to persuade the reader that the treatment is effective. Also, negative studies are much less likely to be published than those with positive findings. This is particularly likely to bias the published results of drug trials.  Fortunately, exercise studies are seldom sponsored by commercial organisations, but nevertheless the authors of such studies do sometimes have an interest in the outcomes. It is instructive to be aware of this possible bias.

One further point on the presentation of evidence. The benefit of a particular intervention is usually expressed in terms of change in risk and may be presented as a change in relative risk or a change in absolute risk. A change in relative risk usually makes the treatment seem much more attractive than a change in absolute risk.

Absolute risk of a disease is the risk of developing the disease over a period of time and may be expressed as a fraction. An example would be say, one in 20. Expressed as a percentage, this case would be 5 per cent, or as a decimal a 0.05 risk.
Relative risk is used to compare the risk in two different groups of people. For example, the groups could be smokers and non-smokers. All sorts of groups are compared to others in medical research to see whether belonging to a particular group increases or decreases the risk of developing certain diseases. For instance, research has shown that smokers have a higher risk of developing heart disease compared to (relative to) non-smokers. As an example the absolute risk of a group of non-smokers developing heart disease might be 20 in 100 (20 percent). If the risk for smokers is 30 in 100 (30 per cent), their relative increase in risk is 50 percent compared with the non-smokers. It sounds a lot but the absolute difference is just ten in 100.

When comparing the results of treatments, the choice of whether the outcomes are expressed as absolute or relative improvements can have a major effect on how effective the treatment appears. The less common the condition, the truer this is. An appropriate example is the risk of a heart attack and how this can be reduced by taking a particular drug. The risk of a heart attack over the next 10 years in a group of women aged between 40 and 50 may be, say, one in 100, ie 1 per cent. If taking the drug in question reduces the risk to 1 in 200 (0.5 per cent), it may be reported that the risk of a heart attack, the relative risk, was halved in this group. However, the absolute risk is a reduction from two deaths to one death for every 200 women – an absolute reduction of one in 200. In other words, 200 women would have to take the drug for 10 years to prevent one new heart attack. This clearly seems rather less impressive than halving the risk. Sometimes the effectiveness of a treatment is then expressed as ‘number needed to treat’. This is the number of people who need to take the treatment for just one person to benefit – in this case 200.

Mortality

Mortality rates are often used as outcome measures to compare the efficacy of different drugs and other treatments. The ultimate mortality for any treatment regime is 100 per cent because everyone will die in the end.  This means that ‘mortality’ when used in this context has to be qualified. There are two ways of doing this:

Mortality is expressed as the death rate over the period of study and compared between the groups being studied.
The death rate of the group being studied is compared with the known death rate of the whole population of the same age and gender. This is usually expressed as deaths per 1,000 persons per year.

Interpreting the evidence

In conclusion, facts unsupported by evidence should be questioned. Even when the evidence seems to support the facts, individuals must be alert to the possibilities of error. Very little evidence is totally unarguable. However, it is a great deal better than any other way of reaching the truth. This applies equally to exercise as to any other treatment.

 

More about this Blog

Over the past few years this Blog has worked its way through various aspects of exercise – physical, social and disease related. To read more on any aspect of the effects of exercise use the “Categories” index on the right side of this page. From now on I am going to report on new evidence and recent advances in the field of exercise and physical activity, so the choice of subject will not be in any logical order.

Waiting to hear from you!

When the Blog was transferred  Alton Cardiac Rehab it lost the easy link for responding with queries, disagreements etc. This has now been restored – see “Leave a reply” below. Please use this and let me know what you think or want to know more about.