What specific questions did your study seek to answer? 

Our study had two primary objectives. The first was to evaluate whether the authors have interpreted the clinical relevance of the
effect of physiotherapy following their a priori definition. 

The second was to reinterpret the clinical relevance of the between-group differences based on the available estimates of what difference between groups can be considered important (ie, smallest worthwhile effect). 

As a secondary objective, we also aimed to evaluate, for descriptive purposes, whether the studies are adequately powered or underpowered, considering the published smallest worthwhile effect values and a power threshold of 80 per cent.

Which trials did you use to examine those questions? 

We included randomised controlled trials on the effects of physiotherapy interventions compared with no intervention in
people with chronic low back pain. 

This choice was made because there are two studies that estimated what differences are important for pain intensity, physical functioning and time to recovery for physiotherapy interventions compared with no intervention in this population. 

These studies used the benefit–harm trade-off method to calculate the smallest worthwhile effect values. 

The studies consistently found that patients need to experience at least a 20 per cent additional improvement in pain and physical functioning and to speed up their recovery by ten days to consider that the effect of physiotherapy is worthwhile.

What thresholds were used in the included studies to examine and interpret clinical relevance?
The most used construct was the minimal important difference (seven studies). 

Cohen’s threshold and difference in mean change between groups were used in three studies, while two studies used within-group mean change. 

Tiziano Innocenti.

Only 39 per cent of the studies discussed the clinical relevance of the results; in some cases, they used two different constructs for sample size calculation and discussed the clinical relevance of the results. 

An important finding was that many studies used within-group estimates for interpreting the relevance of between-group differences.

What about the statistical power of the studies?
We calculated the power reached a posteriori for each included study considering the alpha error used in the study, sample size and a between-group difference of 20 per cent (ie, the smallest worthwhile effect values already published) on the primary outcome.

We found that the study power ranged from nine per cent to 98 per cent (median = 44; interquartile range = 53). Only four studies were well-powered (ie, power > 80 per cent).

Where does research in this field need to look next?
We encourage researchers to carefully plan randomised controlled trials to detect the smallest worthwhile effect of intervention, informing readers about whether the effect of an intervention is large enough to be worthwhile. 

This should be done using approaches such as benefit–harm trade-off methods (eg, the smallest worthwhile effect). 

These methods are more appropriate than anchor-based methods (eg, minimal important difference), which cannot tell us whether patients feel that the effect of an intervention is large enough to make the costs, risks and inconveniences associated with intervention worthwhile. 

It is important to realise that statements on the importance or relevance of between-group differences are value judgements that cannot be answered by statistical significance testing.

>> Tiziano Innocenti is a PhD candidate at the Vrije Universiteit Amsterdam and the coordinator of the Research Methodology Unit of the GIMBE Foundation, Italy. Tiziano’s research interests relate to the field of meta-research.