The design of randomised trials often include stratified randomisation or minimisation to achieve balance with regard to important prognostic factors. One consequence of such balancing is that the treatment groups become correlated, which is at variance with the underlying assumption of statistically independent observations. In order to ensure correct p-values and confidence intervals it is important to adjust for the balancing factors in the statistical analysis. Otherwise, p-values will be too large and confidence intervals too wide.
Kahan and Morris (1) performed a review of the British Medical Journal, Journal of the American Medical Association, Lancet, and the New England Journal of Medicine with respect to randomised trials published in 2010. The purpose was to see if the method of randomisation was adequately reported, how often balancing was used and whether the balancing factors were adjusted for in the statistical analysis.
The results were that balanced randomisation was common. While the randomisation method was unclear in 37% of the 258 published reports, 63% included balancing on at least one factor. A majority of the trials with balanced randomisation were inadequately analysed. In only 26% of them included the statistical analysis adjustment for all balancing factors. The trials in which the statistical analysis did not include adjustment for balancing factors were less likely to show a statistically significant result, 57% versus 78%.
Kahan and Morris conclude that balancing is common but often poorly understood.
- Kahan BC, Morris TP. Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. BMJ 2012;345:e5840.