User Tools

Site Tools


Study design

Case-control study

The study is described as a case-control study but includes follow of cases and controls, which indicates a cohort study. Please clarify, see Mayo NE,Goldberg MS. When is a case-control study a case-control study? J Rehabil Med 2009;41:217-222.

Description of the study design

The relation between the aim of the study and the statistical analyses is unclear. This information is, however, important for evaluating the validity of the findings. Please describe the study design more clearly and in more details. What is the analysis unit and sample size? What are the tested null hypotheses and why are they tested? What is the purpose of using statistical models?

Statistics section

The description of the statistical analysis is too brief. For example, it is unclear whether or not the assumptions underlying the methods (e.g. statistically independent observations, Gaussian distribution, homogeneity of variance, etc.) were fulfilled. The ICMJE recommendation is to “Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to judge its appropriateness for the study and to verify the reported results”.

Parameter estimation and prediction

The statistical analysis seems to be based on a conflation between parameter estimation and prediction. Both of these analysis approaches are based on statistical models but the purposes are different. A prediction model is optimized with respect to its predictive accuracy, which is measured in terms of sensitivity and specificity. A major problem in the development of prediction models is overfitting, adaption to sample-specific random variation, and the solution is validation. Parameter estimation is performed using an explanatory model, optimized with respect to the validity of the parameter estimates. The model must, therefore, be developed on the basis of known or assumed cause-effect relations. For example, confounders are included in the model to avoid confounding bias, but including a variable with another cause-effect relation (mediator or collider) induces adjustment bias. A good explanatory model does, therefore, not necessarily provide good predictions and a good prediction model does not necessarily provide unbiased parameter estimates.

Correlated observations

The statistical analysis and the presentation of the results focus on knees instead of patients. This means that bilateral (correlated) observations are included in the statistical analysis. Has this correlation been accounted for in the statistical analysis? Are the presented results reliable? See e.g. Ranstam J. Repeated measurements, bilateral observations and pseudoreplicates, why does it matter. Osteoarthritis Cartilage 2012;20:473-475.

Normal distribution

It is stated in the statistics section that “The normal distribution of data … was confirmed with the Kolmogorov-Smirnov test”. I assume that the distributional assumption was tested using the Kolmogorov-Smirnov test. The statistical power to detect a deviation from normality depends, however, on the sample size. Given the limited sample size of the study, the word “confirm” is probably an overstatement.

Bonferroni correction

The statistical analysis includes Bonferroni correction. Please describe the strategy for addressing multiplicity issues and clarify how the Bonferroni correction is performed with respect to number of tested null hypotheses. Has the strategy been pre-specified and takin into account in the sample size calculation? How does it affect the interpretation how the results from the statistical analysis (i.e. vis-à-vis unaddressed multiplicity effects)?

Please explain the Bonferroni correction with respect to why such corrections are meaningful in an observational study without pre-specification of a strategy for addressing multiplicity issues, and describe what multiplicity problems that are solved and not solved by the performed correction.

Primary outcome

Primary and secondary outcomes are presented in the manuscript. A structuring of endpoints as primary and secondary is usually a part of a strategy for addressing multiplicity issues in confirmatory randomised trials, but such a strategy is hardly relevant in this observational study. What is its purpose here?

studydesign.txt · Last modified: 2020/05/06 08:26 by ranstam