Statistical Mistakes

The statistical reviewer's handbook

Observational studies

Confounding adjustment (ANCOVA)

As for the confounding problem, please note that assumed cause-effect relations form the basis on which confounding adjustments are made. While including a confounder in the model reduces bias, the inclusion of a mediator or collider can induce bias, see Shrier I, Platt RW. Reducing bias through directed acyclic graphs. BMC Med Res Methodol 2008;8:70. Please describe how variables are measured and provide a rationale for a priori selection of potential confounders, ideally in the form of a causal graph. Even with performed adjustment, confounding can persist, an issue that should be addressed in the discussion. Sensitivity analyses can be useful for assessing residual confounding resulting from unmeasured and imperfectly measured variables.

Confounding adjustment (PS)

The propensity scores were estimated using a statistical model including sex, age, weight, height, EQ-5D. Please describe how these variables are measured and provide a rationale for a priori selection of potential confounders, ideally in the form of a causal graph. Please clarify also if all of these covariates have been measured prior to treatment, or if the used propensity scores could induce adjustment bias, see e.g. Sauer B, Brookhart MA, Roy JA, VanderWeele TJ. Developing a Protocol for Observational Comparative Effectiveness Research: A User’s Guide. Rockville (MD), Agency for Healthcare Research and Quality (US); 2013 Jan. Publication No.: 12(13)-EHC099. Chapter 7. Covariate Selection.

Table 2 fallacy

Could Tables x-z reflect cases of the so-called Table 2 fallacy? See e.g. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol 2013;177:292-298. Please describe the fitted models more clearly.

Primary endpoint

A primary endpoint is mentioned in the manuscript. This is usually something pre-defined in a randomised trial within the framework of a strategy for addressing multiplicity issues. What is the purpose of the primary endpoint in this study?

Stepwise regression

The multiple regression models are developed using stepwise regression, a statistical method that does not account for cause-effect relations, i.e. defining variables as independent, confounders, mediators and colliders. While the inclusion of a confounder in a statistical model is used for reducing bias, the inclusion of a mediator or collider increases bias. It is thus important to base the model development on what is known or assumed about cause-effect, see e.g. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol 2013;177:292-298.

 

%d bloggers like this: