Nuijten et al (1) developed a computer program, statcheck (2), that finds and checks p-values reported in APA style in published papers. Over 250 000 p-values presented in 30 717 psychological papers published in the Journal of Applied Psychology, Journal of Consulting and Clinical Psychology, Developmental Psychology, Journal of Experimental Psychology, Journal of Personality and Social Psychology, Psychological Science, Frontiers in Psychology, and the Public Library of Science.

Of the reviewed papers 16,700 included tests of statistical significance in a format that it could check. Half of these reported at least one erroneous p-value. One in eight contained at least one erroneous p-value, which may have affected the conclusion of the paper. The prevalence of erroneous p-values was stable over time or declined, but was higher in p-values reported as significant than in p-values reported as non-significant.

The authors suggest that data sharing, letting co-authors check results, and checking manuscripts using statcheck could reduce the number of such reporting errors.

Reference

  1. Nuijten MB, Hartgerink CHJ, van Assen MALM, Epskamp S, Wicherts JM. The prevalence of statistical reporting errors in psychology (1985–2013). Behav Res 2015 DOI: 10.3758/s13428-015-0664-2.
  2. Epskamp, S. & Nuijten, M. B.  (2016). statcheck: Extract statistics from articles and recompute p values. Retrieved from http://CRAN.R-project.org/package=statcheck. (R package version 1.2.2).