Stat 5870 section 2 - References to additional material

This page contains references that may interest you if you wish to explore an issue further. None of these are required.

Importance of random sampling and / or random assignment of treatments
- Wikipedia description of the 1936 Literary Digest poll that predicted Alf Landon would beat FDR in the presidential election.
- Re-evaluation of the Literary Digest story
- Retraction of Med diet study (republished with details of protocol deviations and a reanalysis in NEJM 378:e34(1)-e34(14), doi 10.1056/NEJMc1806491)
Interpretation of p-values and the replication "crisis" in social science
- American Statistical Association 2016 statement on p-values
- Elaboration (history/context, process, purpose) of the ASA statement. Open access article. Lots of references. Links to the papers in the Supplemental material are at the bottom of this webpage.
- Summary web article on the social science reproducibility "crisis" from Vox
- Aug 2018 Nature Human Behaviour on reproducibility article (requires on campus or VPN access to library resources)
Reporting numeric results
- Kelley, T.L. 1924. How many figures are significant? SCIENCE 60:524.
  Note: Probable error is an old name for 2/3 what is now called the standard error.
- Terman, G.L. 1978. Comments on significant figures in relation to experimental errors. AGRONOMY JOURNAL 70:519-520.
  Suggests rounding "treatments means to 10% of their standard errors".
Deleting outliers
- Emory research consortium statement is here
- Examples from two famous scientists, both discussed on web pages
  - Gregor Mendel: segregation ratios in peas. Seemingly 'too close' to 3:1 or 2:1
  - Millikan oil drop studies (electron charge): Did not report all his observations
Understanding confidence intervals
- Article in Science News Note: comes with ads. Has link to a blog post by Andrew Gelman. His work is top quality and always worth reading.
Multiple comparisons
- Example where adjusting or not adjusting leads to very different policy, Women's Health Initiative study of long duration estrogen: Discussed in Science (26 July 2002) 297:492. Two similar studies of risk and benefits of long term estrogen replacement therapy. One study is terminated early because one comparison indicates increased risk to participants. Simultaneous confidence intervals indicate no evidence of increased risk. Other study continued.
- Two papers from agricultural literature illustrating the diversity of opinions:
  Carmer, S.G. and Walker, W.M. 1982. Baby bear's dilemma: a statistical tale. Agronomy Journal 74:122-124.
  Argues for never using m.c.p. (i.e. always using l.s.d.) because it provides highest power.
  Peterson, R.G. 1977. Use and misuse of multiple comparison procedures. Agronomy Journal 69:205-208.
  Recommends m.c.p. when 'data snooping', i.e. when looking for interesting patterns. m.c.p. are 'almost never appropriate' for quantitative treatments, factorial treatment structures, and answering specific questions.
- Hsu, J.C. 1996. Multiple comparisons: theory and methods. Chapman and Hall/CRC. A good intermediate level text on many methods. Chapter 6 (pp 175-180) is full of good, useful advice.
False Discovery Rate
- Original paper: Benjamini and Hochberg, 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Statistical Soc. B 57:289-300.
- Four comments/applications:
  Garci, L.V. 2003. Controlling the false discovery rate in ecological research. TREE 18:553-554.
  Fernando, R., Nettleton, D. et al. 2004. Controlling the proportion of false positives in multiple dependent tests. Genetics 166:611-619.
  By the "home team" (Rohan Fernando is in Animal Science and Dan Nettleton is chair of Statistics).
  Genovese, C.R. et al. 2002. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15:870-878.
  Garcia, L. 2004. Escaping the Bonferroni iron claw in ecological research. Oikos 105(3): 657-663.
- A very critical commentary on FDR (section 7) and multiple comparisons (section 6):
  Hurlbert, S.H. and Lombardi, C.M. 2012. Lopsided reasoning on lopsided tests and multiple comparisons. Australian & New Zealand Journal of Statistics. 54(1):23–42.
Anscombe's plea for more looking at data: Anscombe, F.J. 1973, Graphs in Statistical Analysis, The American Statistician 27(1):17-21 Link to paper May require access through the library