I found an obscure (to me, at least) paper on “philosophy of statistics” by Donald Sharpe at the University of Regina. Or rather, the philosophy of teaching statistics, not necessarily in the formal sense, but in the my trainees need to learn this sense.
His hypothesis is that statistics as a discipline moves forward with new methods and innovations, as well as the hmm, we weren’t quite right about this one, let’s change the accepted wisdom, as many fields do. Yet, the practitioners of data analysis are resistant to those kinds of changes (a comfort in which, I sheepishly admit, I have indulged).
First the bad: The abstract of the paper is frustrating as hell, because it is full of passive voice and the infuriating “Traditional explanations for this resistance are reviewed“. Didn’t anyone tell him not to write content free statements in the abstract (nb: if you did that in a grant you would be sliced up and served for breakfast). Let’s leave the abstract, because it is very nearly useless. After reading that abstract I wanted to say “researchers are resistant to opaque, impenetrable writing”.
Moving on, there is much in this paper to which I take exception. He says that the standard hypothesis testing, Fisher-ian null hypothesis, etc is a “persistent irritation”. I wonder if he would like to take drugs that had not gone through such a testing. The author cites psychologists who want alternatives such as effect size measures, confidence intervals for understanding or representing the precision of a finding. This alternative in fact goes back to Neymand & Pearson in the 1920’s. For a relatively accessible paper on the history and differences between these two ideas, try Steven Goodman’s work (Goodman is an unrepentant Bayesian, which is another cup of tea). To quote Goodman: “The p-value was meant to be a flexible inferential measure, whereas the hypothesis test was a rule for behavior, not inference”. [Am J Epi. 137:485].
Yet, the reason for this post is that there is much to appreciate in Sharpe’s paper, some which is worth thinking about, if you do any kind of quantitative data analysis. In the reasons section, he highlights the issues of power and sample size, something frequently overlooked. Second, he brings out the issues of assumptions embedded in standard tests. I think that some testing/methods assumptions are killers (ie your answers are wrong), while others may be more flexible (it can produce a bias in this direction, or only subtle problems), and finally, others are beyond nearly everybody’s ability to test for them in a realistic and functional way (multivariate normal distributions).
His discussion of “resistance” is remarkably qualitative, including a few citations and no numbers.
His discussion of “why resistance” is actually kinda interesting, and big chunks of it apply to harder sciences, too. There is lack of awareness. If you don’t have time to read the primary literature in your field, who the hell has time to keep up with data analysis advances (except for a couple of us nerdy types)? He also takes on the idea the journal editors should be leading the charge for change. He talks about a couple of psych journals that tried to do this, and mostly seem to have failed. He says, and I suspect this one is a big issue, there is a lack of qualified reviewers. Its not just the reviewers, its the editors who balance reviewers. I’ve seen so much crap published in my subfield, including things I’ve reviewed and objected to and subsequently thrown up my hands when the editors won’t listen. Being the statistician on the editorial board is like being the woman on the committee – you get tired of doing it real fast.
His next two points are publish or perish, which leads often to sloppiness everywhere, and the nature of statistical packages, which leads to even more sloppiness. His final point is about inadequate eduction. Buried in this para is something I’ve thought about for a long time: that the classes emphasize theoretical, deriving equations, and how to pick “the correct test”, rather than a more Tukey-esque approach of asking: what are my data saying to me? Grad students hate stats classes, can’t wait to get them done, and absolutely fail to make the connection from the class to their projects. Then they end up in my office as postdocs, trying to figure why they can’t analyze their data. Did no one explain the experimental design needs to be done before the experiments?
There is a lot more about communication, but you’ve probably stopped reading already (reminds me of a friend in grad school who sprinkled his final to his committee, in those days paper, copy of his thesis with bad jokes, cartoons and other things just to see if anybody bothered to read it).
One small idea: stop teaching calculus to biology/life science majors. Teach them statistics, or rather data analysis, instead. Its a way of thinking that can change how you look at the world. Don’t teach it as equations, but as the glory of the world made concrete and testable.