Title: The Two-Sample t-Test and Randomization Test
Author: Keith M. Bower
Publication Source: American Society for Quality, Six Sigma Forum, June 2003
Abstract
In this article, situations in which the two-sample t-test may be considered robust to certain assumptions (including normality) are addressed. The randomization test procedure is illustrated using a hypothetical example.
Notes
As a student of statistics, I can recall several “hand waving” instances when the assumptions required for a particular procedure (e.g. ANOVA) were violated, yet the audience was assured that the procedure was “robust”. I have to admit that I was never quite convinced.
By looking at Fisher’s original work on designed experiments, I became enamored by his “pure” methodology via the randomization test approach. It certainly clarified in my own mind the theoretical legitimacy behind attempting a cause-and-effect argument, as we may base our conclusions strictly along probability lines. A nice discussion of the “tea experiment” is contained in the highly readable “The Lady Tasting Tea” by David Salsburg.
Over the years, several important studies have shown the relationship between “exact” tests and results obtained via more familiar procedures to agree adequately. The references cited in the paper provide more detailed information on this topic.
As an aside, I’ve noticed more articles over recent years discussing the appropriateness of randomization test interpretations instead of using approximate results obtained via ANOVA. This appears to be occurring in the field of medical research in particular. I believe this is an entirely appropriate discussion to be having, in particular owing to ever increasing computing power.
The more mathematical companion paper to this is "Some Comments on the Robustness of Student t Procedures" by Dr. Josh Tebbs and myself.
To download Adobe Acrobat (for free) click here.