A reanalysis of Lord’s statistical treatment of football numbers

Stevens’ theory of admissible statistics [Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103, 677680] states that measurement levels should guide the choice of statistical test, such that the truth value of statements based on a statistical analysis remains invariant under admissible transformations of the data. Lord [Lord, F. M. (1953). On the statistical treatment of football numbers. American Psychologist, 8, 750–751] challenged this theory. In a thought experiment, a parametric test is performed on football numbers (identifying players: a nominal representation) to decide whether a sample from the machine issuing these numbers should be considered non-random. This is an apparently illegal test, since its outcomes are not invariant under admissible transformations for the nominal measurement level. Nevertheless, it results in a sensible conclusion: the number-issuing machine was tampered with. In the ensuing measurement-statistics debate Lord’s contribution has been influential, but has also led to much confusion. The present aim is to show that the thought experiment contains a serious flaw. First it is shown that the implicit assumption that the numbers are nominal is false. This disqualifies Lord’s argument as a valid counterexample to Stevens’ dictum. Second, it is argued that the football numbers do not represent just the nominal property of non-identity of the players; they also represent the amount of bias in the machine. It is a question about this property–not a property that relates to the identity of the football players–that the statistical test is concerned with. Therefore, only this property is relevant to Lord’s argument. We argue that the level of bias in the machine, indicated by the population mean, conforms to a bisymmetric structure, which means that it lies on an interval scale. In this light, Lord’s thought experiment–interpreted by many as a problematic counterexample to Stevens’ theory of admissible statistics–conforms perfectly to Stevens’ dictum.