Evidence-Based Statistics. Peter M. B. Cahusac

approaches associated with p values is relegated to Appendix C.

      1.1.1 The Literature

      The use of evidence based on likelihoods and likelihood ratios (LRs) strikes those unfamiliar with it as highly specialized and esoteric, even arcane. There is widespread belief, though misguided, that evidential methodology can only be used safely and credibly by highly experienced or professional statisticians. A contributing factor supporting this belief is the fact that, compared with other areas of statistical methodology, there are relatively few books and research papers on the evidential approach. However, the quality of the texts makes up for their quantity.

The most important book on the subject is Likelihood by Edwards. Originally published in 1972, it represented a highly original text. An expanded edition was subsequently published in 1992 [3]. A.W.F. Edwards (below photo) is a statistician and geneticist who did his PhD with R.A. Fisher, who was also a statistician and geneticist. Edwards's ground-breaking book covers a remarkable range of topics. Sometimes densely written, other times appearing to cover important topics, such as the F ratio, in a cursory fashion. The succinct text, peppered with dry humour and understatement, repays careful reading and re-reading. Many glittering gems relevant to applied statistics await to be mined and polished.

      Professor A.W.F. Edwards FRS. Source: Photo from Gonville and Caius College, Cambridge.

      Royall's book [4], Statistical Evidence: A Likelihood Paradigm, published 25 years later is a remarkable monograph, providing a tour de force of carefully argued prose and examples to convince anyone still in doubt about the merits of the evidential approach. The book adds to Edwards's work, for example by explaining how sample size calculations relevant to the evidential approach can be done.

      The books by Edwards and Royall are outstanding sources of reference for theory and examples. They make an appeal to reason as to why statistical inferences based on statistical tests and Bayesian methods are flawed, and that only the likelihood approach is valid. These books may appear somewhat inaccessible to readers who lack sufficient mathematical or statistical expertise.

      A deep theoretical and philosophical treatment of the likelihood approach is given by Hacking [5]. This may appeal to philosophers and theoreticians but there is little there for the applied statistician or researcher.

There are some excellent books with large sections devoted to the evidential approach. First up is the book by Dienes with his excellent, cogent, and entertaining Understanding Psychology as a Science: An introduction to Scientific and Statistical Inference [6]. Then, there is the very solid and thorough treatment by Baguley in Serious Stats: A Guide to Advanced Statistics for the Behavioral Sciences [7]. Both these books offer limited computer code to perform LR calculations. Taper and Lele edited The Nature of Scientific Evidence: Statistical, Philosophical, and Empirical Considerations which consists of a compilation of chapters including some notable authors, such as Royall, Mayo, and others [1]. There are commentaries to the chapters, including by D.R. Cox who was critical of Royall's approach, which was followed by a robust and memorable rejoinder by Royall.

      The book by Aitken is a useful addition, but is limited in scope to forensic statistical evidence [8]. Pawitan's In All Likelihood is a useful mathematical treatment of a range of likelihood topics [9]. Clayton and Hills's Statistical Models in Epidemiology [10] is excellent but limits itself to epidemiological statistics. Lindsey's book Introductory Statistics: A Modelling Approach [11], makes extensive use of the likelihood approach. Kirkwood and Sterne's Medical Statistics [12] is a useful practical book that devotes a chapter to likelihood. Armitage et al's Statistical Methods in Medical Research [13] is a solid standard reference work for medical statistics which makes passing references to the likelihood approach. There are some excellent books that use a modelling approach, although without likelihoods, for example Maxwell and Delaney's Designing Experiments and Analyzing Data: A Model Comparison Perspective [14] and Judd et al's Data Analysis: A Model Comparison Approach to Regression, ANOVA, and Beyond [15].

      Perhaps the most concentrated account of likelihood, given in just a few pages, is by Edwards in a 2015 entry for an encyclopaedia [16]. There are a number of accessible research papers. Those by Goodman [17–21] (one of these jointly with Royall), and Dixon and Glover [22, 23] are exemplary in explaining and demonstrating a range of evidential techniques.

1.2 Statistical Inference – The Basics

Together with the data, statistical hypotheses and statistical models are essential components for us to be able to draw inferences. Hypotheses and models provide an adequate probabilistic explanation of the process by which the observed data were generated. By statistical hypothesis, we mean attributing a specified quantitative or qualitative value to an identified parameter of interest within the statistical model. A simple statistical hypothesis specifies a particular value. For example, the null hypothesis for a measured difference between two populations might be exactly 0. A hypothesis may also be a range of values, known as a composite hypothesis, for example the direction of difference in a measurement of two populations (e.g. A > B). By statistical model, we mean the mathematical assumptions we make about how sample data (and similar samples from the same population) were generated. Typically, a model is a convenient simplification of a more complex reality. Statistical inference and estimation are conditional on a model. For example, in comparing heights of malnourished and well-nourished adults, our model could assume that the measurements are normally distributed. We might specify two simple hypotheses to be compared, for example: the null of 0 difference and a population mean difference of more than 3 cm between the two populations. The distinction between hypothesis and model is not absolute since it is possible to consider one of these components to be part of the model on one occasion and then contested as a hypothesis on another. Hence, our model assumption of normally distributed data could itself be questioned by becoming a hypothesis.

      1.2.1 Different Statistical Approaches

      There are three main statistical approaches to data analysis. These are neatly summarized by Royall's three questions that follow the collection and analysis of some data [24]:

      1 What should I do?

      2 What should I believe?

      3 How should I interpret the evidence?

      They describe the different ways in which the data are analyzed and interpreted. Each approach is important within their specific domain. The first of these is pragmatic, where a decision must be made on the basis of the analysis. It represents the frequentist approaches of statistical tests and hypothesis testing. Typically, either the null hypothesis is rejected (evidence for an effect is found) or not rejected (insufficient evidence found). The decision is based upon a critical probability, usually .05. The significance testing approach measures the strength of evidence against the null hypothesis by the diminutiveness of a calculated probability of obtaining the data (or more extreme) assuming the null hypothesis is true. This probability is known as a p value.

      The second approach represents the strength of belief for a specified hypothesis. It too is based upon probability and is conditioned by the probability of the hypothesis prior to the collection of the data. If the prior probability is known, then the calculation using Bayes' theorem logically provides the (posterior) probability for the specified hypothesis.

The third approach also uses probability but provides objective evidence which is expressed as the likelihood for one hypothesis versus another in the form of a LR. The LR is not a probability but a relative measure of evidence for competing hypotheses. The technical meaning of the word ‘likelihood’ in statistics is very similar to its use in common parlance by non-statisticians. For example we might say, seeing dark clouds