Frequentist inference

Frequentist inference is a specific method of statistical reasoning. It relies on the concept of frequentist probability. In this framework, probability is treated as being equivalent to frequency. This means scientists look at how often a specific outcome occurs within a data sample. By emphasizing these proportions, they can draw conclusions about a larger population. This approach is the foundation for many well-established methodologies. These include statistical hypothesis testing and the creation of confidence intervals.

To understand the mechanism, we must look at how parameters are handled. A statistic is used to make inferences about an unknown parameter. This parameter is often partitioned into two distinct parts. The first is the parameter of interest, which is what the scientist wants to know. The second is a nuisance parameter, which is an extra variable that must be accounted for. For example, if a scientist wants to find the population mean, the standard deviation might act as a nuisance parameter. To manage uncertainty, frequentists use a mathematical tool called a pivot. A pivot is a function that helps define an area around a statistic. This area provides an interval to estimate how much uncertainty exists.

There are two complementary concepts used to construct these intervals. The first is known as the Fisherian reduction. This method helps determine the specific interval where a true value might lie. It follows a precise sequence of steps. First, a scientist determines the likelihood function by gathering data. Next, they reduce that data to a sufficient statistic. This statistic must have the same dimension as the parameter of interest. The scientist then finds a distribution that depends only on that parameter. By inverting this distribution, they can obtain limits for the parameter at different probability levels.

The second concept is the Neyman-Pearson operational criteria. This serves as a decision rule for making probability assumptions before an experiment begins. It defines the likelihood of a range being adequate or inadequate. Specifically, it identifies a range of a probability distribution that remains below the true population statistic. This criteria is essential for identifying Type I and Type II errors. A Type I error occurs when a false hypothesis is rejected. A Type II error occurs when a true hypothesis is not rejected. The Neyman-Pearson approach is only possible when a Fisherian reduction can first be achieved.

The history of this field involves several influential mathematicians. Ronald Fisher was a primary developer of frequentist statistics. He contributed the concept of significance testing. This involves studying the significance of a statistic when compared to a specific hypothesis. Later, Jerzy Neyman and Egon Pearson extended Fisher's work. They developed methods to apply these ideas to multiple hypotheses at once. They discovered that maximizing the difference between two hypotheses leads to maximizing the p-value. This mathematical relationship became the basis for modern error analysis and confidence intervals.

Frequentist inference can be viewed through two different philosophical lenses. The first is the epistemic approach, which focuses on the study of variability. It asks how often a statistic might deviate from an observed value. The second is the epidemiological approach, which focuses on the study of uncertainty. In this view, the statistic is considered fixed, but our understanding of it is incomplete. For instance, the stock market is often viewed through an epistemic lens because prices fluctuate constantly. In contrast, the price of a specific asset might be viewed through an epidemiological lens to find its true value.

Finally, frequentist inference is distinct from Bayesian inference. Bayesian inference treats probability as a measure of certainty. Frequentist statistics, however, is conditioned on the entire experimental design. This means the results can change based on the model selected for the experiment. For example, using a binomial distribution versus a negative binomial distribution can yield different levels of statistical significance for the same data. This occurs because the "tails" of these distributions are different. This characteristic highlights how frequentist conclusions are tied to the specific way an experiment is structured and repeated.