Statistical inference

Imagine you want to know about all the fish in a big lake. You cannot catch them all. Instead, you catch a few fish. This small group is called a sample.

You look at your small group to learn things. You might guess how many fish live in the whole lake. This is called making an inference. It is like using a small clue to solve a big puzzle.

Sometimes, you use a model to help you. A model is a set of ideas about how things work. You must use the right ideas to get a good answer. If your ideas are wrong, your guess might be wrong too.

Scientists use these tools to learn about the world. It helps them understand things they cannot see all at once.

Imagine you want to know about all the fish in a big lake. You cannot catch them all. Instead, you catch a few fish. This small group is called a sample.

Statistical inference is the way we use that sample to learn about the whole group. The whole group is called a population. We use the data to make a guess. This guess is called a statistical proposition. One way to do this is to find a point estimate. This is a single value that best fits the truth. Another way is to find an interval estimate. This is a range of values, like a confidence interval. A confidence interval helps us say how sure we are.

To do this, we use a statistical model. A model is a set of assumptions. It is a set of ideas about how the data was made. There are different kinds of models. Some are fully parametric. These use a small number of rules to describe the data. Others are non-parametric. These use very few rules. Some are in between, called semi-parametric. We must pick the right model. If our assumptions are wrong, our conclusions might be wrong too.

Imagine you want to know about every fish in a giant lake. You cannot catch every single fish to count them. Instead, you catch a small group to study. This small group is called a sample. Statistical inference is the way we use that sample to learn about the whole group. The whole group is called a population. We use the data to make a smart guess. This guess is called a statistical proposition.

There are different ways to make these guesses. One way is to find a point estimate. This is a single number that best fits the truth. Another way is to find an interval estimate. This is a range of numbers, like a confidence interval. A confidence interval tells us how sure we are. If we used the same method many times, a certain percentage of those intervals would contain the true answer. For example, a 95% confidence interval is very common. You might also use a credible interval to show what you believe is true.

To make these guesses, we must use a statistical model. A model is a set of assumptions. It is a set of ideas about how the data was made. There are three main levels of models. Fully parametric models use a few specific rules to describe data. Non-parametric models use very few rules at all. Semi-parametric models are right in the middle. They might use a rule for one part of the data but not for another. The Cox model is a famous example of a semi-parametric model.

Choosing the right model is a very important job. If our assumptions are wrong, our conclusions might be wrong too. For example, assuming a population follows a normal distribution can be a mistake. A normal distribution looks like a bell curve. If we use the wrong model for an economic population, it could be a big error. Experts often use very large samples to help make things more accurate. They use something called the central limit theorem to help with this. This theorem helps the data act more like a normal distribution.

Math helps us understand how close our guesses are to the truth. We can use math to see how much error might be in our work. Some scientists use simulation to check their results. This means they run many tests to see what happens. We can also use math to study what happens as a sample gets bigger and bigger. This is called asymptotic theory. Even though we usually work with a finite number of items, these big ideas help us guide our work. It helps us turn small samples into big discoveries.

Statistical inference is the scientific process of using data analysis to learn about an underlying probability distribution. In many studies, researchers cannot observe every single member of a group. This large group is called the population. Instead, they collect a smaller group called a sample. Statistical inference allows scientists to make educated propositions about the entire population based only on that sample. This is different from descriptive statistics, which only describes the specific data that was actually collected.

The process of inference typically follows a specific sequence of logical steps. First, a researcher must select a statistical model to describe the process that generated the data. This model acts as a mathematical framework for understanding how the sample relates to the population. Second, the researcher uses that model to deduce propositions. These propositions are the conclusions drawn from the data. Common forms of these conclusions include a point estimate, which is a single value that best approximates a population parameter. Another common form is an interval estimate, such as a confidence interval. A 95% confidence interval is a range of values constructed from sample data. If the procedure were repeated many times, 95% of those resulting intervals would contain the true population parameter.

Statisticians categorize statistical models into three distinct levels based on their assumptions. Fully parametric models are the most specific. They assume the data-generation process is described by a family of distributions with a finite number of unknown parameters. An example is assuming a population follows a Normal distribution with an unknown mean and variance. Non-parametric models make much fewer assumptions about the data. For instance, one might estimate a median without assuming a specific shape for the distribution. Semi-parametric models sit between these two approaches. They might use a parametric assumption for one part of the data and a non-parametric approach for another. The well-known Cox model is a famous example of a semi-parametric model.

History shows that the transition from a real-world problem to a mathematical model is a critical step. Sir David Cox noted that translating a subject-matter problem into a statistical model is often the most vital part of any analysis. Many problems in this field are essentially problems of statistical modeling. In the 1950s, the work of Andrey Kolmogorov helped advance the field through approximation theory. This allowed researchers to quantify the error when using one distribution to approximate another. Today, advanced statistics uses functional analysis to study the geometry of probability distributions. This helps scientists measure the distance between different mathematical models.

Accuracy in statistical inference depends heavily on the validity of the initial assumptions. If a researcher uses an incorrect model, the resulting conclusions may be faulty. For example, assuming a population follows a Normal distribution can invalidate certain types of regression-based inference. This is especially dangerous in economic populations, where a Normal distribution might be an unrealistic assumption. To avoid these errors, many experts rely on the central limit theorem. This theorem states that the distribution of a sample mean will be approximately normal if the sample size is very large. This allows for more reliable inferences even when the underlying population is not perfectly normal.

Because it is difficult to know the exact distribution of a sample, mathematicians use various approximation methods. For example, the Berry–Esseen theorem shows that with 10,000 independent samples, a normal distribution can approximate a sample mean to two digits of accuracy. In many practical cases, even 10 independent samples can provide a good approximation. Researchers also use asymptotic theory to study what happens as a sample size tends toward infinity. While limit theorems describe infinite samples, they are often used to guide work on finite samples. To check the magnitude of error in these approximations, scientists often use computer simulations.

Statistical inference also connects deeply to the concept of randomization. In randomized experiments, the data is produced by a specific design that can be mathematically evaluated. In frequentist inference, this allows conclusions to be based on the randomization itself rather than a subjective model. This is very important in survey sampling and the design of experiments. Bayesian inference also utilizes randomization to ensure that samples are exchangeable with the population. By using these rigorous methods, statisticians can turn small, finite sets of observations into powerful tools for understanding the world.