Jensen's inequality

Imagine a shape that curves like a bowl.

This is a convex shape. A straight line can sit on top of it. The line stays above the bowl.

A man named Johan Jensen studied this. He found a rule for these shapes. It is called Jensen's inequality.

This rule helps us with averages. It compares two different ways to find a middle. One way is to find the middle first. Then you use the curve. The other way is to use the curve first.

Sometimes the two ways give the same answer. This happens if the shape is a straight line. It is a very useful tool in math.

Imagine a shape that curves like a bowl.

This is called a convex shape. A straight line can sit on top of this bowl. The line always stays above the curve.

A math expert named Johan Jensen studied these shapes. He lived in Denmark. In 1906, he proved a special rule. We call this rule Jensen's inequality.

This rule compares two ways to use a curve and an average. The first way is to find the average of some numbers first. Then, you apply the curve to that average. The second way is to apply the curve to each number first. Then, you find the average of those new values.

Jensen's inequality says the first way will be smaller than or equal to the second way. This rule is very helpful in many areas. It helps in a field called probability. It also helps in a field called information theory. In information theory, it helps find the best way to send messages. The rule also works if the shape is a straight line. In that case, both ways give the same answer.

Imagine a shape that curves like a deep bowl. In math, we call this a convex shape.

If you draw a straight line between two points on this bowl, that line will always sit above the curve. This is a very important idea. Jensen's inequality is a rule that describes how these curves work with averages. It helps us understand the relationship between a curve and the middle point of a group of values. This rule is used by many people to solve hard problems in science and math. It shows us that the way we calculate things can change depending on the shape we use.

To understand how it works, think about two different steps. The first way is to find the average of a group of numbers first. Then, you apply the curve to that single average value. The second way is to apply the curve to every number in the group first. After that, you find the average of all those new curved values. Jensen's inequality tells us that the first way will be smaller than or equal to the second way. This gap between the two results is sometimes called the Jensen gap. If the shape is just a straight line, both ways will give you the exact same answer.

This math rule is named after a man named Johan Jensen. He was a mathematician from Denmark. He proved this specific inequality in the year 1906. Before him, another mathematician named Otto Hölder worked on a similar idea. In 1889, Hölder proved a version of this rule for certain types of smooth curves. Jensen's work was special because it was much more general. This means his rule could be used for many more kinds of shapes and situations. His discovery helped math grow into new areas like probability.

There are many ways to write this rule using different math tools. In probability theory, we use it to talk about random variables.

If we have a random variable and a convex function, the inequality still holds true. Scientists also use it in a field called information theory. It helps them find the best way to send messages using codes. In statistical physics, the rule is very important when using exponential shapes. It even helps in a rule called the Rao-Blackwell theorem. This theorem helps people find better ways to estimate unknown numbers.

You can see this math in action in your everyday life. Think about how people feel about risk and money. Some people prefer a certain, smaller prize over a chance at a huge prize. This feeling can be explained using Jensen's inequality and the way people value things. It also appears when we look at how much information is in a message.

By using these rules, experts can make sure systems work as well as possible. Whether it is physics or sending a text, these curves are everywhere. Math helps us see the hidden patterns in the world around us.

Jensen's inequality is a fundamental principle in mathematics that relates the value of a convex function of an integral to the integral of that convex function. In simpler terms, it describes a predictable relationship between a curve and the averages of values on that curve. This inequality is essential because it provides a way to compare two different methods of calculation. One method involves finding an average first and then applying a function to it. The second method applies the function to individual values first and then calculates their average.

This rule is used across many scientific fields, including probability theory, information theory, and statistical physics.

To understand the mechanism, we must first define a convex function. A function is convex if a straight line drawn between any two points on its graph stays above or on the graph itself. When we apply a convex function to a set of numbers, the inequality states that the function of the mean is less than or equal to the mean of the function. If the function is concave, which is the opposite of convex, the inequality is reversed. The difference between the two sides of the inequality is known as the Jensen gap. If the function is a straight line, the two sides are equal, and there is no gap.

There are several ways to express this inequality depending on the mathematical context. The finite form uses specific numbers and positive weights to show how weighted averages behave. In this version, the inequality applies to any real convex function and any set of numbers in its domain. The measure-theoretic form is more advanced and uses the language of integration. It applies to non-negative integrable functions and probability spaces.

Finally, the probabilistic form is widely used in statistics. It describes a random variable X and a convex function, stating that the expected value of the function of X is greater than or equal to the function of the expected value of X.

The history of this discovery involves several important mathematicians. The inequality is named after Johan Jensen, a Danish mathematician. He provided a general proof for the inequality in 1906. His work was significant because it was much more broad than previous versions. Before Jensen, Otto Hölder proved a similar inequality in 1889. However, Hölder's proof only worked for doubly-differentiable functions, which are specific types of smooth curves. Jensen's contribution allowed the rule to apply to a much wider variety of mathematical situations.

The significance of Jensen's inequality is seen in its many specific applications. In information theory, it leads to Gibbs' inequality. This result shows that the average message length is minimized when codes are assigned based on true probabilities. It also defines the Kullback–Leibler divergence, which measures the difference between two probability distributions. In statistical physics, the inequality is vital when working with exponential functions. It helps scientists understand how systems behave under different probability distributions.

Another notable application is the Rao–Blackwell theorem in statistics. This theorem uses the conditional version of Jensen's inequality to improve estimators. If a researcher is trying to estimate an unknown parameter, they can use a sufficient statistic to find a better estimate. This new estimate has a smaller expected loss, making it more accurate. The theorem shows how we can use known information to reduce uncertainty in our calculations. This makes the inequality a powerful tool for anyone working with data and observations.

Jensen's inequality also connects to how humans perceive value and risk. In economics, it can describe risk aversion. This occurs when people prefer a certain outcome over a gamble with a higher average value. This preference is linked to the idea of declining marginal utility. The inequality provides a formal way to model these human behaviors mathematically. By connecting abstract curves to real-world decisions, Jensen's inequality shows how mathematics helps us understand the logic of the world.