Variance

Variance is a mathematical way to measure dispersion. Dispersion describes how spread out a set of numbers is from their average value, known as the mean. While the mean tells you the center of a group, variance tells you how much the individual values wander away from that center.

This concept is vital in statistics for describing data, testing hypotheses, and performing Monte Carlo sampling. It helps researchers understand if data points are tightly clustered or widely scattered.

To calculate variance, you follow a specific sequence of steps. First, you find the mean of the dataset. Next, you determine the deviation for each number by subtracting the mean from it. Because some deviations will be negative, you square each of these differences. Squaring ensures all values are positive and gives more weight to larger distances.

Finally, you find the expected value of these squared deviations. This result is the variance, often represented by the symbol $\sigma^2$ (sigma squared).

Mathematicians distinguish between two specific types of variance. The first is population variance, which is used when you have every possible observation from a system. The second is sample variance, which is used when you only have a subset of the data. In most real-world studies, researchers only have access to a sample. Therefore, they use the sample variance as an estimate to guess the true population variance.

There is a deep connection between these two concepts. A theoretical probability distribution can act as a generator for hypothetical observations. If you were to generate an infinite number of observations from a distribution, the sample variance would eventually match the theoretical variance of the distribution. This link allows mathematicians to use equations to predict how real-world samples will behave.

Variance possesses unique properties that make it useful for complex algebra. One major advantage is that it is more amenable to algebraic manipulation than other measures, such as the expected absolute deviation. For example, if you have two uncorrelated random variables, the variance of their sum is simply the sum of their individual variances.

This specific rule is known as the Bienaymé formula. It was discovered in 1853 by I.-J. Bienaymé and is a cornerstone of statistical theory.

Despite its power, variance has some practical disadvantages. One issue is that its units are the square of the original units. If you measure distance in meters, the variance is expressed in meters squared. This makes the number difficult to visualize in a real-world context. To fix this, scientists often report the standard deviation, which is the square root of the variance. The standard deviation returns the measurement to its original unit, such as meters.

Another challenge is that variance is not always finite. Some mathematical distributions, like the Cauchy distribution, do not have a finite expected value. If the expected value is not finite, the variance cannot be finite either. Even some distributions with a finite mean, such as certain Pareto distributions, may lack a finite variance. This means that in some systems, the spread is so extreme that it cannot be captured by a single number.

Variance is also closely related to the concept of covariance. In fact, the variance of a random variable is equivalent to its covariance with itself. This relationship connects variance to broader fields like linear regression analysis. In regression, the total observed score is seen as the sum of a predicted score and an error score. Because these two parts are uncorrelated, their variances can be added together to understand the total spread of the data.