Projection (linear algebra)

In linear algebra, a projection is a specific type of linear transformation called an endomorphism. This means the transformation maps a vector space back onto itself. The defining characteristic of a projection is a property called idempotence. A transformation is idempotent if applying it twice results in the same outcome as applying it once. Mathematically, if we call the transformation $P$, then $P^2 = P$. This means that once a vector has been projected into its new position, any further attempts to project it will leave it unchanged.

To understand how a projection works, we must look at its two fundamental components: the range and the kernel. The range, or image, is the subspace where all the projected vectors land. The kernel, or null space, consists of all the vectors that the transformation moves to zero. These two subspaces are complementary. In a finite-dimensional vector space, this means every single vector can be decomposed uniquely into two parts. One part belongs to the range, and the other belongs to the kernel. This relationship allows mathematicians to split a large, complex space into two distinct, manageable pieces.

Projections are categorized into two main types: orthogonal and oblique. An orthogonal projection is a special case where the range and the kernel are orthogonal to each other. This means every vector in the range is perpendicular to every vector in the kernel. In a Hilbert space, which is a complete vector space with an inner product, an orthogonal projection is also self-adjoint. This means the transformation behaves symmetrically with respect to the inner product.

An oblique projection is any projection that does not satisfy this perpendicular requirement. In an oblique projection, the range and kernel are still complementary, but they meet at an angle rather than at a right angle.

We can represent these transformations using square matrices. A matrix is a projection matrix if it equals its own square. For real matrices, an orthogonal projection matrix must also be equal to its transpose. If the matrix is complex, it must be equal to its adjoint, also known as the Hermitian transpose. The eigenvalues of any projection matrix are restricted to only two possible values: 0 or 1. The eigenspace corresponding to the eigenvalue 1 is the range of the projection. Meanwhile, the eigenspace corresponding to the eigenvalue 0 is the kernel.

There are many ways to calculate an orthogonal projection, depending on the information available. If you are projecting onto a line defined by a unit vector $u$, the operation is a simple outer product of $u$ with itself. For more complex subspaces, you can use an orthonormal basis. If you have a matrix $Q$ whose columns form an orthonormal basis for the subspace, the projection matrix is calculated as $QQ^T$. If the basis is not orthonormal, a normalizing factor is required to recover the correct norm. This is often expressed using the formula $M(M^TM)^{-1}M^T$.

In practical applications, orthogonal projections are vital for finding the shortest distance between a point and a subspace. This is a core concept in the Hilbert projection theorem. This principle is used heavily in the method of least squares. In statistics, ordinary least squares regression relies on orthogonal projections to find fitted values. However, some specialized tasks, like instrumental variables regression, actually require the use of oblique projections. These mathematical tools allow us to approximate complex data by finding the closest possible representation within a simpler subspace.

Projections also interact with other mathematical operations in specific ways. For example, the product of two projections is not always a projection itself. However, if two projections commute, their product is guaranteed to be a projection. If they are both orthogonal projections and they commute, their product is also an orthogonal projection. These rules help mathematicians understand how different subspaces overlap and interact. From machine learning to spherical trigonometry, the ability to project data into different dimensions is a fundamental part of modern science.