Why are Confidence Regions Elliptic? Simple Explanation

A 90% confidence region is a domain of minimum area, containing 90% of the mass of a distribution. By distribution, here I mean a bivariate probability distribution, though the concept is not specific to machine learning. The 90% is called the confidence level, and I denote it as γ. Confidence regions are a generalization of confidence intervals, to two dimensions. They are typically represented using contour maps.

One may argue that ellipses (a particular case of quadratic functions) are the simplest generalization of linear functions, thus their widespread use. But here, there is a much deeper reason. And it is much easier to understand than you think. Many statisticians take it for granted that it should be an ellipse, but I never found a real justification. This article fills this gap. I discuss the elliptic case first, and then provide a non-elliptic example.

The Shape of a Confidence Region

While this is nowhere mentioned in the statistical literature, it makes sense to assume that the confidence region is of minimum area. Determining the shape is then a variational problem. Such problems are solved using mathematical methods of functional analysis and calculus of variation. It involves functional, differential and integral equations. These topics are rather advanced. The most famous example is the brachistochrone problem. Interestingly, finding the shape of a confidence region of minimum area, is perhaps the most elementary in this class of problems.

Read the explanation (solution) and see example of non-elliptic confidence regions, in the full article, here.