Visualizing the Multivariate Normal

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

Visualizing the Multivariate Normal

 

Spectral Decomposition

P is orthogonal if PT P = 1 and PPT = 1.

Theorem: Let A be symmetric n × n. Then we can write

A = PDPT ,

where D = diag (λ₁, . . . , λ_n) and P is orthogonal. The λs are the eigenvalues of A and ith column of P is an eigenvector corresponding to λ_i .

Orthogonal matrices represent rotations of the coordinates. Diagonal matrices represent stretchings/shrinkings of coordinates.

Properties

• The covariance matrix Σ is symmetric and positive definite, so we know from the spectral decomposition theorem that it can be written as

Σ = PΛPT .

• Λ is the diagonal matrix of the eigenvalues of Σ.
• P is the matrix whose columns are the orthonormal eigenvectors of Σ (hence V is an orthogonal matrix).

) Geometrically, orthogonal matrices represent rotations.

) Multiplying by P rotates the coordinate axes so that they are parallel to the eigenvectors of Σ.

) Probabilistically, this tells us that the axes of the probability-contour ellipse are parallel to those eigenvectors.

) The radii of those axes are proportional to the square roots of the eigenvalues.

Can we view the det(Σ) as a “variance“?

•  

Q

• Variance of one-dimensional
• From the SDT: det(Σ) = _i λ_i .
• Eigenvalues (λ_i ) tell us how stretched or compressed the distribution
• View det(Σ) as stretching/compressing factor for the MVN
• We will see this from the contour plots

Our focus is visualizing MVN distributions in R.

What is a Contour Plot?

• Contour plot is a graphical technique for representing a 3-dimensional
• We plot constant z slices (contours) on a 2-D
• The contour plot is an alternative to a 3-D surface The contour plot is formed by:
• Vertical axis: Independent variable
• Horizontal axis: Independent variable
• Lines: iso-response

Contour Plot

The lines of the contour plots denote places of equal probability mass for the MVN distribution

• The lines represent points of both variables that lead to the same height on the z-axis (the height of the surface)
• These contours can be constructed from the eigenvalues and eigenvectors of the covariance matrix
• The direction of the ellipse axes are in the direction of the eigenvalues
• The length of the ellipse axes are proportional to the constant times the eigenvector
• More specifically

||Σ−1/2(X − µ)|| = c2

has ellipsoids centered at µ and axes at √(λ_i v_i )

Visualizing the MVN Distribution Using Contour Plots

The next figure below shows a contour plot of the joint pdf of a bivariate normal distribution. Note: we are plotting the theoretical contour plot. This particular distribution has mean

1

µ = . 1 Σ

 

(Solid dot), and variance matrix

1 1

Σ =. 2 1 Σ

Code to construct plot

0.04   0.06 0.08

library(mvtnorm)

x.points <- seq(-3,3,length.out=100) y.points <-x.points

z <- matrix(0,nrow=100,ncol=100) mu <- c(1,1)

sigma <- matrix(c(2,1,1,1),nrow=2) for (i in1:100) {

for (j in1:100) {

z[i,j] <- dmvnorm(c(x.points[i],y.points[j]),

mean=mu,sigma=sigma)

}

}

contour(x.points,y.points,z)

Our findings

• Probability contours are
• Density changes comparatively slowly along the major axis, and quickly along the minor
• The two points marked + in the figure have equal geometric distance from µ.
• But the one to its right lies on a higher probability contour than the one above it, because of the directions of their displacements from the means

Kernel density estimation (KDE)

• KDE allows us to estimate the density from which each sample was
• This method (which you will learn about in other classes) allows us to approximate the density using a
• There are R packages that use kde’s such as density().

What did we learn?

• The contour plot of X (bivariate density): Color is the probability density at each point (red is low density and white is high density).
• Contour lines define regions of probability density (from high to low).
• Single point where the density is highest (in the white region) and the contours are approximately ellipses (which is what you expect from a Gaussian).

 

What can we say in general about the MVN density?

 

• The spectral decomposition theorem tells us that the contours of the multivariate normal distribution are
• The axes of the ellipsoids correspond to eigenvectors of the covariance
• The radii of the ellipsoids are proportional to square roots of the eigenvalues of the covariance

Related Questions

. The fundamental operations of create, read, update, and delete (CRUD) in either Python or Java

CS 340 Milestone One Guidelines and Rubric  Overview: For this assignment, you will implement the fundamental operations of create, read, update,

. Develop a program to emulate a purchase transaction at a retail store. This  program will have two classes, a LineItem class and a Transaction class

Retail Transaction Programming Project  Project Requirements:  Develop a program to emulate a purchase transaction at a retail store. This

. The following program contains five errors. Identify the errors and fix them

7COM1028   Secure Systems Programming   Referral Coursework: Secure

. Accepts the following from a user: Item Name Item Quantity Item Price Allows the user to create a file to store the sales receipt contents

Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip

. The final project will encompass developing a web service using a software stack and implementing an industry-standard interface. Regardless of whether you choose to pursue application development goals as a pure developer or as a software engineer

CS 340 Final Project Guidelines and Rubric  Overview The final project will encompass developing a web service using a software stack and impleme