Burton Wold Wind Farm

2013-07-11T08:49:25Z

86.160.166.9:

In [[statistics]], the '''hat matrix''', '''''H''''', sometimes also called '''projection matrix''', maps the vector of [[observed value]]s to the vector of [[fitted value]]s. It describes the influence each observed value has on each fitted value.<ref name="Hoaglin1977">
{{Citation
| title = The Hat Matrix in Regression and ANOVA
| first1= David C. | last1= Hoaglin
|first2= Roy E. | last2=Welsch
|journal= [[The American Statistician]]
| volume=32 | month=February| year= 1978| pages=17–22
| doi = 10.2307/2683469 |issue=1| jstor = 2683469}}
</ref> The diagonal elements of the hat matrix are the [[leverage (statistics)|leverage]]s, which describe the influence each observed value has on the fitted value for that same observation.

If the vector of observed values is denoted by '''y''' and the vector of fitted values by '''ŷ''',
:<math>\hat{\mathbf{y}} = H \mathbf{y}.</math>
As '''ŷ''' is usually pronounced "y-hat", the hat matrix is so named as it "puts a [[circumflex|hat]] on '''y'''".

Suppose that we wish to solve a [[linear model]] using [[linear least squares (mathematics)|linear least squares]]. The model can be written as
:<math>\mathbf{y} = X \boldsymbol \beta + \boldsymbol \varepsilon,</math>
where ''X'' is a matrix of [[explanatory variable]]s (the [[design matrix]]), '''''β''''' is a vector of unknown parameters to be estimated, and '''''ε''''' is the error vector.

== Uncorrelated errors ==
For uncorrelated [[errors and residuals in statistics|errors]], the estimated parameters are

:<math>\hat{\boldsymbol \beta} = \left(X^\top X \right)^{-1} X^\top \mathbf{y},</math>

so the fitted values are

:<math>\hat{\mathbf{y}} = X \hat{\boldsymbol \beta} = X \left(X^\top X \right)^{-1} X^\top \mathbf{y}.</math>

Therefore the hat matrix is given by

:<math>H = X \left(X^\top X \right)^{-1} X^\top.</math>

In the language of [[linear algebra]], the hat matrix is the [[orthogonal projection]] onto the [[column space]] of the design matrix ''X''. (Note that <math>\left(X^\top X \right)^{-1} X^\top</math> is the [[Moore–Penrose_pseudoinverse#Full_rank|pseudoinverse of X]].)

The hat matrix corresponding to a [[linear model]] is [[symmetric matrix|symmetric]] and [[idempotent]], that is, ''H''2 = ''H''. However, this is not always the case; in [[local regression|locally weighted scatterplot smoothing (LOESS)]], for example, the hat matrix is in general neither symmetric nor idempotent.

The formula for the vector of [[errors and residuals in statistics|residual]]s '''r''' can be expressed compactly using the hat matrix:

:<math>\mathbf{r} = \mathbf{y} - \mathbf{\hat{y}} = \mathbf{y} - H \mathbf{y} = (I - H) \mathbf{y}.</math>

The [[covariance matrix]] of the residuals is therefore, by [[error propagation]], equal to <math>\left(I-H \right)^\top \Sigma\left(I-H \right) </math>, where Σ is the covariance matrix of the errors (and by extension, the observations as well). For the case of linear models with [[independent and identically distributed]] errors in which Σ = ''σ''2''I'', this reduces to (''I'' − ''H'')''σ''2.<ref name="Hoaglin1977"/>

For [[linear models]], the [[trace (linear algebra)|trace]] of the hat matrix is equal to the [[rank (linear algebra)|rank]] of ''X'', which is the number of independent parameters of the linear model. For other models such as LOESS that are still linear in the observations '''y''', the hat matrix can be used to define the [[degrees of freedom (statistics)#Effective degrees of freedom|effective degrees of freedom]] of the model.

The hat matrix has a number of useful algebraic properties.<ref>Gans, P. (1992) ''Data Fitting in the Chemical Sciences,'', Wiley. ISBN 978-0-471-93412-7</ref><ref>Draper, N.R., Smith, H. (1998) ''Applied Regression Analysis'', Wiley. ISBN 0-471-17082-6 {{Please check ISBN|reason=Check digit (6) does not correspond to calculated figure.}}</ref> Practical applications of the hat matrix in regression analysis include [[Leverage (statistics)|leverage]] and [[Cook's distance]], which are concerned with identifying observations which have a large effect on the results of a regression.

==Correlated errors==
The above may be generalized to the case of correlated errors. Suppose that the [[covariance matrix]] of the errors is Σ. Then since

:<math> \hat{\boldsymbol{\beta}} = \left(X^\top \Sigma^{-1} X \right)^{-1} X^\top \Sigma^{-1}\,\mathbf{y}, </math>

the hat matrix is thus

:<math> H = X \left(X^\top \Sigma^{-1} X\right)^{-1} X^\top \Sigma^{-1}, \, </math>

and again it may be seen that ''H''2 = ''H''

==Blockwise formula==

Suppose the design matrix <math>C</math> can be decomposed by columns as <math>C = [A, B]</math>.
Define the Hat operator as <math>H(X) = X \left(X^\top X \right)^{-1} X^\top</math>. Similarly, define the residual operator as <math>M(X) = I - H(X)</math>.
Then the Hat matrix of <math>C</math> can be decomposed as follows:

<math>
H(C) = H(A) + H(M(A) B)
</math>
<ref>{{cite book|last=Rao|first=C. Radhakrishna|title=Linear Models and Generalizations|year=2008|publisher=Springer|location=Berlin|isbn=978-3-540-74226-5|pages=323|edition=3rd|coauthors=Toutenburg, Shalabh, Heumann}}</ref>

There are a number of applications of such a partitioning. The classical application has <math>A</math> a column of all ones, which allows one to analyze the effects of adding an intercept term to a regression. Another use is in the fixed effects model, where <math>A</math> is a large [[sparse matrix]] of the dummy variables for the fixed effect terms. One can use this partition to compute the hat matrix of <math>C</math> without explicitly forming the matrix <math>C</math>, which might be too large to fit into computer memory.

== See also ==
*[[Moore–Penrose pseudoinverse]]
*[[Studentized residuals]]
*[[Degrees of freedom (statistics)#Effective degrees of freedom|Effective degrees of freedom]]
*[[Idempotent matrix]]

== References ==
<references />

[[Category:Statistical terminology]]
[[Category:Regression analysis]]
[[Category:Matrices]]

formulasearchengine - User contributions [en]

Burton Wold Wind Farm