Skorokhod's embedding theorem

2013-06-27T03:32:06Z

128.12.187.216: /* Skorokhod's second embedding theorem */ corrected somebody's poor english

: ''Not to be confused with [[Kernel principal component analysis]].''
The '''kernel regression''' is a [[non-parametric]] technique in statistics to estimate the [[conditional expectation]] of a [[random variable]]. The objective is to find a non-linear relation between a pair of random variables '''''X''''' and '''''Y'''''.

In any [[nonparametric regression]], the [[conditional expectation]] of a variable <math>Y</math> relative to a variable <math>X</math> may be written:

<math>\operatorname{E}(Y | X) = m(X)</math>

where <math>m</math> is an unknown function.

== Nadaraya-Watson kernel regression ==
{{harvnb|Nadaraya|1964}} and {{harvnb|Watson|1964}} proposed to estimate <math>m</math> as a locally weighted average, using a [[kernel (statistics)|kernel]] as a weighting function. The Nadaraya-Watson estimator is:

<math> \widehat{m}_h(x)=\frac{\sum_{i=1}^n K_h(x-X_i) Y_i}{\sum_{i=1}^nK_h(x-X_i)}
</math>

where <math>K</math> is a kernel with a bandwidth <math>h</math>. The fraction is a weighting term with sum 1.

=== Derivation ===
<math>
\operatorname{E}(Y | X) = \int y f(y|x) dy = \int y \frac{f(x,y)}{f(x)} dy
</math>

Using the [[kernel density estimation]] for the joint distribution ''f(x,y)'' and ''f(x)'' with a kernel '''''K''''',

<math>
\hat{f}(x,y) = n^{-1} h^{-2} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right) K\left(\frac{y-y_i}{h}\right)
</math>,<br />
<math>
\hat{f}(x) = n^{-1} h^{-1} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right)
</math>

we obtain the Nadaraya-Watson estimator.

== Priestley-Chao kernel estimator ==
<math>
\widehat{m}_{PC}(x) = h^{-1} \sum_{i=1}^n (x_i - x_{i-1}) K\left(\frac{x-x_i}{h}\right) y_i
</math>

== Gasser-Müller kernel estimator ==
<math>
\widehat{m}_{GM}(x) = h^{-1} \sum_{i=1}^n \left[\int_{s_{i-1}}^{s_i} K\left(\frac{x-u}{h}\right) du\right] y_i
</math>

where <math>s_i = \frac{x_{i-1} + x_i}{2}</math>

== Example ==

This example is based upon Canadian cross-section wage data consisting
of a random sample taken from the 1971 Canadian Census Public Use
Tapes for male individuals having common education (grade 13). There
are 205 observations in total.

We consider estimating the unknown regression function using
Nadaraya-Watson kernel regression via the
[http://cran.r-project.org/web/packages/np/index.html R np package]
that uses automatic (data-driven) bandwidth selection; see the [http://cran.r-project.org/web/packages/np/vignettes/np.pdf np vignette] for an introduction to the np package.

The figure below shows the estimated regression function using a
second order Gaussian kernel along with asymptotic variability bounds

[[File:cps71 lc mean.png|center|360px]]
<center>Estimated Regression Function.</center>

=== Script for example ===

The following commands of the [[R programming language]] use the
<tt>npreg()</tt> function to deliver optimal smoothing and to create
the figure given above. These commands can be entered at the command
prompt via cut and paste.

library(np) # non parametric library
data(cps71)
attach(cps71)

m <- npreg(logwage~age)

plot(m,plot.errors.method="asymptotic",
plot.errors.style="band",
ylim=c(11,15.2))

points(age,logwage,cex=.25)

== Related ==
According to {{harvnb|Salsburg|2002|pp=290–1}}, the algorithms used in kernel regression were independently developed and used in [[fuzzy system]]s: "Coming up with almost exactly the same computer algorithm, fuzzy systems and kernel density-based regressions appear to have been developed completely independently of one another."

== References ==

{{Reflist}}

*{{cite journal
| last = Nadaraya
| first = E. A.
| title = On Estimating Regression
| journal = Theory of Probability and its Applications
| volume = 9
| issue = 1
| pages = 141–2
| year = 1964
| doi = 10.1137/1109020 | ref=harv
}}

*{{cite book
| last = Li
| first = Qi
| coauthors = Racine, Jeffrey S.
| title = Nonparametric Econometrics: Theory and Practice
| publisher = Princeton University Press
| year = 2007
| isbn = 0-691-12161-3}}

*{{cite book
| last = Simonoff
| first = Jeffrey S.
| title = Smoothing Methods in Statistics
| publisher = Springer
| year = 1996
| isbn = 0-387-94716-7}}

*{{cite book |last=Salsburg |first=D. |title=The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century |publisher=W.H. Freeman |year=2002 |isbn=0-8050-7134-2 |ref=harv}}

*{{cite journal |author=Richard, C.; Bermudez, J.-C. M.; Honeine, P. |title=Online prediction of time series data with kernels |journal=IEEE Transactions on Signal Processing |volume=57 |issue=3 |pages=1058–67 |date=March 2009 |doi=10.1109/TSP.2008.2009895 |url=http://www.cedric-richard.fr/Articles/richard2009online.pdf|format=PDF}}

*{{cite journal |author=Parreira, W.; Bermudez, J.-C. M.; Richard, C.; Tourneret, J.-Y. |title=Stochastic behavior analysis of the Gaussian kernel-least-mean-square algorithm. |journal=IEEE Transactions on Signal Processing |volume=60 |issue=5 |pages=2208–2222 |date=May 2012 |doi=10.1109/TSP.2012.2186132 |url=http://www.cedric-richard.fr/Articles/parreira2012stochastic.pdf|format=PDF}}

*{{cite journal |author=Richard, C.; Bermudez, J.-C. M. |title=Closed-form conditions for convergence of the Gaussian kernel-least-mean-square algorithm. |journal=Proc. of Asilomar'12 |pages=1797–1801 |date=November 2012 |doi=10.1109/ACSSC.2012.6489344 |url=http://www.cedric-richard.fr/Articles/richard2012closed.pdf|format=PDF}}

*{{cite journal |first=G. S. |last=Watson |authorlink=Geoffrey Watson |title=Smooth regression analysis |journal=Sankhyā: The Indian Journal of Statistics, Series A |volume=26 |issue=4 |pages=359–372 |year=1964 |jstor=25049340 |ref=harv}}

==Statistical implementation==
* [[Stata]] [http://ideas.repec.org/c/boc/bocode/s372601.html kernreg2]
<pre> kernreg2 y x, bwidth(.5) kercode(3) npoint(500) gen(kernelprediction gridofpoints)</pre>
* [[R (programming language)|R]]: [http://cran.r-project.org/web/packages/np/index.html npreg (package ''np'')]
* [[GNU Octave|GNU/octave]] mathematical program package:

==External links==
* [http://www.cs.tut.fi/~lasip Scale-adaptive kernel regression] (with Matlab software).
* [http://people.revoledu.com/kardi/tutorial/Regression/KernelRegression/index.html Tutorial of Kernel regression using spreadsheet] (with Microsoft Excel).
* [http://pcarvalho.com/things/kernelregressor/ An online kernel regression demonstration] Requires .NET 3.0 or later.
* [http://cran.r-project.org/web/packages/np/index.html The np package] An [[R (programming language)|R]] package that provides a variety of nonparametric and semiparametric kernel methods that seamlessly handle a mix of continuous, unordered, and ordered factor data types.

[[Category:Non-parametric statistics]]

Classical Wiener space

2013-06-27T03:17:05Z

128.12.187.216: /* Properties of classical Wiener space */

{{unreferenced|date=August 2009}}
{{Expert-subject|Mathematics|date=February 2009}}

In [[mathematics]], the '''Malliavin derivative''' is a notion of [[derivative]] in the [[Malliavin calculus]]. Intuitively, it is the notion of derivative appropriate to paths in [[classical Wiener space]], which are "usually" not differentiable in the usual sense. {{Citation Needed|date=August 2011}}

==Definition==
Let <math>H</math> be the [[Cameron-Martin space]], and <math>C_{0}</math> denote [[classical Wiener space]]:

:<math>H := \{ f \in W^{1,2} ([0, T]; \mathbb{R}^{n}) \;|\; f(0) = 0 \} := \{ \text{paths starting at 0 with first derivative in } L^{2} \}</math>;

:<math>C_{0} := C_{0} ([0, T]; \mathbb{R}^{n}) := \{ \text{continuous paths starting at 0} \}</math>;

By the [[Sobolev_inequality#Sobolev_embedding_theorem|Sobolev embedding theorem]], <math>H \subset C_0</math>. Let
:<math>i : H \to C_{0}</math>
denote the [[inclusion map]].

Suppose that <math>F : C_{0} \to \mathbb{R}</math> is [[Fréchet derivative|Fréchet differentiable]]. Then the [[Fréchet derivative]] is a map

:<math>\mathrm{D} F : C_{0} \to \mathrm{Lin} (C_{0}; \mathbb{R})</math>;

i.e., for paths <math>\sigma \in C_{0}</math>, <math>\mathrm{D} F (\sigma)\;</math> is an element of <math>C_{0}^{*}</math>, the [[dual space]] to <math>C_{0}\;</math>. Denote by <math>\mathrm{D}_{H} F(\sigma)\;</math> the [[continuous function|continuous]] [[linear map]] <math>H \to \mathbb{R}</math> defined by

:<math>\mathrm{D}_{H} F (\sigma) := \mathrm{D} F (\sigma) \circ i : H \to \mathbb{R}, </math>

sometimes known as the [[H-derivative|''H''-derivative]]. Now define <math>\nabla_{H} F : C_{0} \to H</math> to be the [[adjoint]]{{dn|date=December 2013}} of <math>\mathrm{D}_{H} F\;</math> in the sense that

:<math>\int_0^T \left(\partial_t \nabla_H F(\sigma)\right) \cdot \partial_t h := \langle \nabla_{H} F (\sigma), h \rangle_{H} = \left( \mathrm{D}_{H} F \right) (\sigma) (h) = \lim_{t \to 0} \frac{F (\sigma + t i(h)) - F(\sigma)}{t}</math>.

Then the '''Malliavin derivative''' <math>\mathrm{D}_{t}</math> is defined by

:<math>\left( \mathrm{D}_{t} F \right) (\sigma) := \frac{\partial}{\partial t} \left( \left( \nabla_{H} F \right) (\sigma) \right).</math>

The [[domain (mathematics)|domain]] of <math>\mathrm{D}_{t}</math> is the set <math>\mathbf{F}</math> of all Fréchet differentiable real-valued functions on <math>C_{0}\;</math>; the [[codomain]] is <math>L^{2} ([0, T]; \mathbb{R}^{n})</math>.

The '''Skorokhod integral''' <math>\delta\;</math> is defined to be the [[adjoint]]{{dn|date=December 2013}} of the Malliavin derivative:

:<math>\delta := \left( \mathrm{D}_{t} \right)^{*} : \operatorname{image} \left( \mathrm{D}_{t} \right) \subseteq L^{2} ([0, T]; \mathbb{R}^{n}) \to \mathbf{F}^{*} = \mathrm{Lin} (\mathbf{F}; \mathbb{R}).</math>

==See also==
*[[H-derivative]]

==References==
{{reflist}}

[[Category:Generalizations of the derivative]]
[[Category:Stochastic calculus]]

formulasearchengine - User contributions [en]

Skorokhod's embedding theorem

Classical Wiener space