Main Page: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
← Older edit
mNo edit summary
No edit summary
 
(463 intermediate revisions by more than 100 users not shown)
Line 1: Line 1:
In [[statistics]], the '''score''', '''score function''', '''efficient score'''<ref name=Cox1>Cox & Hinkley (1974), p 107</ref> or '''informant'''<ref>{{SpringerEOM| title=Informant |id=i/i051030 |first=N.N. |last=Chentsov}}</ref> indicates how sensitively a [[likelihood function]] <math>L(\theta; X)</math> depends on its [[parametric model|parameter]] <math>\theta</math>. Explicitly, the score for <math>\theta</math> is the [[gradient]] of the log-likelihood with respect to <math>\theta</math>.
This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users.


The score plays an important role in several aspects of [[statistical inference|inference]]. For example:
If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]]
:*in formulating a [[test statistic]] for a locally most powerful test;<ref>Cox & Hinkley (1974), p 113</ref>
* Only registered users will be able to execute this rendering mode.
:*in approximating the error in a [[maximum likelihood]] estimate;<ref name="Cox & Hinkley 1974, p 295">Cox & Hinkley (1974), p 295</ref>
* Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.
:*in demonstrating the asymptotic sufficiency of a [[maximum likelihood]] estimate;<ref name="Cox & Hinkley 1974, p 295"/>
:*in the formulation of [[confidence interval]]s;<ref>Cox & Hinkley (1974), p 222–3</ref>
:*in demonstrations of the [[Cramér–Rao bound|Cramér–Rao inequality]].<ref>Cox & Hinkley (1974), p 254</ref>


The score function also plays an important role in [[computational statistics]], as it can play a part in the computation of
Registered users will be able to choose between the following three rendering modes:
maximum likelihood estimates.


==Definition==
'''MathML'''
:<math forcemathmode="mathml">E=mc^2</math>


The score or efficient score <ref name="Cox1"/> is the [[gradient]] (the vector of [[partial derivative]]s), with respect to some parameter <math>\theta</math>, of the [[logarithm]] (commonly the [[natural logarithm]]) of the [[likelihood function]] (the log-likelihood).
<!--'''PNG'''  (currently default in production)
If the observation is <math>X</math> and its likelihood is <math>L(\theta;X)</math>, then the score <math>V</math> can be found through the [[chain rule]]:
:<math forcemathmode="png">E=mc^2</math>


:<math>
'''source'''
V \equiv V(\theta, X)
:<math forcemathmode="source">E=mc^2</math> -->
=
\frac{\partial}{\partial\theta} \log L(\theta;X)
=
\frac{1}{L(\theta;X)} \frac{\partial L(\theta;X)}{\partial\theta}.
</math>


Thus the score <math>V</math> indicates the [[Sensitivity analysis|sensitivity]] of <math>L(\theta;X)</math> (its derivative normalized by its value). Note that <math>V</math> is a function of <math>\theta</math> and the observation <math>X</math>, so that, in general, it is not a [[statistic]]. However in certain applications, such as the [[score test]], the score is evaluated at a specific value of <math>\theta</math> (such as a null-hypothesis value, or at the maximum likelihood estimate of  <math>\theta</math>), in which case the result is a statistic.
<span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples].


In older literature, the term "linear score" may be used to refer to the score with respect to infintesimal translation of a given density.  This convention arises from a time when the primary parameter of interest was the mean or median of a distribution. In this case, the likelihood of an observation is given by a density of the form <math>L(\theta;X)=f(X+\theta)</math>.  The "linear score" is then defined as
==Demos==


:<math>
Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]:
V_{\rm linear}
=
\frac{\partial}{\partial X} \log f(X)
</math>


==Properties==
===Mean===
Under some regularity conditions, the [[expected value]] of <math>V</math> with respect to the observation <math>x</math>, given <math>\theta</math>, written <math>\mathbb{E}(V\mid\theta)</math>, is zero.
To see this rewrite the likelihood function L as a [[probability density function]]  <math>L(\theta; x) = f(x; \theta)</math>. Then:


:<math>
* accessibility:
\mathbb{E}(V\mid\theta)
** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]]
=\int_{-\infty}^{+\infty}
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)]
f(x; \theta) \frac{\partial}{\partial\theta} \log L(\theta;X)
** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)]
\,dx
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)]
=\int_{-\infty}^{+\infty}
** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]].
\frac{\partial}{\partial\theta} \log L(\theta;X) f(x; \theta) \, dx
** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]].
</math>
** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode.


:<math>
==Test pages ==
=\int_{-\infty}^{+\infty}
\frac{1}{f(x; \theta)}\frac{\partial f(x; \theta)}{\partial \theta}f(x; \theta)\, dx
=\int_{-\infty}^{+\infty} \frac{\partial f(x; \theta)}{\partial \theta} \, dx
</math>


If certain differentiability conditions are met (see [[Leibniz integral rule]]), the integral may be rewritten as
To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages:
*[[Displaystyle]]
*[[MathAxisAlignment]]
*[[Styling]]
*[[Linebreaking]]
*[[Unique Ids]]
*[[Help:Formula]]


:<math>
*[[Inputtypes|Inputtypes (private Wikis only)]]
\frac{\partial}{\partial\theta} \int_{-\infty}^{+\infty}
*[[Url2Image|Url2Image (private Wikis only)]]
f(x; \theta) \, dx
==Bug reporting==
=
If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de .
\frac{\partial}{\partial\theta}1 = 0.
</math>
 
It is worth restating the above result in words: the expected value of the score is zero.
Thus, if one were to repeatedly sample from some distribution, and repeatedly calculate the score, then the mean value of the scores would tend to zero as the number of repeat samples approached infinity.
 
===Variance===
{{Main|Fisher information}}
The variance of the score is known as the [[Fisher information]] and is written <math>\mathcal{I}(\theta)</math>.  Because the expectation of the score is zero, this may be written as
 
:<math>
\mathcal{I}(\theta)
=
\mathbb{E}
\left\{\left.
\left[
  \frac{\partial}{\partial\theta} \log L(\theta;X)
\right]^2
\right|\theta\right\}.
</math>
 
Note that the Fisher information, as defined above, is not a function of any particular observation, as the random variable <math>X</math> has been averaged out.
This concept of information is useful when comparing two methods of observation of some [[random process]].
 
==Examples==
 
===Bernoulli process===
 
Consider a [[Bernoulli process]], with ''A'' successes and ''B'' failures; the probability of success is&nbsp;''θ''.
 
Then the likelihood ''L'' is
 
:<math>
L(\theta;A,B)=\frac{(A+B)!}{A!B!}\theta^A(1-\theta)^B,</math>
 
so the score ''V'' is
 
:<math>
V=\frac{1}{L}\frac{\partial L}{\partial\theta} = \frac{A}{\theta}-\frac{B}{1-\theta}.
</math>
 
We can now verify that the expectation of the score is zero. Noting that the expectation of ''A'' is ''n''θ and the expectation of ''B'' is ''n''(1&nbsp;&minus;&nbsp;θ) [recall that ''A'' and ''B'' are random variables], we can see that the expectation of ''V'' is
 
:<math>
E(V)
= \frac{n\theta}{\theta} - \frac{n(1-\theta)}{1-\theta}
= n - n
= 0.
</math>
 
We can also check the variance of <math>V</math>. We know that ''A'' + ''B'' = ''n'' (so ''B'' =&nbsp;''n''&nbsp;&minus;&nbsp;''A'') and the variance of ''A'' is ''n''θ(1&nbsp;&minus;&nbsp;θ) so the variance of ''V'' is
 
:<math>
\begin{align}
\operatorname{var}(V) & =\operatorname{var}\left(\frac{A}{\theta}-\frac{n-A}{1-\theta}\right)
=\operatorname{var}\left(A\left(\frac{1}{\theta}+\frac{1}{1-\theta}\right)\right) \\
& =\left(\frac{1}{\theta}+\frac{1}{1-\theta}\right)^2\operatorname{var}(A)
=\frac{n}{\theta(1-\theta)}.
\end{align}
</math>
 
===Binary outcome model===
 
For models with binary outcomes (''Y'' = 1 or 0), the model can be scored with the logarithm of predictions
 
<math> S = Y \log( p ) + ( Y - 1 ) ( \log( 1 - p ) ) </math>
 
where ''p'' is the probability in the model to be estimated and ''S'' is the score.<ref name=Steyerberg2010>Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M,  Obuchowski N, Pencina MJ and Kattan MW (2010) Assessing the performance of prediction models. A framework for traditional and novel measures. Epidemiology 21 (1) 128–138 [[DOI: 10.1097/EDE.0b013e3181c30fb2]]</ref>
 
==Applications==
===Scoring algorithm===
{{Main|Scoring algorithm}}
The scoring algorithm is an iterative method for numerically determining the [[maximum likelihood]] [[estimator]].
 
===Score test===
{{Main|Score test}}
{{Expand section|date=December 2009}}
 
==See also==
*[[Fisher information]]
*[[Information theory]]
*[[Score test]]
*[[Scoring algorithm]]
*[[Support curve]]
 
==Notes==
{{Reflist}}
==References==
*Cox, D.R., Hinkley, D.V. (1974) ''Theoretical Statistics'', Chapman & Hall. ISBN 0-412-12420-3
*{{cite book
| last = Schervish
| first = Mark J.
| title = Theory of Statistics
| publisher =Springer
| date =1995
| location =New York
| pages = Section 2.3.1
| isbn = 0-387-94546-6
| nopp = true}}
 
[[Category:Estimation theory]]

Latest revision as of 23:52, 15 September 2019

This is a preview for the new MathML rendering mode (with SVG fallback), which is availble in production for registered users.

If you would like use the MathML rendering mode, you need a wikipedia user account that can be registered here [[1]]

  • Only registered users will be able to execute this rendering mode.
  • Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.

Registered users will be able to choose between the following three rendering modes:

MathML


Follow this link to change your Math rendering settings. You can also add a Custom CSS to force the MathML/SVG rendering or select different font families. See these examples.

Demos

Here are some demos:


Test pages

To test the MathML, PNG, and source rendering modes, please go to one of the following test pages:

Bug reporting

If you find any bugs, please report them at Bugzilla, or write an email to math_bugs (at) ckurs (dot) de .

Retrieved from "https://en.formulasearchengine.com/index.php?title=Main_Page&oldid=329754"