Fisher–Tippett–Gnedenko theorem

In statistics, the Fisher–Tippett–Gnedenko theorem (also the Fisher–Tippett theorem or the extreme value theorem) is a general result in extreme value theory regarding asymptotic distribution of extreme order statistics. The maximum of a sample of iid random variables after proper renormalization can only converge in distribution to one of only 3 possible distribution families: the Gumbel distribution, the Fréchet distribution, or the Weibull distribution. Credit for the extreme value theorem and its convergence details are given to Fréchet (1927),^[1] Fisher and Tippett (1928),^[2] Mises (1936),^[3]^[4] and Gnedenko (1943).^[5]

The role of the extremal types theorem for maxima is similar to that of central limit theorem for averages, except that the central limit theorem applies to the average of a sample from any distribution with finite variance, while the Fisher–Tippet–Gnedenko theorem only states that if the distribution of a normalized maximum converges, then the limit has to be one of a particular class of distributions. It does not state that the distribution of the normalized maximum does converge.

Statement

Let $\ X_{1},X_{2},\ldots ,X_{n}\$ be an $n$ -sized sample of independent and identically-distributed random variables, each of whose cumulative distribution function is $\ F~.$ Suppose that there exist two sequences of real numbers $\ a_{n}>0\$ and $\ b_{n}\in \mathbb {R} \$ such that the following limits converge to a non-degenerate distribution function:

\lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\left\{{\frac {\ \max\{X_{1},\dots ,X_{n}\}-b_{n}\ }{a_{n}}}\leq x\ \right\}=G(x)\ ,

or equivalently:

\lim _{n\to \infty }{\Bigl (}\ F\left(\ a_{n}\ x+b_{n}\ \right){\Bigr )}^{n}=G(x)~.

In such circumstances, the limiting distribution $\ G\$ belongs to either the Gumbel, the Fréchet, or the Weibull distribution family.^[6]

In other words, if the limit above converges, then up to a linear change of coordinates $G(x)$ will assume either the form:^[7]

G_{\gamma }(x)=\exp \left(-{\Bigl (}1+\gamma \ x{\Bigr )}^{\left({\tfrac {-1\;}{\gamma }}\right)}\right)\quad

for

\quad \gamma \neq 0\ ,

with the non-zero parameter $\ \gamma \$ also satisfying $\ 1+\gamma \ x>0\$ for every $\ x\$ value supported by $\ F\$ (for all values $\ x\$ for which $\ F(x)\neq 0\$ ). Otherwise it has the form:

G_{0}(x)=\exp {\bigl (}\ -\exp(-x)\ {\bigr )}\quad

for

\quad \gamma =0~.

This is the cumulative distribution function of the generalized extreme value distribution (GEV) with extreme value index $\ \gamma ~.\$ The GEV distribution groups the Gumbel, Fréchet, and Weibull distributions into a single composite form.

Conditions of convergence

The Fisher–Tippett–Gnedenko theorem is a statement about the convergence of the limiting distribution $\ G(x)\ ,$ above. The study of conditions for convergence of $\ G\$ to particular cases of the generalized extreme value distribution began with Mises (1936)^[3]^[5]^[4] and was further developed by Gnedenko (1943).^[5]

Let

\ F\

be the distribution function of

\ X\ ,

and

\ X_{1},\dots ,X_{n}\

be some i.i.d. sample thereof.

Also let

\ x_{\mathsf {max}}\

be the population maximum:

\ x_{\mathsf {max}}\equiv \sup \ \{\ x\ \mid \ F(x)<1\ \}~.\

The limiting distribution of the normalized sample maximum, given by $G$ above, will then be:^[7]

Fréchet distribution $\ \left(\ \gamma >0\ \right)$

For strictly positive

\ \gamma >0\ ,

the limiting distribution converges if and only if

\ x_{\mathsf {max}}=\infty \

and

\ \lim _{t\rightarrow \infty }{\frac {\ 1-F(u\ t)\ }{1-F(t)}}=u^{\left({\tfrac {-1~}{\gamma }}\right)}\

for all

\ u>0~.

In this case, possible sequences that will satisfy the theorem conditions are

b_{n}=0

and

\ a_{n}={F^{-1}}\!\!\left(1-{\tfrac {1}{\ n\ }}\right)~.

Strictly positive

\ \gamma \

corresponds to what is called a heavy tailed distribution.

Gumbel distribution $\ \left(\ \gamma =0\ \right)$

For trivial

\ \gamma =0\ ,

and with

\ x_{\mathsf {max}}\

either finite or infinite, the limiting distribution converges if and only if

\ \lim _{t\rightarrow x_{\mathsf {max}}}{\frac {\ 1-F{\bigl (}\ t+u\ {\tilde {g}}(t)\ {\bigr )}\ }{1-F(t)}}=e^{-u}\

for all

\ u>0\

with

\ {\tilde {g}}(t)\equiv {\frac {\ \int _{t}^{x_{\mathsf {max}}}{\Bigl (}\ 1-F(s)\ {\Bigr )}\ \mathrm {d} \ s\ }{1-F(t)}}~.

Possible sequences here are

\ b_{n}={F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\

and

\ a_{n}={\tilde {g}}{\Bigl (}\;{F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\;{\Bigr )}~.

Weibull distribution $\ \left(\ \gamma <0\ \right)$

For strictly negative

\ \gamma <0\

the limiting distribution converges if and only if

\ x_{\mathsf {max}}\ <\infty \quad

(is finite)

and

\ \lim _{t\rightarrow 0^{+}}{\frac {\ 1-F\!\left(\ x_{\mathsf {max}}-u\ t\ \right)\ }{1-F(\ x_{\mathsf {max}}-t\ )}}=u^{\left({\tfrac {-1~}{\ \gamma \ }}\right)}\

for all

\ u>0~.

Note that for this case the exponential term

\ {\tfrac {-1~}{\ \gamma \ }}\

is strictly positive, since

\ \gamma \

is strictly negative.

Possible sequences here are

\ b_{n}=x_{\mathsf {max}}\

and

\ a_{n}=x_{\mathsf {max}}-{F^{-1}}\!\!\left(\ 1-{\frac {1}{\ n\ }}\ \right)~.

Note that the second formula (the Gumbel distribution) is the limit of the first (the Fréchet distribution) as $\ \gamma \$ goes to zero.

Examples

Fréchet distribution

The Cauchy distribution's density function is:

f(x)={\frac {1}{\ \pi ^{2}+x^{2}\ }}\ ,

and its cumulative distribution function is:

F(x)={\frac {\ 1\ }{2}}+{\frac {1}{\ \pi \ }}\arctan \left({\frac {x}{\ \pi \ }}\right)~.

A little bit of calculus show that the right tail's cumulative distribution $\ 1-F(x)\$ is asymptotic to $\ {\frac {1}{\ x\ }}\ ,$ or

\ln F(x)\rightarrow {\frac {-1~}{\ x\ }}\quad {\mathsf {~as~}}\quad x\rightarrow \infty \ ,

so we have

\ln \left(\ F(x)^{n}\ \right)=n\ \ln F(x)\sim -{\frac {-n~}{\ x\ }}~.

Thus we have

F(x)^{n}\approx \exp \left({\frac {-n~}{\ x\ }}\right)

and letting $\ u\equiv {\frac {x}{\ n\ }}-1\$ (and skipping some explanation)

\lim _{n\to \infty }{\Bigl (}\ F(n\ u+n)^{n}\ {\Bigr )}=\exp \left({\tfrac {-1~}{\ 1+u\ }}\right)=G_{1}(u)\

for any $\ u~.$ The expected maximum value therefore goes up linearly with $n$ .

Gumbel distribution

Let us take the normal distribution with cumulative distribution function

F(x)={\frac {1}{2}}\operatorname {erfc} \left({\frac {-x~}{\ {\sqrt {2\ }}\ }}\right)~.

We have

\ln F(x)\rightarrow -{\frac {\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty

and thus

\ln \left(\ F(x)^{n}\ \right)=n\ln F(x)\rightarrow -{\frac {\ n\exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty ~.

Hence we have

F(x)^{n}\approx \exp \left(-\ {\frac {\ n\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ x\ }}\right)~.

If we define $\ c_{n}\$ as the value that exactly satisfies

{\frac {\ n\exp \left(-\ {\tfrac {1}{2}}c_{n}^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ c_{n}\ }}=1\ ,

then around $\ x=c_{n}\$

{\frac {\ n\ \exp \left(-\ {\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\approx \exp \left(\ c_{n}\ (c_{n}-x)\ \right)~.

As $\ n\$ increases, this becomes a good approximation for a wider and wider range of $\ c_{n}\ (c_{n}-x)\$ so letting $\ u\equiv c_{n}\ (c_{n}-x)\$ we find that

\lim _{n\to \infty }{\biggl (}\ F\left({\tfrac {u}{~c_{n}\ }}+c_{n}\right)^{n}\ {\biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.

Equivalently,

\lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\ {\Biggl (}{\frac {\ \max\{X_{1},\ \ldots ,\ X_{n}\}-c_{n}\ }{\left({\frac {u}{~c_{n}\ }}\right)}}\leq u{\Biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.

With this result, we see retrospectively that we need $\ \ln c_{n}\approx {\frac {\ \ln \ln n\ }{2}}\$ and then

c_{n}\approx {\sqrt {2\ln n\ }}\ ,

so the maximum is expected to climb toward infinity ever more slowly.

Weibull distribution

We may take the simplest example, a uniform distribution between $0$ and $1$ , with cumulative distribution function

F(x)=x\

for any

x

value from

0

to

1

.

For values of $\ x\ \rightarrow \ 1\$ we have

\ln {\Bigl (}\ F(x)^{n}\ {\Bigr )}=n\ \ln F(x)\ \rightarrow \ n\ (\ 1-x\ )~.

So for $\ x\approx 1\$ we have

\ F(x)^{n}\approx \exp(\ n\ x-n\ )~.

Let $\ u\equiv 1+n\ (\ 1-x\ )\$ and get

\lim _{n\to \infty }{\Bigl (}\ F\!\left({\tfrac {\ u\ }{n}}+1-{\tfrac {\ 1\ }{n}}\right)\ {\Bigr )}^{n}=\exp \!{\bigl (}\ -(1-u)\ {\bigr )}=G_{-1}(u)~.

Close examination of that limit shows that the expected maximum approaches $1$ in inverse proportion to $n$ .

References

↑ Fréchet, M. (1927). "Sur la loi de probabilité de l'écart maximum". Annales de la Société Polonaise de Mathématique. 6 (1): 93–116.
↑ Fisher, R.A.; Tippett, L.H.C. (1928). "Limiting forms of the frequency distribution of the largest and smallest member of a sample". Proc. Camb. Phil. Soc. 24 (2): 180–190. Bibcode:1928PCPS...24..180F. doi:10.1017/s0305004100015681. S2CID 123125823.
1 2 von Mises, R. (1936). "La distribution de la plus grande de $n$ valeurs" [The distribution of the largest of $n$ values]. Rev. Math. Union Interbalcanique. 1 (in French): 141–160.
1 2 Falk, Michael; Marohn, Frank (1993). "von Mises conditions revisited". The Annals of Probability: 1310–1328.
1 2 3 Gnedenko, B.V. (1943). "Sur la distribution limite du terme maximum d'une serie aleatoire". Annals of Mathematics. 44 (3): 423–453. doi:10.2307/1968974. JSTOR 1968974.
↑ Mood, A.M. (1950). "5. Order Statistics". Introduction to the theory of statistics. New York, NY: McGraw-Hill. pp. 251–270.
1 2 Haan, Laurens; Ferreira, Ana (2007). Extreme Value Theory: An introduction. Springer.

Statement

Conditions of convergence

Examples

Fréchet distribution

Gumbel distribution

Weibull distribution

See also

References

Further reading