Univariate case

The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF)

${\displaystyle f_{X}(x\mid \theta )=h(x)\ \exp {\Big (}\ \eta (\theta )T(x)-A(\theta )\ {\Big )}\,\!,}$

where ${\displaystyle h(x)}$ and ${\displaystyle A(\theta )}$ are known functions. A distribution in a natural exponential family with parameter θ can thus be written with PDF

${\displaystyle f_{X}(x\mid \theta )=h(x)\ \exp {\Big (}\ \theta x-A(\theta )\ {\Big )}\,\!.}$

[Note that slightly different notation is used by the originator of the NEF, Carl Morris.^[1] Morris uses ω instead of η and ψ instead of A.]

General multivariate case

Suppose that ${\displaystyle \mathbf {x} \in {\mathcal {X}}\subseteq \mathbb {R} ^{p}}$, then a natural exponential family of order p has density or mass function of the form:

${\displaystyle f_{X}(\mathbf {x} \mid {\boldsymbol {\theta }})=h(\mathbf {x} )\ \exp {\Big (}{\boldsymbol {\theta }}^{\rm {T}}\mathbf {x} -A({\boldsymbol {\theta }})\ {\Big )}\,\!,}$

where in this case the parameter ${\displaystyle {\boldsymbol {\theta }}\in \mathbb {R} ^{p}.}$

Moment and cumulant generating functions

A member of a natural exponential family has moment generating function (MGF) of the form

${\displaystyle M_{X}(\mathbf {t} )=\exp {\Big (}\ A({\boldsymbol {\theta }}+\mathbf {t} )-A({\boldsymbol {\theta }})\ {\Big )}\,.}$

The cumulant generating function is by definition the logarithm of the MGF, so it is

${\displaystyle K_{X}(\mathbf {t} )=A({\boldsymbol {\theta }}+\mathbf {t} )-A({\boldsymbol {\theta }})\,.}$

Univariate case

1. The cumulants of an NEF can be calculated as derivatives of the NEF's cumulant generating function. The nth cumulant is the nth derivative of the cumulant generating function with respect to t evaluated at t = 0.

The cumulant generating function is

${\displaystyle K_{X}(t)=A(\theta +t)-A(\theta )\,.}$

The first cumulant is

${\displaystyle \kappa _{1}=K_{X}'(t){\Big |}_{t=0}=\left.{\frac {d}{dt}}A(\theta +t)\right|_{t=0}\,.}$

The mean is the first moment and always equal to the first cumulant, so

${\displaystyle \mu _{1}=\kappa _{1}=\operatorname {E} [X]=K'_{X}(0)=A'(\theta )\,.}$

The variance is always the second cumulant, and it is always related to the first and second moments by

${\displaystyle \operatorname {Var} [X]=\kappa _{2}=\mu _{2}-\mu _{1}^{2}\,,}$

so that

${\displaystyle \operatorname {Var} [X]=K''_{X}(0)=A''(\theta )\,.}$

Likewise, the nth cumulant is

${\displaystyle \kappa _{n}=A^{(n)}(\theta )\,.}$

2. Natural exponential families (NEF) are closed under convolution.^[2]

Given independent identically distributed (iid) ${\displaystyle X_{1},\ldots ,X_{n}}$ with distribution from an NEF, then ${\displaystyle \sum _{i=1}^{n}X_{i}\,}$ is an NEF, although not necessarily the original NEF. This follows from the properties of the cumulant generating function.

3. The variance function for random variables with an NEF distribution can be written in terms of the mean.^[2]

${\displaystyle \operatorname {Var} (X)=V(\mu ).}$

4. The first two moments of a NEF distribution uniquely specify the distribution within that family of distributions.^[2]

${\displaystyle X\sim \operatorname {NEF} [\mu ,V(\mu )].}$

Multivariate case

In the multivariate case, the mean vector and covariance matrix are

${\displaystyle \operatorname {E} [X]=\nabla A({\boldsymbol {\theta }}){\text{ and }}\operatorname {Cov} [X]=\nabla \nabla ^{\rm {T}}A({\boldsymbol {\theta }})\,,}$

where${\displaystyle \nabla }$ is the gradient and ${\displaystyle \nabla \nabla ^{\rm {T}}}$ is the Hessian matrix.

The six NEF-QVFs

The six NEF-QVF are written here in increasing complexity of the relationship between variance and mean.

1. The normal distribution with fixed variance ${\displaystyle X\sim N(\mu ,\sigma ^{2})}$ is NEF-QVF because the variance is constant. The variance can be written ${\displaystyle \operatorname {Var} (X)=V(\mu )=\sigma ^{2}}$, so variance is a degree 0 function of the mean.

2. The Poisson distribution ${\displaystyle X\sim \operatorname {Poisson} (\mu )}$ is NEF-QVF because all Poisson distributions have variance equal to the mean ${\displaystyle \operatorname {Var} (X)=V(\mu )=\mu }$, so variance is a linear function of the mean.

3. The Gamma distribution ${\displaystyle X\sim \operatorname {Gamma} (r,\lambda )}$ is NEF-QVF because the mean of the Gamma distribution is ${\displaystyle \mu =r\lambda }$ and the variance of the Gamma distribution is ${\displaystyle \operatorname {Var} (X)=V(\mu )=\mu ^{2}/r}$, so the variance is a quadratic function of the mean.

4. The binomial distribution ${\displaystyle X\sim \operatorname {Binomial} (n,p)}$ is NEF-QVF because the mean is ${\displaystyle \mu =np}$ and the variance is ${\displaystyle \operatorname {Var} (X)=np(1-p)}$ which can be written in terms of the mean as ${\displaystyle V(X)=-np^{2}+np=-\mu ^{2}/n+\mu .}$

5. The negative binomial distribution ${\displaystyle X\sim \operatorname {NegBin} (n,p)}$ is NEF-QVF because the mean is ${\displaystyle \mu =np/(1-p)}$ and the variance is ${\displaystyle V(\mu )=\mu ^{2}/n+\mu .}$

6. The (not very famous) distribution generated by the generalized hyperbolic secant distribution (NEF-GHS) has ${\displaystyle V(\mu )=\mu ^{2}/n+n}$ and ${\displaystyle \mu >0.}$

Properties of NEF-QVF

The properties of NEF-QVF can simplify calculations that use these distributions.

1. Natural exponential families with quadratic variance functions (NEF-QVF) are closed under convolutions of a linear transformation.^[4] That is, a convolution of a linear transformation of an NEF-QVF is also an NEF-QVF, although not necessarily the original one.

Given independent identically distributed (iid) ${\displaystyle X_{1},\ldots ,X_{n}}$ with distribution from a NEF-QVF. A convolution of a linear transformation of an NEF-QVF is also an NEF-QVF.

Let ${\displaystyle Y=\sum _{i=1}^{n}(X_{i}-b)/c\,}$ be the convolution of a linear transformation of X. The mean of Y is ${\displaystyle \mu ^{*}=n(\mu -b)/c\,}$. The variance of Y can be written in terms of the variance function of the original NEF-QVF. If the original NEF-QVF had variance function

${\displaystyle \operatorname {Var} (X)=V(\mu )=\nu _{0}+\nu _{1}\mu +\nu _{2}\mu ^{2},}$

then the new NEF-QVF has variance function

${\displaystyle \operatorname {Var} (Y)=V^{*}(\mu ^{*})=\nu _{0}^{*}+\nu _{1}^{*}\mu +\nu _{2}^{*}\mu ^{2},}$

where

${\displaystyle \nu _{0}^{*}=nV(b)/c^{2}\,,}$

${\displaystyle \nu _{1}^{*}=V'(b)/c\,,}$

${\displaystyle \nu _{2}^{*}/n=\nu _{2}/n\,.}$

2. Let ${\displaystyle X_{1}}$ and ${\displaystyle X_{2}}$ be independent NEF with the same parameter θ and let ${\displaystyle Y=X_{1}+X_{2}}$. Then the conditional distribution of ${\displaystyle X_{1}}$ given ${\displaystyle Y}$ has quadratic variance in ${\displaystyle Y}$ if and only if ${\displaystyle X_{1}}$ and ${\displaystyle X_{2}}$ are NEF-QVF. Examples of such conditional distributions are the normal, binomial, beta, hypergeometric and geometric distributions, which are not all NEF-QVF.^[1]

3. NEF-QVF have conjugate prior distributions on μ in the Pearson system of distributions (also called the Pearson distribution although the Pearson system of distributions is actually a family of distributions rather than a single distribution.) Examples of conjugate prior distributions of NEF-QVF distributions are the normal, gamma, reciprocal gamma, beta, F-, and t- distributions. Again, these conjugate priors are not all NEF-QVF.^[1]

4. If ${\displaystyle X\mid \mu }$ has an NEF-QVF distribution and μ has a conjugate prior distribution then the marginal distributions are well-known distributions.^[1]

These properties together with the above notation can simplify calculations in mathematical statistics that would normally be done using complicated calculations and calculus.