Classical total variation definition

Following Saks (1937, p. 10), consider a signed measure ${\displaystyle \mu }$ on a measurable space ${\displaystyle (X,\Sigma )}$: then it is possible to define two set functions ${\displaystyle {\overline {\mathrm {W} }}(\mu ,\cdot )}$ and ${\displaystyle {\underline {\mathrm {W} }}(\mu ,\cdot )}$, respectively called upper variation and lower variation, as follows

${\displaystyle {\overline {\mathrm {W} }}(\mu ,E)=\sup \left\{\mu (A)\mid A\in \Sigma {\text{ and }}A\subset E\right\}\qquad \forall E\in \Sigma }$

${\displaystyle {\underline {\mathrm {W} }}(\mu ,E)=\inf \left\{\mu (A)\mid A\in \Sigma {\text{ and }}A\subset E\right\}\qquad \forall E\in \Sigma }$

clearly

${\displaystyle {\overline {\mathrm {W} }}(\mu ,E)\geq 0\geq {\underline {\mathrm {W} }}(\mu ,E)\qquad \forall E\in \Sigma }$

Definition 1.3. The variation (also called absolute variation) of the signed measure ${\displaystyle \mu }$ is the set function

${\displaystyle |\mu |(E)={\overline {\mathrm {W} }}(\mu ,E)+\left|{\underline {\mathrm {W} }}(\mu ,E)\right|\qquad \forall E\in \Sigma }$

and its total variation is defined as the value of this measure on the whole space of definition, i.e.

${\displaystyle \|\mu \|=|\mu |(X)}$

Modern definition of total variation norm

Saks (1937, p. 11) uses upper and lower variations to prove the Hahn–Jordan decomposition: according to his version of this theorem, the upper and lower variation are respectively a non-negative and a non-positive measure. Using a more modern notation, define

${\displaystyle \mu ^{+}(\cdot )={\overline {\mathrm {W} }}(\mu ,\cdot )\,,}$

${\displaystyle \mu ^{-}(\cdot )=-{\underline {\mathrm {W} }}(\mu ,\cdot )\,,}$

Then ${\displaystyle \mu ^{+}}$ and ${\displaystyle \mu ^{-}}$ are two non-negative measures such that

${\displaystyle \mu =\mu ^{+}-\mu ^{-}}$

${\displaystyle |\mu |=\mu ^{+}+\mu ^{-}}$

The last measure is sometimes called, by abuse of notation, total variation measure.

Total variation norm of complex measures

If the measure ${\displaystyle \mu }$ is complex-valued i.e. is a complex measure, its upper and lower variation cannot be defined and the Hahn–Jordan decomposition theorem can only be applied to its real and imaginary parts. However, it is possible to follow Rudin (1966, pp. 137–139) and define the total variation of the complex-valued measure ${\displaystyle \mu }$ as follows

Definition 1.4. The variation of the complex-valued measure ${\displaystyle \mu }$ is the set function

${\displaystyle |\mu |(E)=\sup _{\pi }\sum _{A\in \pi }|\mu (A)|\qquad \forall E\in \Sigma }$

where the supremum is taken over all partitions ${\displaystyle \pi }$ of a measurable set ${\displaystyle E}$ into a countable number of disjoint measurable subsets.

This definition coincides with the above definition ${\displaystyle |\mu |=\mu ^{+}+\mu ^{-}}$ for the case of real-valued signed measures.

Total variation norm of vector-valued measures

The variation so defined is a positive measure (see Rudin (1966, p. 139)) and coincides with the one defined by 1.3 when ${\displaystyle \mu }$ is a signed measure: its total variation is defined as above. This definition works also if ${\displaystyle \mu }$ is a vector measure: the variation is then defined by the following formula

${\displaystyle |\mu |(E)=\sup _{\pi }\sum _{A\in \pi }\|\mu (A)\|\qquad \forall E\in \Sigma }$

where the supremum is as above. This definition is slightly more general than the one given by Rudin (1966, p. 138) since it requires only to consider finite partitions of the space ${\displaystyle X}$: this implies that it can be used also to define the total variation on finite-additive measures.

Total variation of probability measures

The total variation of any probability measure is exactly one, therefore it is not interesting as a means of investigating the properties of such measures. However, when μ and ν are probability measures, the total variation distance of probability measures can be defined as ${\displaystyle \|\mu -\nu \|}$ where the norm is the total variation norm of signed measures. Using the property that ${\displaystyle (\mu -\nu )(X)=0}$, we eventually arrive at the equivalent definition

${\displaystyle \|\mu -\nu \|=|\mu -\nu |(X)=2\sup \left\{\,\left|\mu (A)-\nu (A)\right|:A\in \Sigma \,\right\}}$

and its values are non-trivial. The factor ${\displaystyle 2}$ above is usually dropped (as is the convention in the article total variation distance of probability measures). Informally, this is the largest possible difference between the probabilities that the two probability distributions can assign to the same event. For a categorical distribution it is possible to write the total variation distance as follows

${\displaystyle \delta (\mu ,\nu )=\sum _{x}\left|\mu (x)-\nu (x)\right|\;.}$

It may also be normalized to values in ${\displaystyle [0,1]}$ by halving the previous definition as follows

${\displaystyle \delta (\mu ,\nu )={\frac {1}{2}}\sum _{x}\left|\mu (x)-\nu (x)\right|}$^[2]

The form of the total variation of a differentiable function of one variable

Theorem 1. The total variation of a differentiable function ${\displaystyle f}$, defined on an interval ${\displaystyle [a,b]\subset \mathbb {R} }$, has the following expression if ${\displaystyle f'}$ is Riemann integrable

${\displaystyle V_{a}^{b}(f)=\int _{a}^{b}|f'(x)|\mathrm {d} x}$

If ${\displaystyle f}$ is differentiable and monotonic, then the above simplifies to

${\displaystyle V_{a}^{b}(f)=|f(a)-f(b)|}$

For any differentiable function ${\displaystyle f}$, we can decompose the domain interval ${\displaystyle [a,b]}$, into subintervals ${\displaystyle [a,a_{1}],[a_{1},a_{2}],\dots ,[a_{N},b]}$ (with ${\displaystyle a<a_{1}<a_{2}<\cdots <a_{N}<b}$) in which ${\displaystyle f}$ is locally monotonic, then the total variation of ${\displaystyle f}$ over ${\displaystyle [a,b]}$ can be written as the sum of local variations on those subintervals:

${\displaystyle {\begin{aligned}V_{a}^{b}(f)&=V_{a}^{a_{1}}(f)+V_{a_{1}}^{a_{2}}(f)+\,\cdots \,+V_{a_{N}}^{b}(f)\\[0.3em]&=|f(a)-f(a_{1})|+|f(a_{1})-f(a_{2})|+\,\cdots \,+|f(a_{N})-f(b)|\end{aligned}}}$

The form of the total variation of a differentiable function of several variables

Theorem 2. Given a ${\displaystyle C^{1}({\overline {\Omega }})}$ function ${\displaystyle f}$ defined on a bounded open set ${\displaystyle \Omega \subseteq \mathbb {R} ^{n}}$, with ${\displaystyle \partial \Omega }$ of class ${\displaystyle C^{1}}$, the total variation of ${\displaystyle f}$ has the following expression

${\displaystyle V(f,\Omega )=\int _{\Omega }\left|\nabla f(x)\right|\mathrm {d} x}$ .