Derivation

Starting with the definition of conditional expectation,

${\displaystyle \operatorname {E} (Y\mid X=x)=\int yf(y\mid x)\,dy=\int y{\frac {f(x,y)}{f(x)}}\,dy}$

we estimate the joint distributions f(x,y) and f(x) using kernel density estimation with a kernel K:

${\displaystyle {\hat {f}}(x,y)={\frac {1}{n}}\sum _{i=1}^{n}K_{h}(x-x_{i})K_{h}(y-y_{i}),}$

${\displaystyle {\hat {f}}(x)={\frac {1}{n}}\sum _{i=1}^{n}K_{h}(x-x_{i}),}$

We get:

${\displaystyle {\begin{aligned}\operatorname {\hat {E}} (Y\mid X=x)&=\int y{\frac {{\hat {f}}(x,y)}{{\hat {f}}(x)}}\,dy,\\[6pt]&=\int y{\frac {\sum _{i=1}^{n}K_{h}(x-x_{i})K_{h}(y-y_{i})}{\sum _{j=1}^{n}K_{h}(x-x_{j})}}\,dy,\\[6pt]&={\frac {\sum _{i=1}^{n}K_{h}(x-x_{i})\int y\,K_{h}(y-y_{i})\,dy}{\sum _{j=1}^{n}K_{h}(x-x_{j})}},\\[6pt]&={\frac {\sum _{i=1}^{n}K_{h}(x-x_{i})y_{i}}{\sum _{j=1}^{n}K_{h}(x-x_{j})}},\end{aligned}}}$

which is the Nadaraya–Watson estimator.

Script for example

The following commands of the R programming language use the npreg() function to deliver optimal smoothing and to create the figure given above. These commands can be entered at the command prompt via cut and paste.

install.packages("np")
library(np) # non parametric library
data(cps71)
attach(cps71)

m <- npreg(logwage~age)

plot(m, plot.errors.method="asymptotic",
     plot.errors.style="band",
     ylim=c(11, 15.2))

points(age, logwage, cex=.25)
detach(cps71)