Delta method

The Delta method is a theorem that can be used to derive the distribution of a function of an asymptotically normal variable.

It is often used to derive standard errors and confidence intervals for functions of parameters whose estimators are asymptotically normal.

Table of contents

The starting point: an asymptotically normal sequence
The transformation
The proposition
Example
Multivariate generalization
Solved exercises

The starting point: an asymptotically normal sequence

Let be a sequence of random variables such thatwhere:

is a normal distribution with mean and variance ;
$heta _{0}$ is a constant;
indicates convergence in distribution.

is said to be asymptotically normal, $heta _{0}$ is called the asymptotic mean of and its asymptotic variance.

For example, could be a sequence of sample means that are asymptotically normal because a Central Limit Theorem applies. Or it could be a sequence of maximum likelihood estimators satisfying a set of conditions that are sufficient for asymptotic normality.

The transformation

Now, consider the sequence where is a function.

The delta method is a method that allows us to derive, under appropriate conditions, the asymptotic distribution of from the asymptotic distribution of .

The proposition

A formal statement of the delta method is given in the following proposition.

Proposition Let be a sequence of random variables such thatLet be a continuously differentiable function. Then, [eq14]

Proof

By the mean value theorem, there exists a point lying between and $heta _{0}$ , such that [eq16] By subtracting from both sides and multiplying by , we obtain [eq18] Since converges in probability to $heta _{0}$ , and lies between and $heta _{0}$ , also converges in probability to $heta _{0}$ . Because the derivative is continuous, by the continuous mapping theorem [eq21] where denotes convergence in probability. Therefore, the first term of the product [eq23] converges in probability to a constant. By assumption, the second term converges in distribution to a normal random variable having mean and variance . As a consequence, Slutsky's theorem applies and the product converges in distribution to [eq24] By elementary rules on linear transformations of normal random variables, this has a normal distribution with mean and variance [eq25]

Example

In this example we show how the delta method can be applied.

Suppose that a sequence is asymptotically normal with asymptotic mean $heta _{0}=1$ and asymptotic variance , that is,

We want to derive the asymptotic distribution of the sequence .

The functionis continuously differentiable, so we can apply the delta method.

The asymptotic mean of the transformed sequence is

In order to compute the asymptotic variance, we need to take the first derivative of the function , which is [eq32] and evaluate it at $heta _{0}=1$ : [eq33]

Therefore, the asymptotic variance is [eq34] and we can write

Multivariate generalization

The delta method generalizes also to multivariate settings, as stated by the following proposition.

Proposition Let be a sequence of random vectors such thatwhere is a multivariate normal distribution with mean and covariance matrix , $heta _{0}$ is a constant vector, and indicates convergence in distribution. Let . If all the entries of have continuous partial derivatives with respect to , then where is the Jacobian of , i.e., the matrix of partial derivatives of the entries of with respect to the entries of .

The next example shows how the multivariate delta method can be applied.

Example Suppose that a sequence of random vectors satisfieswhere the asymptotic mean isand the asymptotic covariance matrix is [eq46] Denote the two components of by and . We want to derive the asymptotic distribution of the sequence . The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic covariance matrix, we need to compute the Jacobian of the function , which is [eq54] and evaluate it at $heta _{0}$ : [eq55] Therefore, the asymptotic variance is [eq56] and we can write [eq57]

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let be an asymptotically normal sequence with asymptotic mean $heta _{0}=0$ and asymptotic variance , that is,

Derive the asymptotic distribution of the sequence .

Solution

The functionis continuously differentiable, so we can apply the delta method. The asymptotic mean of the transformed sequence isIn order to compute the asymptotic variance, we need to take the first derivative of the function , which is [eq64] and evaluate it at $heta _{0}=0$ : [eq65] Therefore, the asymptotic variance is [eq66] and we can write

Exercise 2

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix is [eq71]

Denote the two entries of by and .

Derive the asymptotic distribution of the sequence of products

Solution

We can apply the delta method because the functionis continuously differentiable. The asymptotic mean of the transformed sequence isThe Jacobian of the function is [eq79] By evaluating it atwe obtain [eq81] Therefore, the asymptotic covariance matrix is [eq82] and we can write

Exercise 3

Let be a sequence of random vectors satisfyingwhere the asymptotic mean isand the asymptotic covariance matrix is [eq87]

Denote the two entries of by and .

Derive the asymptotic distribution of the sequence of vectors where the two entries of satisfy [eq93]

Solution

We can apply the delta method because the functions [eq94] are continuously differentiable. The asymptotic mean of the transformed sequence is a vector $eta _{0}$ whose entries are [eq95] The Jacobian of the function is [eq97] By evaluating it atwe obtain [eq99] As a consequence, the asymptotic covariance matrix is [eq100] Thus,where $eta _{0}$ and $V_{eta }$ have been calculated above.

How to cite

Please cite as:

Taboga, Marco (2021). "Delta method", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/delta-method.