Likelihood Ratio, Wald, Lagrange Multiplier Tests

The F test is applicable whenever we are testing linear restrictions in the classic normal linear regression model. However, if, (1) the restrictions are nonlinear; (2) the model is nonlinear in the parameters; (3) the errors are distributed non-normally; then we need other asymptotically equivalent tests.

Suppose the restriction being tested is written as g(β), satisfied at the value βMLE-R where the function g(β) cuts the horizontal axis (please refer to the graph at the bottom). Then we have three asymptotically equivalent tests available to do the test and make reference, all of them are distributed asymptotically as chi-square with degrees of freedom equal to the number of restrictions being tested.

(1) The Likelihood Ratio Test: if the restrictions is true, then ln(LR), the maximized value of ln(L) imposing the restrictions, should not be significantly less than ln(Lmax), then unrestricted maximum value of ln(L). The Likelihood Ratio test tests whether [ln(LR)-ln(Lmax)] is significantly different from zero.

(2) Wald Test: if the restriction g(β)=0 is true, then g(βMLE) should not be significantly different from zero. The Wald test tests whether βMLE (the unrestricted estimate of β) violates the restriction by a significant amount.

(3) Lagrange Multiplier Test: The log-likelihood function of ln(L) is maximized at point A where the slope of ln(L) with respect to β is zero. If the restriction is true, then the slope of ln(L) at point B should be significantly different from zero. The Lagrange Multiplier test tests whether the slope of ln(L), evaluated at the restricted estimate, is significantly different from zero.

Graph for reference: