Fast EM Implementations for Mixed-Effects Models
Xiao-Li Meng
Department of Statistics, The University of Chicago,
Chicago, IL 60637, U.S.A.
David A. van Dyk
Department of Statistics, Harvard University, Cambridge, MA
02138, U.S.A.
The mixed-effects model, in its various forms, is one of the most used
models in applied statistics. A common strategy for fitting this model
implements EM-type algorithms by treating the random effects as missing
data. Such implementations, however, can be painfully slow
when the variances of the random effects are small relative to the
residual variance. In this paper,
we apply the ``working parameter" approach discussed in Meng and van Dyk
(1997) to derive alternative EM-type implementations for fitting
mixed-effects models, which we show empirically can be hundreds of
times faster than the common EM-type implementations.
In our limited simulations, they also compare well to the routines in
S-plus and STATA in terms of both speed and reliability.
The central idea of the ``working parameter" approach adopted
in Meng and van Dyk (1997) is to search for efficient
data-augmentation schemes for implementing the EM
algorithm by minimizing the augmented information over the working
parameter, and in the mixed-effects setting this leads to a transfer of
the mixed-effects variances into the regression slope parameters.
We also describe a variation for computing REML and
an adaptive algorithm that takes
advantage of both the standard and alternative EM-type implementations.
Keywords: DATA AUGMENTATION;
INCOMPLETE DATA; MISSING DATA; RANDOM-EFFECTS MODELS;
RATE OF CONVERGENCE; REML; VARIANCE-COMPONENTS MODELS.
Return to David van Dyk's homepage.