Fitting Mixed-Effects Models Using Efficient EM-type Algorithms

The Journal of Computational and Graphical Statistic

David A. van Dyk
Department of Statistics, Harvard University, Cambridge, MA 02138, U.S.A.

In recent years numerous advances in EM methodology have lead to algorithms which can be very efficient when compared with both their EM predecessors and other numerical methods (e.g., algorithms based on Newton-Raphson). In this paper we combine several of these new methods to develop a set of mode-finding algorithms for the popular mixed-effects model which are both fast and more reliable than standard algorithms such as proc mixed in SAS. We present efficient algorithms for Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), and computing posterior modes (with conjugate proper and improper priors). These algorithms are not only useful in their own right, but also illustrate how parameter expansion, conditional data augmentation, and the ECME algorithm can be used in conjunction to form efficient algorithms. In particular, we illustrate a difficulty in using the typically very efficient PXEM (parameter-expanded EM) for posterior calculations, but show how algorithms based on conditional data-augmentation can be used. Finally, we present a result that extends Hobert and Casella's (1996) result on the propriety of the posterior for the mixed-effects model under an improper prior, an important concern in Bayesian analysis involving these models that when not properly understood has lead to difficulties in several applications.

Key Words: EM algorithm, ECME algorithm, Gaussian hierarchical models, Posterior inference, PXEM algorithm, Random-effects models, REML, Variance-component models, working parameters.

Return to David van Dyk's homepage.