Construction, Implementation, and Theory of
Algorithms Based on Data Augmentation and Model
Reduction
David A. van Dyk
Department of Statistics, The University of Chicago,
Chicago, IL 60637, U.S.A.
In the thesis we provide a
general framework for maximum likelihood algorithms based on
data augmentation and model reduction.
Starting with this theoretical
framework, we explore methods of constructing and implementing efficient
algorithms. We show how to derive faster algorithms by optimizing the
rate of
convergence as a function of a working parameter which is introduced
into the data-augmentation scheme. We then propose the Alternating
Expectation/Conditional Maximization or AECM algorithm which includes
the EM, ECM, ECME, and SAGE
algorithms as special cases.
We also show how the matrix rate of convergence can be used to compute the
asymptotic variance-covariance matrix of the maximum likelihood estimates.
The relative efficiency of competing model-reduction schemes is explored
via permutation of the conditional maximization steps within the ECM
algorithm.
Finally, we explore the inferential use of the data-augmentation
scheme in the context of estimating
the number of components in a finite mixture, with possible
extensions to other model-fitting problems.
Return to David van Dyk's homepage.