An Example of Algorithms Mining: Covariance Adjustment to Accelerate EM and Gibbs

The EM and Data Augmentation (or more generally, Gibbs sampling) algorithms are popular tools for parameter estimation but are often criticized for their slow convergence. In the last decade, there have been tremendous efforts to create efficient EM-type and Gibbs sampling algorithms, including interest in the recent parameter expanded (PX)-EM algorithm and its stochastic versions. With the "covariance adjustment" interpretation of PX-EM as the theme, this paper provides an overview of the work on PX-EM and PX-EM inspired Data Augmentation algorithms. The student-t distribution is used as an illustrative example.